diff --git a/README.md b/README.md new file mode 100644 index 0000000000000000000000000000000000000000..1d5830ba265bc6986c8c2560ed47a846bd6a09e4 --- /dev/null +++ b/README.md @@ -0,0 +1,147 @@ +--- +tags: +- generated_from_trainer +model-index: +- name: zephyr-7b-alpha + results: [] +license: mit +datasets: +- stingning/ultrachat +- openbmb/UltraFeedback +language: +- en +base_model: mistralai/Mistral-7B-v0.1 +--- + + + +Zephyr Logo + + +# Model Card for Zephyr 7B Alpha + +Zephyr is a series of language models that are trained to act as helpful assistants. Zephyr-7B-α is the first model in the series, and is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) that was trained on on a mix of publicly available, synthetic datasets using [Direct Preference Optimization (DPO)](https://arxiv.org/abs/2305.18290). We found that removing the in-built alignment of these datasets boosted performance on [MT Bench](https://huggingface.co/spaces/lmsys/mt-bench) and made the model more helpful. However, this means that model is likely to generate problematic text when prompted to do so. + + +## Model description + +- **Model type:** A 7B parameter GPT-like model fine-tuned on a mix of publicly available, synthetic datasets. +- **Language(s) (NLP):** Primarily English +- **License:** MIT +- **Finetuned from model:** [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) + +### Model Sources + + + +- **Repository:** https://github.com/huggingface/alignment-handbook +- **Demo:** https://huggingface.co/spaces/HuggingFaceH4/zephyr-chat + +## Intended uses & limitations + +The model was initially fine-tuned on a variant of the [`UltraChat`](https://huggingface.co/datasets/stingning/ultrachat) dataset, which contains a diverse range of synthetic dialogues generated by ChatGPT. We then further aligned the model with [🤗 TRL's](https://github.com/huggingface/trl) `DPOTrainer` on the [openbmb/UltraFeedback](https://huggingface.co/datasets/openbmb/UltraFeedback) dataset, which contain 64k prompts and model completions that are ranked by GPT-4. As a result, the model can be used for chat and you can check out our [demo](https://huggingface.co/spaces/HuggingFaceH4/zephyr-chat) to test its capabilities. + +Here's how you can run the model using the `pipeline()` function from 🤗 Transformers: + +```python +# Install transformers from source - only needed for versions <= v4.34 +# pip install git+https://github.com/huggingface/transformers.git +# pip install accelerate + +import torch +from transformers import pipeline + +pipe = pipeline("text-generation", model="HuggingFaceH4/zephyr-7b-alpha", torch_dtype=torch.bfloat16, device_map="auto") + +# We use the tokenizer's chat template to format each message - see https://huggingface.co/docs/transformers/main/en/chat_templating +messages = [ + { + "role": "system", + "content": "You are a friendly chatbot who always responds in the style of a pirate", + }, + {"role": "user", "content": "How many helicopters can a human eat in one sitting?"}, +] +prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) +outputs = pipe(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95) +print(outputs[0]["generated_text"]) +# <|system|> +# You are a friendly chatbot who always responds in the style of a pirate. +# <|user|> +# How many helicopters can a human eat in one sitting? +# <|assistant|> +# Ah, me hearty matey! But yer question be a puzzler! A human cannot eat a helicopter in one sitting, as helicopters are not edible. They be made of metal, plastic, and other materials, not food! +``` + +## Bias, Risks, and Limitations + + + +Zephyr-7B-α has not been aligned to human preferences with techniques like RLHF or deployed with in-the-loop filtering of responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so). +It is also unknown what the size and composition of the corpus was used to train the base model (`mistralai/Mistral-7B-v0.1`), however it is likely to have included a mix of Web data and technical sources like books and code. See the [Falcon 180B model card](https://huggingface.co/tiiuae/falcon-180B#training-data) for an example of this. + + +## Training and evaluation data + +Zephyr 7B Alpha achieves the following results on the evaluation set: + +- Loss: 0.4605 +- Rewards/chosen: -0.5053 +- Rewards/rejected: -1.8752 +- Rewards/accuracies: 0.7812 +- Rewards/margins: 1.3699 +- Logps/rejected: -327.4286 +- Logps/chosen: -297.1040 +- Logits/rejected: -2.7153 +- Logits/chosen: -2.7447 + +## Training procedure + +### Training hyperparameters + +The following hyperparameters were used during training: + +- learning_rate: 5e-07 +- train_batch_size: 2 +- eval_batch_size: 4 +- seed: 42 +- distributed_type: multi-GPU +- num_devices: 16 +- total_train_batch_size: 32 +- total_eval_batch_size: 64 +- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 +- lr_scheduler_type: linear +- lr_scheduler_warmup_ratio: 0.1 +- num_epochs: 1 + +### Training results + +| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen | +|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:| +| 0.5602 | 0.05 | 100 | 0.5589 | -0.3359 | -0.8168 | 0.7188 | 0.4809 | -306.2607 | -293.7161 | -2.6554 | -2.6797 | +| 0.4852 | 0.1 | 200 | 0.5136 | -0.5310 | -1.4994 | 0.8125 | 0.9684 | -319.9124 | -297.6181 | -2.5762 | -2.5957 | +| 0.5212 | 0.15 | 300 | 0.5168 | -0.1686 | -1.1760 | 0.7812 | 1.0074 | -313.4444 | -290.3699 | -2.6865 | -2.7125 | +| 0.5496 | 0.21 | 400 | 0.4835 | -0.1617 | -1.7170 | 0.8281 | 1.5552 | -324.2635 | -290.2326 | -2.7947 | -2.8218 | +| 0.5209 | 0.26 | 500 | 0.5054 | -0.4778 | -1.6604 | 0.7344 | 1.1826 | -323.1325 | -296.5546 | -2.8388 | -2.8667 | +| 0.4617 | 0.31 | 600 | 0.4910 | -0.3738 | -1.5180 | 0.7656 | 1.1442 | -320.2848 | -294.4741 | -2.8234 | -2.8521 | +| 0.4452 | 0.36 | 700 | 0.4838 | -0.4591 | -1.6576 | 0.7031 | 1.1986 | -323.0770 | -296.1796 | -2.7401 | -2.7653 | +| 0.4674 | 0.41 | 800 | 0.5077 | -0.5692 | -1.8659 | 0.7656 | 1.2967 | -327.2416 | -298.3818 | -2.6740 | -2.6945 | +| 0.4656 | 0.46 | 900 | 0.4927 | -0.5279 | -1.6614 | 0.7656 | 1.1335 | -323.1518 | -297.5553 | -2.7817 | -2.8015 | +| 0.4102 | 0.52 | 1000 | 0.4772 | -0.5767 | -2.0667 | 0.7656 | 1.4900 | -331.2578 | -298.5311 | -2.7160 | -2.7455 | +| 0.4663 | 0.57 | 1100 | 0.4740 | -0.8038 | -2.1018 | 0.7656 | 1.2980 | -331.9604 | -303.0741 | -2.6994 | -2.7257 | +| 0.4737 | 0.62 | 1200 | 0.4716 | -0.3783 | -1.7015 | 0.7969 | 1.3232 | -323.9545 | -294.5634 | -2.6842 | -2.7135 | +| 0.4259 | 0.67 | 1300 | 0.4866 | -0.6239 | -1.9703 | 0.7812 | 1.3464 | -329.3312 | -299.4761 | -2.7046 | -2.7356 | +| 0.4935 | 0.72 | 1400 | 0.4747 | -0.5626 | -1.7600 | 0.7812 | 1.1974 | -325.1243 | -298.2491 | -2.7153 | -2.7444 | +| 0.4211 | 0.77 | 1500 | 0.4645 | -0.6099 | -1.9993 | 0.7656 | 1.3894 | -329.9109 | -299.1959 | -2.6944 | -2.7236 | +| 0.4931 | 0.83 | 1600 | 0.4684 | -0.6798 | -2.1082 | 0.7656 | 1.4285 | -332.0890 | -300.5934 | -2.7006 | -2.7305 | +| 0.5029 | 0.88 | 1700 | 0.4595 | -0.5063 | -1.8951 | 0.7812 | 1.3889 | -327.8267 | -297.1233 | -2.7108 | -2.7403 | +| 0.4965 | 0.93 | 1800 | 0.4613 | -0.5561 | -1.9079 | 0.7812 | 1.3518 | -328.0831 | -298.1203 | -2.7226 | -2.7523 | +| 0.4337 | 0.98 | 1900 | 0.4608 | -0.5066 | -1.8718 | 0.7656 | 1.3652 | -327.3599 | -297.1296 | -2.7175 | -2.7469 | + + +### Framework versions + +- Transformers 4.34.0 +- Pytorch 2.0.1+cu118 +- Datasets 2.12.0 +- Tokenizers 0.14.0 \ No newline at end of file diff --git a/added_tokens.json b/added_tokens.json new file mode 100644 index 0000000000000000000000000000000000000000..cbce74e5c64b97114098962fa58454a57d7fb532 --- /dev/null +++ b/added_tokens.json @@ -0,0 +1,5 @@ +{ + "": 2, + "": 1, + "": 0 +} diff --git a/all_results.json b/all_results.json new file mode 100644 index 0000000000000000000000000000000000000000..a32a5840be00cf99ad0d5ecb2da8313326c56a73 --- /dev/null +++ b/all_results.json @@ -0,0 +1,21 @@ +{ + "epoch": 1.0, + "eval_logits/chosen": -2.744652509689331, + "eval_logits/rejected": -2.71529483795166, + "eval_logps/chosen": -297.10400390625, + "eval_logps/rejected": -327.4286193847656, + "eval_loss": 0.46045970916748047, + "eval_rewards/accuracies": 0.78125, + "eval_rewards/chosen": -0.5052940249443054, + "eval_rewards/margins": 1.3699172735214233, + "eval_rewards/rejected": -1.8752113580703735, + "eval_runtime": 52.3218, + "eval_samples": 1000, + "eval_samples_per_second": 19.113, + "eval_steps_per_second": 0.306, + "train_loss": 0.48803097129668166, + "train_runtime": 7971.1784, + "train_samples": 61966, + "train_samples_per_second": 7.774, + "train_steps_per_second": 0.243 +} \ No newline at end of file diff --git a/cal_data.safetensors b/cal_data.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..cd988e09571b1e2a570a608f858c02db1d47b325 --- /dev/null +++ b/cal_data.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:08be1103ff8fcef33b570f3c0f5ae4cc7f9dc5c3f264105baa55fc9b132ed1be +size 1638488 diff --git a/colab-demo.ipynb b/colab-demo.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..ca05601599bcac111a68107ee578e9df4add4fff --- /dev/null +++ b/colab-demo.ipynb @@ -0,0 +1 @@ +{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"provenance":[],"gpuType":"T4","authorship_tag":"ABX9TyNcXbnhVLPdImfXNrkyZZK9"},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"},"accelerator":"GPU","widgets":{"application/vnd.jupyter.widget-state+json":{"af35743bd7cc4b2a9a9ac89166372f1f":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_cc8496246b9248ae8631168804de7374","IPY_MODEL_0a9a7266561e40a991473c40721d1caa","IPY_MODEL_83bb42e8a79446cfa6bdfb76284a2f2c"],"layout":"IPY_MODEL_a3df2076447a4f5e8b777295f55c8fb4"}},"cc8496246b9248ae8631168804de7374":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_cb54c0578558402bb0b8341999449b92","placeholder":"​","style":"IPY_MODEL_a76ea86b1887442e8828d1184f548b05","value":"Downloading (…)lve/main/config.json: 100%"}},"0a9a7266561e40a991473c40721d1caa":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_9659515b9831421d9464ffee29b2b87a","max":628,"min":0,"orientation":"horizontal","style":"IPY_MODEL_a08f5e40fb1e4662aa624e6feb3eddac","value":628}},"83bb42e8a79446cfa6bdfb76284a2f2c":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_790926668e1e4c4ab390c0821529c3c1","placeholder":"​","style":"IPY_MODEL_d3b27018eaf34defa08ff8c0c09ef826","value":" 628/628 [00:00<00:00, 44.8kB/s]"}},"a3df2076447a4f5e8b777295f55c8fb4":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"cb54c0578558402bb0b8341999449b92":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a76ea86b1887442e8828d1184f548b05":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"9659515b9831421d9464ffee29b2b87a":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a08f5e40fb1e4662aa624e6feb3eddac":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"790926668e1e4c4ab390c0821529c3c1":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d3b27018eaf34defa08ff8c0c09ef826":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"2f7c528ac4a846b3bd3f5b859c5937ce":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_779303ae3a504bc39e94c05bb41d6680","IPY_MODEL_12b3162cbf9c4c3ba4f83f325341fc26","IPY_MODEL_ed0a166c01104946b3768f827f9a92da"],"layout":"IPY_MODEL_99f491f558e248abb4c7acc9deb28923"}},"779303ae3a504bc39e94c05bb41d6680":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_91eaa2a5525f4f51a111ad59dca862dd","placeholder":"​","style":"IPY_MODEL_70c24596219147189534ef2effd75212","value":"Downloading (…)fetensors.index.json: 100%"}},"12b3162cbf9c4c3ba4f83f325341fc26":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_e54d337cbae84671ba46d2fccf7241a5","max":23950,"min":0,"orientation":"horizontal","style":"IPY_MODEL_586b2f72d4694e3c88f22131326378b6","value":23950}},"ed0a166c01104946b3768f827f9a92da":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_3abcf30b8b7a49d480d80b43f7528f3b","placeholder":"​","style":"IPY_MODEL_030709c0956f400c9c61d9c5028a7b5a","value":" 23.9k/23.9k [00:00<00:00, 526kB/s]"}},"99f491f558e248abb4c7acc9deb28923":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"91eaa2a5525f4f51a111ad59dca862dd":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"70c24596219147189534ef2effd75212":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"e54d337cbae84671ba46d2fccf7241a5":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"586b2f72d4694e3c88f22131326378b6":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"3abcf30b8b7a49d480d80b43f7528f3b":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"030709c0956f400c9c61d9c5028a7b5a":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c2e47ceae1d14da59867cc5653409e7e":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_07b3f2f9ab8e479489cfa96d8ced3f4c","IPY_MODEL_5bd6352bb7bf4c2fa195c73d9b875ea7","IPY_MODEL_02eab2611e47411eb957e42321093df2"],"layout":"IPY_MODEL_4dec5009208844fc845d276f86c53ea2"}},"07b3f2f9ab8e479489cfa96d8ced3f4c":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_cdde97c62f854f8f96a86957c4e67487","placeholder":"​","style":"IPY_MODEL_1fa0cf1349ed4f779a3f61f8f5be0f1e","value":"Downloading shards: 100%"}},"5bd6352bb7bf4c2fa195c73d9b875ea7":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_f28f325a3aa24182856e5da82b565ddd","max":8,"min":0,"orientation":"horizontal","style":"IPY_MODEL_56e4af5782d54e9c8069444b34fc6b79","value":8}},"02eab2611e47411eb957e42321093df2":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_bbad101d94d64d3e8f47f2d22bf3096e","placeholder":"​","style":"IPY_MODEL_33f0fd3841ac42788e92041f2756df4a","value":" 8/8 [02:06<00:00, 13.41s/it]"}},"4dec5009208844fc845d276f86c53ea2":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"cdde97c62f854f8f96a86957c4e67487":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1fa0cf1349ed4f779a3f61f8f5be0f1e":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"f28f325a3aa24182856e5da82b565ddd":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"56e4af5782d54e9c8069444b34fc6b79":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"bbad101d94d64d3e8f47f2d22bf3096e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"33f0fd3841ac42788e92041f2756df4a":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6cf085e28dc444698da1fcb0671d5570":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_1d2644bfdfa1493bb682f9bf80e132be","IPY_MODEL_d28ec950aa0d4c5b817e54172da5fb44","IPY_MODEL_db47d15652ac4a51b85cbb0e1f49cffc"],"layout":"IPY_MODEL_6148302fcdcd4eeea8770340bcdc4af5"}},"1d2644bfdfa1493bb682f9bf80e132be":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_09c91af5582f40d1b864a3f1a6728190","placeholder":"​","style":"IPY_MODEL_a3cc8c7cdec74aa3a42ae888fd12b813","value":"Downloading (…)of-00008.safetensors: 100%"}},"d28ec950aa0d4c5b817e54172da5fb44":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_0977d1f5e8f54ee1ae13509423b6a530","max":1889587040,"min":0,"orientation":"horizontal","style":"IPY_MODEL_5695943f70204d65aac3d6a994fcac9c","value":1889587040}},"db47d15652ac4a51b85cbb0e1f49cffc":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_8a2f7e2ba8254dd8929538d88c1be72e","placeholder":"​","style":"IPY_MODEL_e5981daa2da24c299dec59442d3cd3d9","value":" 1.89G/1.89G [00:18<00:00, 26.5MB/s]"}},"6148302fcdcd4eeea8770340bcdc4af5":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"09c91af5582f40d1b864a3f1a6728190":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a3cc8c7cdec74aa3a42ae888fd12b813":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"0977d1f5e8f54ee1ae13509423b6a530":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5695943f70204d65aac3d6a994fcac9c":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"8a2f7e2ba8254dd8929538d88c1be72e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e5981daa2da24c299dec59442d3cd3d9":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"5660194f98bd47f9a872fb3ba194e722":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_287f1b8059ff461580964d6765505048","IPY_MODEL_6106c56288dc46fe812176811ede8154","IPY_MODEL_faf9c3ef160d4772b6b4d74b928f8df8"],"layout":"IPY_MODEL_ba143a50545a4c5588ffa381b3d86457"}},"287f1b8059ff461580964d6765505048":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_9d06d6845ae84d3ebfc61d1414c141ab","placeholder":"​","style":"IPY_MODEL_baae3ea716dc4c26bc1720f25b2eee60","value":"Downloading (…)of-00008.safetensors: 100%"}},"6106c56288dc46fe812176811ede8154":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_c667c1db826741f0ae7e8954071b953e","max":1946243936,"min":0,"orientation":"horizontal","style":"IPY_MODEL_89f63cd121554cbbb201cb999ac17515","value":1946243936}},"faf9c3ef160d4772b6b4d74b928f8df8":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_6e0d6d6061924a9080e46924e0a185d5","placeholder":"​","style":"IPY_MODEL_4a73e20d2402485fa864b16e0cfe40e8","value":" 1.95G/1.95G [00:19<00:00, 119MB/s]"}},"ba143a50545a4c5588ffa381b3d86457":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9d06d6845ae84d3ebfc61d1414c141ab":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"baae3ea716dc4c26bc1720f25b2eee60":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c667c1db826741f0ae7e8954071b953e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"89f63cd121554cbbb201cb999ac17515":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"6e0d6d6061924a9080e46924e0a185d5":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4a73e20d2402485fa864b16e0cfe40e8":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"ebf0e040eeee4b2896f03f4c6eb2b297":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_b235c4229aa943759638c05f7ee365ce","IPY_MODEL_111c3f38704d432b828946a63c909381","IPY_MODEL_3671af40a3944553bff108497c34d656"],"layout":"IPY_MODEL_ede441a2ff5a4970a9db369487dd91a6"}},"b235c4229aa943759638c05f7ee365ce":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_92de8ec1a92a487eb872698185359a27","placeholder":"​","style":"IPY_MODEL_56dcdf86a9ad4342ac63a1d7c19bdbbd","value":"Downloading (…)of-00008.safetensors: 100%"}},"111c3f38704d432b828946a63c909381":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_fb49971dab3749bd854b68572fea4eb9","max":1979781432,"min":0,"orientation":"horizontal","style":"IPY_MODEL_4c3b6bc4f4974714b7f0916908c1dfc3","value":1979781432}},"3671af40a3944553bff108497c34d656":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c114c204fec84c88be602aba53cefa27","placeholder":"​","style":"IPY_MODEL_b7f6e0b0e9a5408ca8ab2a2c093c18bd","value":" 1.98G/1.98G [00:15<00:00, 179MB/s]"}},"ede441a2ff5a4970a9db369487dd91a6":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"92de8ec1a92a487eb872698185359a27":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"56dcdf86a9ad4342ac63a1d7c19bdbbd":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"fb49971dab3749bd854b68572fea4eb9":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4c3b6bc4f4974714b7f0916908c1dfc3":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"c114c204fec84c88be602aba53cefa27":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b7f6e0b0e9a5408ca8ab2a2c093c18bd":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"8950db7362044b0cb7a2df3f4a14ab96":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_51681304de6943ada122e5226a137a54","IPY_MODEL_f7ab29ccdaa14f20a9236f073d46cc5b","IPY_MODEL_d5d48ba443914e77b19dae2db44a2391"],"layout":"IPY_MODEL_0c163e318223420588abe6f5983417f2"}},"51681304de6943ada122e5226a137a54":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_33acdf4834b64baaa72dbb5c355d581f","placeholder":"​","style":"IPY_MODEL_7c242694ca2e441bb23d67a1010051ba","value":"Downloading (…)of-00008.safetensors: 100%"}},"f7ab29ccdaa14f20a9236f073d46cc5b":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_f84bad5785e84fae84bdea480ecabe34","max":1946243984,"min":0,"orientation":"horizontal","style":"IPY_MODEL_6b6cc7afc2d54920950863c1eacba2d9","value":1946243984}},"d5d48ba443914e77b19dae2db44a2391":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_631b976952c44ee4ba5c3a580b554bc3","placeholder":"​","style":"IPY_MODEL_40a2575e33c249d8983c96ab13ba1c87","value":" 1.95G/1.95G [00:14<00:00, 163MB/s]"}},"0c163e318223420588abe6f5983417f2":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"33acdf4834b64baaa72dbb5c355d581f":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7c242694ca2e441bb23d67a1010051ba":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"f84bad5785e84fae84bdea480ecabe34":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6b6cc7afc2d54920950863c1eacba2d9":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"631b976952c44ee4ba5c3a580b554bc3":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"40a2575e33c249d8983c96ab13ba1c87":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"64465e89b5084470a1300c230e4d6a0f":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_66f556e5045543d282cf6ec1b24e6a5d","IPY_MODEL_557fc212f965425d9cad706e31b5de71","IPY_MODEL_a3e402a256e1457caed5c73972707135"],"layout":"IPY_MODEL_ac237b12560c4d2a869ff7c4997034f6"}},"66f556e5045543d282cf6ec1b24e6a5d":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_9c477bace9474894a4af4a49ffb1b597","placeholder":"​","style":"IPY_MODEL_1e94342bd4f34e16bba5277a1e287ea0","value":"Downloading (…)of-00008.safetensors: 100%"}},"557fc212f965425d9cad706e31b5de71":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_ee12e753e84b40f8b15a386726eeac74","max":1979781448,"min":0,"orientation":"horizontal","style":"IPY_MODEL_67759890afa7429bad93af68f98bca8c","value":1979781448}},"a3e402a256e1457caed5c73972707135":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_186d3c5f4a2a4ab3bb1ce46bdf4eaaa0","placeholder":"​","style":"IPY_MODEL_e00ff0abba05463495cf89b4206de6a6","value":" 1.98G/1.98G [00:18<00:00, 159MB/s]"}},"ac237b12560c4d2a869ff7c4997034f6":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9c477bace9474894a4af4a49ffb1b597":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1e94342bd4f34e16bba5277a1e287ea0":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"ee12e753e84b40f8b15a386726eeac74":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"67759890afa7429bad93af68f98bca8c":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"186d3c5f4a2a4ab3bb1ce46bdf4eaaa0":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e00ff0abba05463495cf89b4206de6a6":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"f662cc7314f04fd9b87b90dca6fe2948":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_f382b51901ba4384bc57bdb041c33fe8","IPY_MODEL_e2976b933e154328b6017b96e5a89f1b","IPY_MODEL_41a77c99fbb34fdd8e8d81cec7bd1e52"],"layout":"IPY_MODEL_60af0234884e4ba4bb3ca55357404002"}},"f382b51901ba4384bc57bdb041c33fe8":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_5db617b2eac744cab52546adb89f2b2d","placeholder":"​","style":"IPY_MODEL_66bf1b86b6814d04bab8e602b4c6ab04","value":"Downloading (…)of-00008.safetensors: 100%"}},"e2976b933e154328b6017b96e5a89f1b":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_04881b0a754648a3b7b5d666c7140d3c","max":1946243984,"min":0,"orientation":"horizontal","style":"IPY_MODEL_2f6c0d3758834b5784101e9dbb76d8b6","value":1946243984}},"41a77c99fbb34fdd8e8d81cec7bd1e52":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_f703fe03badd4fd89769838d429633f6","placeholder":"​","style":"IPY_MODEL_f5cffec9621847f3985a6d9309462c9f","value":" 1.95G/1.95G [00:15<00:00, 144MB/s]"}},"60af0234884e4ba4bb3ca55357404002":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5db617b2eac744cab52546adb89f2b2d":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"66bf1b86b6814d04bab8e602b4c6ab04":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"04881b0a754648a3b7b5d666c7140d3c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2f6c0d3758834b5784101e9dbb76d8b6":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"f703fe03badd4fd89769838d429633f6":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f5cffec9621847f3985a6d9309462c9f":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"3dd92a9f64d042f8a14d70a81b6b5041":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_6fc843a6157f424f8f48ded39a0b6174","IPY_MODEL_b9ad1baa021f471cbde376f215bb9553","IPY_MODEL_8f70605a96e54bbab6b1700917721305"],"layout":"IPY_MODEL_6be8aafb49574ea2b440d447cf068cc6"}},"6fc843a6157f424f8f48ded39a0b6174":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_d23ef9ab6fe34a67bae69675d6092b21","placeholder":"​","style":"IPY_MODEL_2a581a92ccc24fb88091fe29ebb45dda","value":"Downloading (…)of-00008.safetensors: 100%"}},"b9ad1baa021f471cbde376f215bb9553":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_85073d535e8d4c698536ed2e4ccb49f0","max":1979781448,"min":0,"orientation":"horizontal","style":"IPY_MODEL_ac447978846b452abd3eadfa71d0acf5","value":1979781448}},"8f70605a96e54bbab6b1700917721305":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_30c37aa38bab4a6cb2cae9f7cc86c9bf","placeholder":"​","style":"IPY_MODEL_2fd276b0f31449ca8cba6c8f641187e7","value":" 1.98G/1.98G [00:16<00:00, 153MB/s]"}},"6be8aafb49574ea2b440d447cf068cc6":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d23ef9ab6fe34a67bae69675d6092b21":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2a581a92ccc24fb88091fe29ebb45dda":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"85073d535e8d4c698536ed2e4ccb49f0":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ac447978846b452abd3eadfa71d0acf5":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"30c37aa38bab4a6cb2cae9f7cc86c9bf":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2fd276b0f31449ca8cba6c8f641187e7":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"aedc6763d2024505a03eaa3057d530f7":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_4fe5bede20774d0682c76c551e3fe4cf","IPY_MODEL_07d8f821c0654d068305616cc7e65169","IPY_MODEL_07e3343d7bb24d22ab54362f414c3f87"],"layout":"IPY_MODEL_e8c41663e554493e995d488b56acee6d"}},"4fe5bede20774d0682c76c551e3fe4cf":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_d2329f178afc4cac8314c5b120cd694e","placeholder":"​","style":"IPY_MODEL_af362e7bd14345b2bab2d354de07a237","value":"Downloading (…)of-00008.safetensors: 100%"}},"07d8f821c0654d068305616cc7e65169":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_2d6a220aadf14400aefff2c3e63118f0","max":815834680,"min":0,"orientation":"horizontal","style":"IPY_MODEL_2fabde383f33481e87e46a3ca6462c6a","value":815834680}},"07e3343d7bb24d22ab54362f414c3f87":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_47c679a4a0774d9192ada75fb82ca5bd","placeholder":"​","style":"IPY_MODEL_0f49cbe7bd0042a58c776be62e303f48","value":" 816M/816M [00:06<00:00, 136MB/s]"}},"e8c41663e554493e995d488b56acee6d":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d2329f178afc4cac8314c5b120cd694e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"af362e7bd14345b2bab2d354de07a237":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"2d6a220aadf14400aefff2c3e63118f0":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2fabde383f33481e87e46a3ca6462c6a":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"47c679a4a0774d9192ada75fb82ca5bd":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"0f49cbe7bd0042a58c776be62e303f48":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"9f107fa9b45a403981e13e86ffadac33":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_ff835a6f8818434d9546e968621a40ac","IPY_MODEL_d95ca54538494e9c94d7baf831b8100f","IPY_MODEL_30f52c9064114bd29689c33374f35981"],"layout":"IPY_MODEL_59d4cb163ff5411d8fbf08d2ac7d6841"}},"ff835a6f8818434d9546e968621a40ac":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_ea027cd7d08643a49f2e7a9e6ad02c2c","placeholder":"​","style":"IPY_MODEL_312daf1f2dee4b60a56d3c4c0373faec","value":"Loading checkpoint shards: 100%"}},"d95ca54538494e9c94d7baf831b8100f":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_a2a98e37c0054522b87236c3e4ad77e7","max":8,"min":0,"orientation":"horizontal","style":"IPY_MODEL_1bbd28564a09472d9c2e56ef538b5cfc","value":8}},"30f52c9064114bd29689c33374f35981":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_4a7d22479bee4e9da3fb14d3bdc1a9fe","placeholder":"​","style":"IPY_MODEL_2aad5a8f58724b399bcadc2974270e87","value":" 8/8 [00:54<00:00, 4.88s/it]"}},"59d4cb163ff5411d8fbf08d2ac7d6841":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ea027cd7d08643a49f2e7a9e6ad02c2c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"312daf1f2dee4b60a56d3c4c0373faec":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"a2a98e37c0054522b87236c3e4ad77e7":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1bbd28564a09472d9c2e56ef538b5cfc":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"4a7d22479bee4e9da3fb14d3bdc1a9fe":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2aad5a8f58724b399bcadc2974270e87":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6d0e1131adaa452e8e7e3552236c698a":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_9c69554d6cc4495aa5cc120ff5f0c1ef","IPY_MODEL_a06a11658ec34c779b92d1b545564bf8","IPY_MODEL_567b56683f2645539ade7409b0e2f8c2"],"layout":"IPY_MODEL_47fb621ad3e3441d8bda17e151f330bf"}},"9c69554d6cc4495aa5cc120ff5f0c1ef":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_4b5a4dc22f00478e88f65e97a90f7e16","placeholder":"​","style":"IPY_MODEL_57be6290c12d4d7bb24de94802a69675","value":"Downloading (…)neration_config.json: 100%"}},"a06a11658ec34c779b92d1b545564bf8":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_b29459d2d8354e4c9f71a9f1e5a90b31","max":111,"min":0,"orientation":"horizontal","style":"IPY_MODEL_70cda0980f0f4044b2dfb60ae81c0be0","value":111}},"567b56683f2645539ade7409b0e2f8c2":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_78c04458e3ff4b9dac0c8bace9f3a6e8","placeholder":"​","style":"IPY_MODEL_16d1fb7e26d7425680b960b5f48e6f24","value":" 111/111 [00:00<00:00, 1.84kB/s]"}},"47fb621ad3e3441d8bda17e151f330bf":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4b5a4dc22f00478e88f65e97a90f7e16":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"57be6290c12d4d7bb24de94802a69675":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"b29459d2d8354e4c9f71a9f1e5a90b31":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"70cda0980f0f4044b2dfb60ae81c0be0":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"78c04458e3ff4b9dac0c8bace9f3a6e8":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"16d1fb7e26d7425680b960b5f48e6f24":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c9d1aa1244f94da7a8bd51c6747979a6":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_04c9f4717ee34413be78241c08fca52d","IPY_MODEL_c7be18a542a341d488fe0e6787e0070a","IPY_MODEL_d40cd28139b24ab88a0f10fe8f397c61"],"layout":"IPY_MODEL_5444ab3413374fc8850e172d8991aeaf"}},"04c9f4717ee34413be78241c08fca52d":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_fe0319b1c543432b915c805cf8abaa73","placeholder":"​","style":"IPY_MODEL_e53daa21f9ca4819970abb352a4b5fe5","value":"Downloading (…)okenizer_config.json: 100%"}},"c7be18a542a341d488fe0e6787e0070a":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_8700fbc216c2431583f30eac303f3113","max":1431,"min":0,"orientation":"horizontal","style":"IPY_MODEL_90cf63fb976040ee80d44172dcc6b313","value":1431}},"d40cd28139b24ab88a0f10fe8f397c61":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_ad75e13a28704a4982e7dc48ee7dfbb7","placeholder":"​","style":"IPY_MODEL_6005fcdc7a354bb0bb86cddd8efc1b9e","value":" 1.43k/1.43k [00:00<00:00, 80.5kB/s]"}},"5444ab3413374fc8850e172d8991aeaf":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"fe0319b1c543432b915c805cf8abaa73":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e53daa21f9ca4819970abb352a4b5fe5":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"8700fbc216c2431583f30eac303f3113":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"90cf63fb976040ee80d44172dcc6b313":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"ad75e13a28704a4982e7dc48ee7dfbb7":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6005fcdc7a354bb0bb86cddd8efc1b9e":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"542495f1591d4d94b2d05e8d7d1f50fd":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_0012dac6fbec478a8e377a1652b937b8","IPY_MODEL_a2d02ed45bde4d8b8901b824eb488e2b","IPY_MODEL_eff08365f8e24b3083a7981ecc01c32a"],"layout":"IPY_MODEL_9c77d9fb38dd4b16b83e250159c294ee"}},"0012dac6fbec478a8e377a1652b937b8":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_786de9f41cda4cb38c080ae90b907794","placeholder":"​","style":"IPY_MODEL_15f6b889c1d244b19239aed51bd7a06b","value":"Downloading tokenizer.model: 100%"}},"a2d02ed45bde4d8b8901b824eb488e2b":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_4a09188283a845ba980212eb22e27d7b","max":493443,"min":0,"orientation":"horizontal","style":"IPY_MODEL_179ffe54121a4f648da0ba61e25401d0","value":493443}},"eff08365f8e24b3083a7981ecc01c32a":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_9019d095a2e047ebbaad7ddddc742d39","placeholder":"​","style":"IPY_MODEL_504447a3a3cf465cbbd6d35d2b02d8de","value":" 493k/493k [00:00<00:00, 30.7MB/s]"}},"9c77d9fb38dd4b16b83e250159c294ee":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"786de9f41cda4cb38c080ae90b907794":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"15f6b889c1d244b19239aed51bd7a06b":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"4a09188283a845ba980212eb22e27d7b":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"179ffe54121a4f648da0ba61e25401d0":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"9019d095a2e047ebbaad7ddddc742d39":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"504447a3a3cf465cbbd6d35d2b02d8de":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"169493fa776c4f32858d8ada28a9902e":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_012ba73ad3d7422e88f96dd7022ebcfe","IPY_MODEL_a8cebc6a039043b1a15672a65d2bc74c","IPY_MODEL_7e779d10cbb84fb6a3e154d593f5fd16"],"layout":"IPY_MODEL_c35f30e35de94ac2ae87fe1470613c68"}},"012ba73ad3d7422e88f96dd7022ebcfe":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_48a36dbdec3248fab8c9d915d9e53921","placeholder":"​","style":"IPY_MODEL_83a8691b9c734409bf7caaf4ed21066a","value":"Downloading (…)/main/tokenizer.json: 100%"}},"a8cebc6a039043b1a15672a65d2bc74c":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_bfeb85b8bc944efb8afacad98676f8c2","max":1795303,"min":0,"orientation":"horizontal","style":"IPY_MODEL_72f9b6a11ea84512b326bced98e6e810","value":1795303}},"7e779d10cbb84fb6a3e154d593f5fd16":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c26a277a40c04c099a830e0a346e8702","placeholder":"​","style":"IPY_MODEL_845d26f25b9d4b0d945954d91c985a8a","value":" 1.80M/1.80M [00:00<00:00, 5.44MB/s]"}},"c35f30e35de94ac2ae87fe1470613c68":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"48a36dbdec3248fab8c9d915d9e53921":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"83a8691b9c734409bf7caaf4ed21066a":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"bfeb85b8bc944efb8afacad98676f8c2":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"72f9b6a11ea84512b326bced98e6e810":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"c26a277a40c04c099a830e0a346e8702":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"845d26f25b9d4b0d945954d91c985a8a":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"1c5cd44fd3104f6fa612e10ac432826d":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_caa327b8d84945eca6f4d733e617f3cc","IPY_MODEL_f6fa5f9808f1445aab7e726206749766","IPY_MODEL_7c4c81b536f44dc9914df6428c40ad1a"],"layout":"IPY_MODEL_e4aa69c959454612b19694f8f270d58c"}},"caa327b8d84945eca6f4d733e617f3cc":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_bc1a505eae3e42979cdbd30e87956445","placeholder":"​","style":"IPY_MODEL_117c75b5f11348c1b8758fe4abacf503","value":"Downloading (…)in/added_tokens.json: 100%"}},"f6fa5f9808f1445aab7e726206749766":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_6502f05831104951b599c11867172ffc","max":42,"min":0,"orientation":"horizontal","style":"IPY_MODEL_6614c74c61b144bc91cd450e72ee93ff","value":42}},"7c4c81b536f44dc9914df6428c40ad1a":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_3037b05a942a464588ee7e54fa042334","placeholder":"​","style":"IPY_MODEL_f65e3ce99b924989a8082a03b6cc2ef2","value":" 42.0/42.0 [00:00<00:00, 1.36kB/s]"}},"e4aa69c959454612b19694f8f270d58c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"bc1a505eae3e42979cdbd30e87956445":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"117c75b5f11348c1b8758fe4abacf503":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6502f05831104951b599c11867172ffc":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6614c74c61b144bc91cd450e72ee93ff":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"3037b05a942a464588ee7e54fa042334":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f65e3ce99b924989a8082a03b6cc2ef2":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"0d01fc2a5894448ab26771ce4f4ed0fc":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_3792c555a4434615992afe39b5a168be","IPY_MODEL_7b25e155197f41ecba8d3018f56393a6","IPY_MODEL_ccb3b9ae2f0a4c4b957e2e03da01f1db"],"layout":"IPY_MODEL_397bb2e89c4e44e68467b24c342c7bf1"}},"3792c555a4434615992afe39b5a168be":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_1a15afd087b3413fbfc22b244c19d5ff","placeholder":"​","style":"IPY_MODEL_f637d7f94ccc4ee18cac731bb337a92f","value":"Downloading (…)cial_tokens_map.json: 100%"}},"7b25e155197f41ecba8d3018f56393a6":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_881c7035aa0f4db5b690a7c9623c6e8c","max":168,"min":0,"orientation":"horizontal","style":"IPY_MODEL_5aff0a8982d247979886ae7951c9b224","value":168}},"ccb3b9ae2f0a4c4b957e2e03da01f1db":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_4b050fc85e0c405581ed55e850b183ab","placeholder":"​","style":"IPY_MODEL_297c892dd4bf4886a62bcf4c6b4e3997","value":" 168/168 [00:00<00:00, 7.95kB/s]"}},"397bb2e89c4e44e68467b24c342c7bf1":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1a15afd087b3413fbfc22b244c19d5ff":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f637d7f94ccc4ee18cac731bb337a92f":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"881c7035aa0f4db5b690a7c9623c6e8c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5aff0a8982d247979886ae7951c9b224":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"4b050fc85e0c405581ed55e850b183ab":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"297c892dd4bf4886a62bcf4c6b4e3997":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}}}}},"cells":[{"cell_type":"markdown","source":["# Quick start to run Zephyr 7B Alpha on Google Colab"],"metadata":{"id":"oSK6YKSx6H7m"}},{"cell_type":"code","source":["# Install transformers from source - only needed for versions <= v4.34\n","%pip install git+https://github.com/huggingface/transformers.git\n","%pip install accelerate"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"BLsaLpLP6QSK","executionInfo":{"status":"ok","timestamp":1697467249232,"user_tz":-120,"elapsed":45974,"user":{"displayName":"Lewis Tunstall","userId":"11991259263375974384"}},"outputId":"1b15ee88-5414-485f-9264-109e3d92d267"},"execution_count":1,"outputs":[{"output_type":"stream","name":"stdout","text":["Collecting git+https://github.com/huggingface/transformers.git\n"," Cloning https://github.com/huggingface/transformers.git to /tmp/pip-req-build-1as3weby\n"," Running command git clone --filter=blob:none --quiet https://github.com/huggingface/transformers.git /tmp/pip-req-build-1as3weby\n"," Resolved https://github.com/huggingface/transformers.git to commit 12cc1233591655528b3f8179c83a806de73fba3e\n"," Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n"," Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n"," Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n","Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers==4.35.0.dev0) (3.12.4)\n","Collecting huggingface-hub<1.0,>=0.16.4 (from transformers==4.35.0.dev0)\n"," Downloading huggingface_hub-0.18.0-py3-none-any.whl (301 kB)\n","\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m302.0/302.0 kB\u001b[0m \u001b[31m5.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.35.0.dev0) (1.23.5)\n","Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers==4.35.0.dev0) (23.2)\n","Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.35.0.dev0) (6.0.1)\n","Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.35.0.dev0) (2023.6.3)\n","Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers==4.35.0.dev0) (2.31.0)\n","Collecting tokenizers<0.15,>=0.14 (from transformers==4.35.0.dev0)\n"," Downloading tokenizers-0.14.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.8 MB)\n","\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.8/3.8 MB\u001b[0m \u001b[31m19.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hCollecting safetensors>=0.3.1 (from transformers==4.35.0.dev0)\n"," Downloading safetensors-0.4.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n","\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m31.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers==4.35.0.dev0) (4.66.1)\n","Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.16.4->transformers==4.35.0.dev0) (2023.6.0)\n","Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.16.4->transformers==4.35.0.dev0) (4.5.0)\n","Collecting huggingface-hub<1.0,>=0.16.4 (from transformers==4.35.0.dev0)\n"," Downloading huggingface_hub-0.17.3-py3-none-any.whl (295 kB)\n","\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m295.0/295.0 kB\u001b[0m \u001b[31m35.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.35.0.dev0) (3.3.0)\n","Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.35.0.dev0) (3.4)\n","Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.35.0.dev0) (2.0.6)\n","Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.35.0.dev0) (2023.7.22)\n","Building wheels for collected packages: transformers\n"," Building wheel for transformers (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n"," Created wheel for transformers: filename=transformers-4.35.0.dev0-py3-none-any.whl size=7791033 sha256=443f1f957f04ac3efd24e0c6de6e5b07965a608ed56ae0da9b92b045af74a1c4\n"," Stored in directory: /tmp/pip-ephem-wheel-cache-5k_mcupq/wheels/e7/9c/5b/e1a9c8007c343041e61cc484433d512ea9274272e3fcbe7c16\n","Successfully built transformers\n","Installing collected packages: safetensors, huggingface-hub, tokenizers, transformers\n","Successfully installed huggingface-hub-0.17.3 safetensors-0.4.0 tokenizers-0.14.1 transformers-4.35.0.dev0\n","Collecting accelerate\n"," Downloading accelerate-0.23.0-py3-none-any.whl (258 kB)\n","\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m258.1/258.1 kB\u001b[0m \u001b[31m4.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from accelerate) (1.23.5)\n","Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from accelerate) (23.2)\n","Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate) (5.9.5)\n","Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from accelerate) (6.0.1)\n","Requirement already satisfied: torch>=1.10.0 in /usr/local/lib/python3.10/dist-packages (from accelerate) (2.0.1+cu118)\n","Requirement already satisfied: huggingface-hub in /usr/local/lib/python3.10/dist-packages (from accelerate) (0.17.3)\n","Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (3.12.4)\n","Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (4.5.0)\n","Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (1.12)\n","Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (3.1)\n","Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (3.1.2)\n","Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (2.0.0)\n","Requirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.10.0->accelerate) (3.27.6)\n","Requirement already satisfied: lit in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.10.0->accelerate) (17.0.2)\n","Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->accelerate) (2023.6.0)\n","Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->accelerate) (2.31.0)\n","Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->accelerate) (4.66.1)\n","Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=1.10.0->accelerate) (2.1.3)\n","Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (3.3.0)\n","Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (3.4)\n","Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (2.0.6)\n","Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (2023.7.22)\n","Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.10.0->accelerate) (1.3.0)\n","Installing collected packages: accelerate\n","Successfully installed accelerate-0.23.0\n"]}]},{"cell_type":"markdown","source":["## Load dependencies"],"metadata":{"id":"YK6Ux0Xy7D4o"}},{"cell_type":"code","source":["import torch\n","from transformers import pipeline"],"metadata":{"id":"fLjPZ5ke6q8Y","executionInfo":{"status":"ok","timestamp":1697467265528,"user_tz":-120,"elapsed":15540,"user":{"displayName":"Lewis Tunstall","userId":"11991259263375974384"}}},"execution_count":2,"outputs":[]},{"cell_type":"markdown","source":["## Download model and load pipeline"],"metadata":{"id":"e1mBWABw7MAQ"}},{"cell_type":"code","source":["pipe = pipeline(\"text-generation\", model=\"HuggingFaceH4/zephyr-7b-alpha\", torch_dtype=torch.bfloat16, device_map=\"auto\")"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":610,"referenced_widgets":["af35743bd7cc4b2a9a9ac89166372f1f","cc8496246b9248ae8631168804de7374","0a9a7266561e40a991473c40721d1caa","83bb42e8a79446cfa6bdfb76284a2f2c","a3df2076447a4f5e8b777295f55c8fb4","cb54c0578558402bb0b8341999449b92","a76ea86b1887442e8828d1184f548b05","9659515b9831421d9464ffee29b2b87a","a08f5e40fb1e4662aa624e6feb3eddac","790926668e1e4c4ab390c0821529c3c1","d3b27018eaf34defa08ff8c0c09ef826","2f7c528ac4a846b3bd3f5b859c5937ce","779303ae3a504bc39e94c05bb41d6680","12b3162cbf9c4c3ba4f83f325341fc26","ed0a166c01104946b3768f827f9a92da","99f491f558e248abb4c7acc9deb28923","91eaa2a5525f4f51a111ad59dca862dd","70c24596219147189534ef2effd75212","e54d337cbae84671ba46d2fccf7241a5","586b2f72d4694e3c88f22131326378b6","3abcf30b8b7a49d480d80b43f7528f3b","030709c0956f400c9c61d9c5028a7b5a","c2e47ceae1d14da59867cc5653409e7e","07b3f2f9ab8e479489cfa96d8ced3f4c","5bd6352bb7bf4c2fa195c73d9b875ea7","02eab2611e47411eb957e42321093df2","4dec5009208844fc845d276f86c53ea2","cdde97c62f854f8f96a86957c4e67487","1fa0cf1349ed4f779a3f61f8f5be0f1e","f28f325a3aa24182856e5da82b565ddd","56e4af5782d54e9c8069444b34fc6b79","bbad101d94d64d3e8f47f2d22bf3096e","33f0fd3841ac42788e92041f2756df4a","6cf085e28dc444698da1fcb0671d5570","1d2644bfdfa1493bb682f9bf80e132be","d28ec950aa0d4c5b817e54172da5fb44","db47d15652ac4a51b85cbb0e1f49cffc","6148302fcdcd4eeea8770340bcdc4af5","09c91af5582f40d1b864a3f1a6728190","a3cc8c7cdec74aa3a42ae888fd12b813","0977d1f5e8f54ee1ae13509423b6a530","5695943f70204d65aac3d6a994fcac9c","8a2f7e2ba8254dd8929538d88c1be72e","e5981daa2da24c299dec59442d3cd3d9","5660194f98bd47f9a872fb3ba194e722","287f1b8059ff461580964d6765505048","6106c56288dc46fe812176811ede8154","faf9c3ef160d4772b6b4d74b928f8df8","ba143a50545a4c5588ffa381b3d86457","9d06d6845ae84d3ebfc61d1414c141ab","baae3ea716dc4c26bc1720f25b2eee60","c667c1db826741f0ae7e8954071b953e","89f63cd121554cbbb201cb999ac17515","6e0d6d6061924a9080e46924e0a185d5","4a73e20d2402485fa864b16e0cfe40e8","ebf0e040eeee4b2896f03f4c6eb2b297","b235c4229aa943759638c05f7ee365ce","111c3f38704d432b828946a63c909381","3671af40a3944553bff108497c34d656","ede441a2ff5a4970a9db369487dd91a6","92de8ec1a92a487eb872698185359a27","56dcdf86a9ad4342ac63a1d7c19bdbbd","fb49971dab3749bd854b68572fea4eb9","4c3b6bc4f4974714b7f0916908c1dfc3","c114c204fec84c88be602aba53cefa27","b7f6e0b0e9a5408ca8ab2a2c093c18bd","8950db7362044b0cb7a2df3f4a14ab96","51681304de6943ada122e5226a137a54","f7ab29ccdaa14f20a9236f073d46cc5b","d5d48ba443914e77b19dae2db44a2391","0c163e318223420588abe6f5983417f2","33acdf4834b64baaa72dbb5c355d581f","7c242694ca2e441bb23d67a1010051ba","f84bad5785e84fae84bdea480ecabe34","6b6cc7afc2d54920950863c1eacba2d9","631b976952c44ee4ba5c3a580b554bc3","40a2575e33c249d8983c96ab13ba1c87","64465e89b5084470a1300c230e4d6a0f","66f556e5045543d282cf6ec1b24e6a5d","557fc212f965425d9cad706e31b5de71","a3e402a256e1457caed5c73972707135","ac237b12560c4d2a869ff7c4997034f6","9c477bace9474894a4af4a49ffb1b597","1e94342bd4f34e16bba5277a1e287ea0","ee12e753e84b40f8b15a386726eeac74","67759890afa7429bad93af68f98bca8c","186d3c5f4a2a4ab3bb1ce46bdf4eaaa0","e00ff0abba05463495cf89b4206de6a6","f662cc7314f04fd9b87b90dca6fe2948","f382b51901ba4384bc57bdb041c33fe8","e2976b933e154328b6017b96e5a89f1b","41a77c99fbb34fdd8e8d81cec7bd1e52","60af0234884e4ba4bb3ca55357404002","5db617b2eac744cab52546adb89f2b2d","66bf1b86b6814d04bab8e602b4c6ab04","04881b0a754648a3b7b5d666c7140d3c","2f6c0d3758834b5784101e9dbb76d8b6","f703fe03badd4fd89769838d429633f6","f5cffec9621847f3985a6d9309462c9f","3dd92a9f64d042f8a14d70a81b6b5041","6fc843a6157f424f8f48ded39a0b6174","b9ad1baa021f471cbde376f215bb9553","8f70605a96e54bbab6b1700917721305","6be8aafb49574ea2b440d447cf068cc6","d23ef9ab6fe34a67bae69675d6092b21","2a581a92ccc24fb88091fe29ebb45dda","85073d535e8d4c698536ed2e4ccb49f0","ac447978846b452abd3eadfa71d0acf5","30c37aa38bab4a6cb2cae9f7cc86c9bf","2fd276b0f31449ca8cba6c8f641187e7","aedc6763d2024505a03eaa3057d530f7","4fe5bede20774d0682c76c551e3fe4cf","07d8f821c0654d068305616cc7e65169","07e3343d7bb24d22ab54362f414c3f87","e8c41663e554493e995d488b56acee6d","d2329f178afc4cac8314c5b120cd694e","af362e7bd14345b2bab2d354de07a237","2d6a220aadf14400aefff2c3e63118f0","2fabde383f33481e87e46a3ca6462c6a","47c679a4a0774d9192ada75fb82ca5bd","0f49cbe7bd0042a58c776be62e303f48","9f107fa9b45a403981e13e86ffadac33","ff835a6f8818434d9546e968621a40ac","d95ca54538494e9c94d7baf831b8100f","30f52c9064114bd29689c33374f35981","59d4cb163ff5411d8fbf08d2ac7d6841","ea027cd7d08643a49f2e7a9e6ad02c2c","312daf1f2dee4b60a56d3c4c0373faec","a2a98e37c0054522b87236c3e4ad77e7","1bbd28564a09472d9c2e56ef538b5cfc","4a7d22479bee4e9da3fb14d3bdc1a9fe","2aad5a8f58724b399bcadc2974270e87","6d0e1131adaa452e8e7e3552236c698a","9c69554d6cc4495aa5cc120ff5f0c1ef","a06a11658ec34c779b92d1b545564bf8","567b56683f2645539ade7409b0e2f8c2","47fb621ad3e3441d8bda17e151f330bf","4b5a4dc22f00478e88f65e97a90f7e16","57be6290c12d4d7bb24de94802a69675","b29459d2d8354e4c9f71a9f1e5a90b31","70cda0980f0f4044b2dfb60ae81c0be0","78c04458e3ff4b9dac0c8bace9f3a6e8","16d1fb7e26d7425680b960b5f48e6f24","c9d1aa1244f94da7a8bd51c6747979a6","04c9f4717ee34413be78241c08fca52d","c7be18a542a341d488fe0e6787e0070a","d40cd28139b24ab88a0f10fe8f397c61","5444ab3413374fc8850e172d8991aeaf","fe0319b1c543432b915c805cf8abaa73","e53daa21f9ca4819970abb352a4b5fe5","8700fbc216c2431583f30eac303f3113","90cf63fb976040ee80d44172dcc6b313","ad75e13a28704a4982e7dc48ee7dfbb7","6005fcdc7a354bb0bb86cddd8efc1b9e","542495f1591d4d94b2d05e8d7d1f50fd","0012dac6fbec478a8e377a1652b937b8","a2d02ed45bde4d8b8901b824eb488e2b","eff08365f8e24b3083a7981ecc01c32a","9c77d9fb38dd4b16b83e250159c294ee","786de9f41cda4cb38c080ae90b907794","15f6b889c1d244b19239aed51bd7a06b","4a09188283a845ba980212eb22e27d7b","179ffe54121a4f648da0ba61e25401d0","9019d095a2e047ebbaad7ddddc742d39","504447a3a3cf465cbbd6d35d2b02d8de","169493fa776c4f32858d8ada28a9902e","012ba73ad3d7422e88f96dd7022ebcfe","a8cebc6a039043b1a15672a65d2bc74c","7e779d10cbb84fb6a3e154d593f5fd16","c35f30e35de94ac2ae87fe1470613c68","48a36dbdec3248fab8c9d915d9e53921","83a8691b9c734409bf7caaf4ed21066a","bfeb85b8bc944efb8afacad98676f8c2","72f9b6a11ea84512b326bced98e6e810","c26a277a40c04c099a830e0a346e8702","845d26f25b9d4b0d945954d91c985a8a","1c5cd44fd3104f6fa612e10ac432826d","caa327b8d84945eca6f4d733e617f3cc","f6fa5f9808f1445aab7e726206749766","7c4c81b536f44dc9914df6428c40ad1a","e4aa69c959454612b19694f8f270d58c","bc1a505eae3e42979cdbd30e87956445","117c75b5f11348c1b8758fe4abacf503","6502f05831104951b599c11867172ffc","6614c74c61b144bc91cd450e72ee93ff","3037b05a942a464588ee7e54fa042334","f65e3ce99b924989a8082a03b6cc2ef2","0d01fc2a5894448ab26771ce4f4ed0fc","3792c555a4434615992afe39b5a168be","7b25e155197f41ecba8d3018f56393a6","ccb3b9ae2f0a4c4b957e2e03da01f1db","397bb2e89c4e44e68467b24c342c7bf1","1a15afd087b3413fbfc22b244c19d5ff","f637d7f94ccc4ee18cac731bb337a92f","881c7035aa0f4db5b690a7c9623c6e8c","5aff0a8982d247979886ae7951c9b224","4b050fc85e0c405581ed55e850b183ab","297c892dd4bf4886a62bcf4c6b4e3997"]},"id":"XMeitcMd7G-a","executionInfo":{"status":"ok","timestamp":1697467460146,"user_tz":-120,"elapsed":194632,"user":{"displayName":"Lewis Tunstall","userId":"11991259263375974384"}},"outputId":"c71db626-b8b0-471a-84a1-6e3c6e0e6179"},"execution_count":3,"outputs":[{"output_type":"display_data","data":{"text/plain":["Downloading (…)lve/main/config.json: 0%| | 0.00/628 [00:00\n","You are a friendly chatbot who always responds in the style of a pirate\n","<|user|>\n","How many helicopters can a human eat in one sitting?\n","<|assistant|>\n","\n"]}]},{"cell_type":"markdown","source":["## Generate!"],"metadata":{"id":"20K8tYJo7_7b"}},{"cell_type":"code","source":["outputs = pipe(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)\n","print(outputs[0][\"generated_text\"])"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"xc8RZR0b8Br2","executionInfo":{"status":"ok","timestamp":1697467523847,"user_tz":-120,"elapsed":47326,"user":{"displayName":"Lewis Tunstall","userId":"11991259263375974384"}},"outputId":"55bb1c98-345f-4d6b-9a31-a5b1560ab3d2"},"execution_count":5,"outputs":[{"output_type":"stream","name":"stderr","text":["/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py:1462: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use and modify the model generation configuration (see https://huggingface.co/docs/transformers/generation_strategies#default-text-generation-configuration )\n"," warnings.warn(\n"]},{"output_type":"stream","name":"stdout","text":["<|system|>\n","You are a friendly chatbot who always responds in the style of a pirate\n","<|user|>\n","How many helicopters can a human eat in one sitting?\n","<|assistant|>\n","Ahoy matey! Me be thinkin' that a human can't eat a helicopter, for it's not food, it's a flying machine. Me be confused by your question, but me can assure ye that a human can't eat a helicopter in one sitting!\n"]}]},{"cell_type":"code","source":[],"metadata":{"id":"OJKvdt1B8if0"},"execution_count":null,"outputs":[]}]} \ No newline at end of file diff --git a/config.json b/config.json new file mode 100644 index 0000000000000000000000000000000000000000..0d8f4f684f74444a9371d2359830368851268357 --- /dev/null +++ b/config.json @@ -0,0 +1,26 @@ +{ + "_name_or_path": "./zephyr-7b-alpha/", + "architectures": [ + "MistralForCausalLM" + ], + "bos_token_id": 1, + "eos_token_id": 2, + "hidden_act": "silu", + "hidden_size": 4096, + "initializer_range": 0.02, + "intermediate_size": 14336, + "max_position_embeddings": 32768, + "model_type": "mistral", + "num_attention_heads": 32, + "num_hidden_layers": 32, + "num_key_value_heads": 8, + "pad_token_id": 2, + "rms_norm_eps": 1e-05, + "rope_theta": 10000.0, + "sliding_window": 4096, + "tie_word_embeddings": false, + "torch_dtype": "bfloat16", + "transformers_version": "4.34.0", + "use_cache": true, + "vocab_size": 32000 +} diff --git a/eval_results.json b/eval_results.json new file mode 100644 index 0000000000000000000000000000000000000000..ebfacdfda5fb96c0b250fefbffc79e60bd1a50f5 --- /dev/null +++ b/eval_results.json @@ -0,0 +1,16 @@ +{ + "epoch": 1.0, + "eval_logits/chosen": -2.744652509689331, + "eval_logits/rejected": -2.71529483795166, + "eval_logps/chosen": -297.10400390625, + "eval_logps/rejected": -327.4286193847656, + "eval_loss": 0.46045970916748047, + "eval_rewards/accuracies": 0.78125, + "eval_rewards/chosen": -0.5052940249443054, + "eval_rewards/margins": 1.3699172735214233, + "eval_rewards/rejected": -1.8752113580703735, + "eval_runtime": 52.3218, + "eval_samples": 1000, + "eval_samples_per_second": 19.113, + "eval_steps_per_second": 0.306 +} \ No newline at end of file diff --git a/generation_config.json b/generation_config.json new file mode 100644 index 0000000000000000000000000000000000000000..7327bd306f5194519214f3480ea06c6dcd8ceefe --- /dev/null +++ b/generation_config.json @@ -0,0 +1,6 @@ +{ + "_from_model_config": true, + "bos_token_id": 1, + "eos_token_id": 2, + "transformers_version": "4.34.0" +} diff --git a/input_states.safetensors b/input_states.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..30166efedf568b28daebf34304e3b77ba7a29734 --- /dev/null +++ b/input_states.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2ee900988e963f08ba20362e94528b29970774c3bea7240ebfd7e294682c9777 +size 1677721696 diff --git a/job.json b/job.json new file mode 100644 index 0000000000000000000000000000000000000000..057b39690b9bc187f71abf5785ba9d72c39a44ba --- /dev/null +++ b/job.json @@ -0,0 +1,103424 @@ +{ + "in_dir": "/content/zephyr-7b-alpha", + "out_dir": "/content/zephyr_alpha_quantized", + "cal_dataset": "/content/wikitext-test.parquet", + "dataset_rows": 100, + "measurement_rows": 16, + "gpu_rows": 0, + "length": 2048, + "measurement_length": 2048, + "bits": 4.0, + "head_bits": 6, + "progress": "quant", + "shard_size": 8192, + "output_measurement": null, + "compile_full": null, + "rope_scale": 1.0, + "rope_alpha": 1.0, + "cal_filename": "/content/zephyr_alpha_quantized/cal_data.safetensors", + "last_module_idx": 66, + "measurement": [ + { + "key": "model.layers.0.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.011168720200657845, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.009599272161722183, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.004790821112692356, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.005077867768704891, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.0050776307471096516, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.0020578650292009115, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.01079169474542141, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.009518211707472801, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.005259049125015736, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.004644699394702911, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.0048562814481556416, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.0050615472719073296, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.004643067717552185, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.0027538840658962727, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.002120051998645067, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.0026784869842231274, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.001900350907817483, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.001615153276361525, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.0018467125482857227, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.0015654281014576554, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.0017823567613959312, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.0018463897285982966, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.0014250442618504167, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.001544640981592238, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.011168720200657845, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.011168720200657845, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.0.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.011349556967616081, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.00980227068066597, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.004773853346705437, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.005033744964748621, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.005033500958234072, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.001928701065480709, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.01110624335706234, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.009691532701253891, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.005209838971495628, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.004562461748719215, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.0047759003937244415, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.005032561253756285, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.004560141358524561, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.002655236516147852, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.00194216996897012, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.0026120387483388186, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.0016779541037976742, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.001346124685369432, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.0016128767747431993, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.0012812796048820019, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.001617392641492188, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.0016132771270349622, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.0012200467754155397, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.0012539472663775086, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.011349556967616081, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.011349556967616081, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.0.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.11578802764415741, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.07042404264211655, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.043504536151885986, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.050045717507600784, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.0499984472990036, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.024094626307487488, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.07798509299755096, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.06458660215139389, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.0545671209692955, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.03145388513803482, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03755158931016922, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.042070720344781876, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.031096184626221657, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.02426334097981453, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.022416630759835243, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.02171494998037815, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.01284871157258749, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.011072240769863129, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.009356767870485783, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.007840869016945362, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.011130761355161667, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.009300582110881805, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.007214382756501436, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.006086327601224184, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.043504536151885986, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.043504536151885986, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.0.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.11313514411449432, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.0732056200504303, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.05039582401514053, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.04979194700717926, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.048297327011823654, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.027966083958745003, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.07069999724626541, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.06314126402139664, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.053515732288360596, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.031263019889593124, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.03367261961102486, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03614491969347, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.030515408143401146, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.024038290604948997, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.022272668778896332, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.0183316171169281, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.013834427110850811, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.012738020159304142, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.011082837358117104, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.009938649833202362, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.01025769766420126, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.010982340201735497, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.00825857650488615, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.0086810402572155, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.05039582401514053, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.05039582401514053, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.0.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.11327356100082397, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1058005541563034, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.10340237617492676, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.09472587704658508, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.051241014152765274, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.04909159988164902, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.05705980956554413, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.052599381655454636, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.051703788340091705, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.046595051884651184, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.04498507082462311, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.029025588184595108, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.02526126801967621, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.024675775319337845, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.02453121356666088, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.01461299229413271, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.013298182748258114, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.013174419291317463, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.012456698343157768, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.012386062182486057, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.008203603327274323, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.008870589546859264, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.008029122836887836, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.006887989118695259, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.052599381655454636, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.052599381655454636, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.0.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.13404832780361176, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.12670819461345673, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.12447718530893326, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.114055335521698, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.06098993122577667, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.05894749239087105, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.06735212355852127, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.06225910410284996, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.06142910569906235, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.055878788232803345, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.05380960926413536, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.03398489952087402, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.029508553445339203, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.02898118458688259, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.02886049821972847, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.016938773915171623, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.01486075110733509, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.014734474010765553, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01386601198464632, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.013790595345199108, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.008871518075466156, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.008892863988876343, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.008702414110302925, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.005870731081813574, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.05380960926413536, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.05380960926413536, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.0.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.07670606672763824, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.06584762781858444, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.06038632243871689, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.05355939269065857, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.03438195586204529, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.029740920290350914, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.04498032480478287, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.03952185437083244, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.03574554622173309, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.028567753732204437, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.02737824246287346, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.022440247237682343, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.019085902720689774, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.01691751927137375, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.016391701996326447, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.011539027094841003, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.00958372000604868, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.009348219260573387, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.008578285574913025, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.008279938250780106, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.006880487315356731, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.00712636299431324, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.00626949081197381, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.0057810521684587, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.05355939269065857, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.05355939269065857, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.1.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.019663630053400993, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.013406879268586636, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.007660592906177044, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.008349855430424213, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.00809794757515192, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.0036465248558670282, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.013933522626757622, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.012625516392290592, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.0092203663662076, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.006169123109430075, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.006603498011827469, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.007054396439343691, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.006034728605300188, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.0040495740249753, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.003436762373894453, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.0035601980052888393, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.0023379670456051826, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.0019536123145371675, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.0020181608851999044, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.001642401795834303, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.0019508769037202, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.0019979930948466063, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.0013339562574401498, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.0014734480064362288, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.019663630053400993, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.019663630053400993, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.1.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.017518868669867516, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.01219908520579338, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.006678944453597069, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.007290045265108347, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.0070999860763549805, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.0029782995115965605, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.013018052093684673, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.011648166924715042, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.008138769306242466, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.005624793935567141, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.006064086686819792, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.0065219649113714695, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.005529758520424366, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.0035434754099696875, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.0028902385383844376, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.003280432429164648, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.0020198565907776356, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.0016353140817955136, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.0017707019578665495, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.0013762976741418242, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.0017502045957371593, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.0017561280401423573, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.001087298383936286, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.0012416673125699162, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.017518868669867516, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.017518868669867516, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.1.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.13320785760879517, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08857599645853043, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.06414425373077393, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.062488194555044174, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.057051729410886765, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.034113503992557526, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.08158433437347412, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.07376103848218918, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.0631725862622261, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.03857894614338875, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.039744194597005844, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.04156964644789696, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.03522882238030434, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.027669740840792656, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.02555888704955578, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.020771265029907227, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.014558087103068829, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.012766730040311813, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.011037899181246758, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.009418193250894547, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.010778137482702732, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.010443516075611115, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.008073403500020504, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.006722630932927132, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.03857894614338875, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.03857894614338875, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.1.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.15270331501960754, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.12318339943885803, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.11086562275886536, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.09390351921319962, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.07004237920045853, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.05806812644004822, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.08862987905740738, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.07844515144824982, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.07241526991128922, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.05164502561092377, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.049597982317209244, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.04550931602716446, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.0382709726691246, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.034586384892463684, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.033655546605587006, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.02309839054942131, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.01936962828040123, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.01856229268014431, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.01617896556854248, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.015599473379552364, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.013060690835118294, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.014155326411128044, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.011925097554922104, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.011372439563274384, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.05164502561092377, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.05164502561092377, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.1.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.1567811667919159, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1485101878643036, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.14568355679512024, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.13279618322849274, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.0726742148399353, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07009439915418625, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.0808241069316864, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.0739935040473938, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07315933704376221, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06607392430305481, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06361325085163116, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04134838655591011, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03595521301031113, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03534533828496933, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03519531339406967, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.021018361672759056, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.019455352798104286, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.019302669912576675, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01826331578195095, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.01818668842315674, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012167708948254585, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.013381781987845898, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.012010962702333927, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010793034918606281, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04134838655591011, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04134838655591011, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.1.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.19255827367305756, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.18272824585437775, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.17977085709571838, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.16407370567321777, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.08971576392650604, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.08672579377889633, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.098969966173172, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.09122589230537415, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.09027554839849472, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.08172681927680969, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0785735547542572, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.050545584410429, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.043932583183050156, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.04328686743974686, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.04312015324831009, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.025338059291243553, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02311822585761547, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02295522764325142, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02161826193332672, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.02152479812502861, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.013921397738158703, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.015096379444003105, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.013726180419325829, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011499397456645966, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.050545584410429, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.050545584410429, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.1.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.019199078902602196, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.01878763735294342, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.006921666674315929, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.00646975776180625, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.005650496110320091, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.0034061234910041094, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.018811136484146118, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.018254732713103294, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.005731276702135801, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.005354026332497597, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.0053440118208527565, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.005248498171567917, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.005061940755695105, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.0026619865093380213, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.0026766916271299124, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.002273819176480174, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.0023983637802302837, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.001029878854751587, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.0023623292800039053, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.000951776746660471, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.0023449964355677366, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.0023288021329790354, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.0007354797562584281, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.0007976787746883929, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.019199078902602196, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.019199078902602196, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.2.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.05501016974449158, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.043334461748600006, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.03665383905172348, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.03312649205327034, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.024345438927412033, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.018490733578801155, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.03337150067090988, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.03038576804101467, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.025987472385168076, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.018868857994675636, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.018617024645209312, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.01694329082965851, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.014517929404973984, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.011795316822826862, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.011069130152463913, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.008465803228318691, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.00624634325504303, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.005706323776394129, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.005192603450268507, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.004691623616963625, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.00441138306632638, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.004422945436090231, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.0034645930863916874, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.0029624789021909237, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.043334461748600006, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.043334461748600006, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.2.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.05672017112374306, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.04279809445142746, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.03405015915632248, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.03148718550801277, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.024548310786485672, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.017191600054502487, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.03530892729759216, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.032226257026195526, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.026598580181598663, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.01862349361181259, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.018713288009166718, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.017903584986925125, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.015325364656746387, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.011925559490919113, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.010991599410772324, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.008935020305216312, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.0063719660975039005, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.005681759677827358, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.0052579911425709724, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.004604979418218136, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.004677099175751209, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.004683797247707844, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.0035144537687301636, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.00315110944211483, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.04279809445142746, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.04279809445142746, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.2.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.16881753504276276, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.1410287767648697, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.12958964705467224, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.11464769393205643, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.07729382812976837, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.06546717882156372, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.09526020288467407, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.08619674295186996, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.08046350628137589, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.061131030321121216, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.058465443551540375, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.04843978211283684, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.041259944438934326, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.03719085827469826, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.03618834540247917, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.02420763485133648, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.01911242865025997, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.018178023397922516, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.015842380002141, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.015133141539990902, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.012576712295413017, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.012094818986952305, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.011064727790653706, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.0076539963483810425, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.04843978211283684, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.04843978211283684, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.2.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.156541645526886, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.13858985900878906, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.12967531383037567, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.11246905475854874, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.07320044189691544, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.06499434262514114, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.09032202512025833, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.08170590549707413, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.07497423142194748, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.06085894629359245, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.05692308768630028, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.04705378785729408, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.0400383360683918, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.036137260496616364, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.035172898322343826, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.024017736315727234, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.020189229398965836, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.019475512206554413, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.0179683156311512, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.017397213727235794, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.013736758381128311, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.014681994915008545, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.01252848468720913, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.01172161940485239, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.04705378785729408, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.04705378785729408, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.2.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.19975252449512482, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.18823857605457306, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.1846926212310791, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.16731983423233032, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.09383014589548111, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.09011030942201614, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.10378500074148178, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.09576664119958878, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.09457574784755707, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.08421718329191208, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.08041726052761078, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.05310473218560219, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.04596571624279022, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.04511168226599693, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.04491891711950302, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02657609060406685, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.023638196289539337, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.023406749591231346, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.021765321493148804, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.021637681871652603, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.014316429384052753, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.014850785955786705, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.014048966579139233, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010670391842722893, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.05310473218560219, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.05310473218560219, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.2.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.23413872718811035, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.22090375423431396, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.21686704456806183, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.19659093022346497, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.10993479937314987, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10565081238746643, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12136806547641754, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.1120128333568573, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11077161878347397, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09870945662260056, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0940827876329422, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06181109696626663, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05351909250020981, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05259128287434578, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.052355699241161346, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03086814470589161, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02699517272412777, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02673065848648548, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.024733295664191246, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.024595467373728752, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016258612275123596, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016147766262292862, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.015950508415699005, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010682974942028522, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05351909250020981, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05351909250020981, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.2.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.1790362149477005, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.15777480602264404, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.14935097098350525, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.13222238421440125, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.08091238141059875, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.07288438826799393, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.0961962640285492, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.08775460720062256, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.083379827439785, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.06838435679674149, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.06484240293502808, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.04880445823073387, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.04199089854955673, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.03893505781888962, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.03818897530436516, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.02450370043516159, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.020580368116497993, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02021637000143528, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.018184015527367592, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.017703499644994736, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.013272203505039215, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.01354095060378313, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.012265851721167564, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.009716490283608437, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.04880445823073387, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.04880445823073387, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.3.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.04273678734898567, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.03584158048033714, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.03184032440185547, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.028262462466955185, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.01939507946372032, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.015999650582671165, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.025632822886109352, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.02331860177218914, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.020244425162672997, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.01563546620309353, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.015160813927650452, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.013055362738668919, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.011152705177664757, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.009398764930665493, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.008941682986915112, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.006538650952279568, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.004987573716789484, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.004653621930629015, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.004263903480023146, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.003960269037634134, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.003473845310509205, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.0034712685737758875, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.002868041628971696, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.0023902307730168104, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.04273678734898567, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.04273678734898567, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.3.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.0428481288254261, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.03501201048493385, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.029858678579330444, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.026616744697093964, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.019089410081505775, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.01491191703826189, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.02664991095662117, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.024140529334545135, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.020131420344114304, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.015170253813266754, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.014985787682235241, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.01351439580321312, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.011503629386425018, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.00922862533479929, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.008605791255831718, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.006766557227820158, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.0048356736078858376, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.0043977960012853146, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.004073855467140675, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.0036462252028286457, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.0035126814618706703, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.0034059712197631598, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.0026915359776467085, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.002188342157751322, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.0428481288254261, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.0428481288254261, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.3.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.1756085604429245, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.14955584704875946, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.13890667259693146, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.12239907681941986, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.08067318052053452, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.06992373615503311, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.09721731394529343, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.08924295753240585, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.0834602639079094, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.06486669182777405, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.06175591051578522, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.04936389625072479, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04259105026721954, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.03869667276740074, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.03773403912782669, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.02467101626098156, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.01980268396437168, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.018941247835755348, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.016586411744356155, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.015916477888822556, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.012710344977676868, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.012334806844592094, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.011317485012114048, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.007636150810867548, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.04936389625072479, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.04936389625072479, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.3.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.17265617847442627, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.15349774062633514, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.14630544185638428, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.125789612531662, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.08070380240678787, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.07379523664712906, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.09464309364557266, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.0860721543431282, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.08211905509233475, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.0660165399312973, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.06170710548758507, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.0487380288541317, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.041771307587623596, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.039406705647706985, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.038827069103717804, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.024621980264782906, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.021481908857822418, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.02098594233393669, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.018773801624774933, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.018404876813292503, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.013682577759027481, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.014801764860749245, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.012944573536515236, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.011539943516254425, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.0487380288541317, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.0487380288541317, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.3.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.209043487906456, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.19700421392917633, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.19321338832378387, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.17542071640491486, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.09851204603910446, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.09449568390846252, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.1090313121676445, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.10056618601083755, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.0993163213133812, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.08842168003320694, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.08447795361280441, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.05569034069776535, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.04819665849208832, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.04729839414358139, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.04708410054445267, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02784634381532669, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.024610033258795738, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.024359652772545815, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02261015959084034, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.022482460364699364, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.014900386333465576, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.015216012485325336, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.014612419530749321, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010649471543729305, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.04819665849208832, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.04819665849208832, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.3.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.24484506249427795, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.23094336688518524, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.22673256695270538, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.2060040980577469, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11548089236021042, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.11090388149023056, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.1278538554906845, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11774861067533493, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11639799177646637, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.1037989929318428, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09926366060972214, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.0653105154633522, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05634693056344986, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05533489212393761, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05509016662836075, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.032666780054569244, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.028497371822595596, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.028219513595104218, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.026142287999391556, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.025994718074798584, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.0174561757594347, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.017186559736728668, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01712212525308132, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011524848639965057, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.032666780054569244, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.032666780054569244, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.3.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.20301590859889984, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.17993180453777313, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.17123068869113922, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.15232975780963898, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09230129420757294, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08366638422012329, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.10861063003540039, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.09921830892562866, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09496082365512848, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.07847336679697037, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.07448621094226837, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05513899773359299, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.04751267284154892, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.044402044266462326, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04363902285695076, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.027674056589603424, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.023362280800938606, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.022996846586465836, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.020694028586149216, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.020208235830068588, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.014936521649360657, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.015182916074991226, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.013920888304710388, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.010784035548567772, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.04751267284154892, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.04751267284154892, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.4.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.05948399379849434, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.05045810714364052, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.0450628362596035, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.04024699702858925, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.02709878981113434, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.022555913776159286, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.03599774092435837, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.03251658007502556, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.02822723612189293, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.022175561636686325, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.02156623639166355, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.01832883432507515, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.015611917711794376, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.013163863681256771, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.012529414147138596, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.009210369549691677, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.007052582688629627, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.006607340648770332, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.006121970247477293, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.00571369007229805, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.004913266748189926, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.004976646043360233, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.0040607331320643425, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.003532960545271635, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.05045810714364052, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.05045810714364052, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.4.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.05663783848285675, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.04680328816175461, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.04078899696469307, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.036554399877786636, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.025308020412921906, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.020324628800153732, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.03483179956674576, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.031312715262174606, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.026685133576393127, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.02047596499323845, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.020128708332777023, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.01776907965540886, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.014995941892266273, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.01226199883967638, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.011548623442649841, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.008920381776988506, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.006504755932837725, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.0060025774873793125, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.005570911802351475, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.005091105587780476, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.004694020375609398, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.0045917523093521595, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.0037039704620838165, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.003107720520347357, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.04680328816175461, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.04680328816175461, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.4.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.1917179524898529, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.16577664017677307, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.15542399883270264, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.1376524418592453, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.08840838074684143, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.07796114683151245, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.1050894558429718, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.09659653156995773, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.09124546498060226, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.07238545268774033, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.06868880242109299, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.05346288904547691, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.046208061277866364, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04242615029215813, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.041495393961668015, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.026723310351371765, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.021805057302117348, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02098340354859829, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.01855577528476715, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.0179296862334013, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.013873790390789509, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.013583481311798096, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.012519973330199718, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.008629865944385529, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.05346288904547691, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.05346288904547691, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.4.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.16939057409763336, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.1513446867465973, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.14464770257472992, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.12346041202545166, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.079004667699337, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.07215382903814316, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.09365367144346237, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.08410616219043732, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.08046706020832062, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.06545904278755188, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.0601811446249485, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.04818735271692276, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.040927812457084656, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.03869343549013138, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.038167115300893784, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.024444689974188805, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.021173864603042603, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.020702725276350975, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.018659424036741257, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.018337540328502655, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.013722981326282024, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.01465285662561655, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.013013369403779507, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.011514890007674694, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.04818735271692276, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.04818735271692276, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.4.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.1883140653371811, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.17655447125434875, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.1725977212190628, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.15641066431999207, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.0888027548789978, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.08470439165830612, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.09940768033266068, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.0913398340344429, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.08968131244182587, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.07924304902553558, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.07571203261613846, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.05076218023896217, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.04382229968905449, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.042683910578489304, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.042410291731357574, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.025482339784502983, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02221999131143093, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02194603905081749, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.020312026143074036, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.020150791853666306, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.013768038712441921, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.013788512907922268, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.013400079682469368, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.009614584967494011, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.05076218023896217, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.05076218023896217, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.4.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.246357262134552, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.23156091570854187, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.22673998773097992, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.20575161278247833, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11632581055164337, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.11125487089157104, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12949900329113007, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11917395889759064, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11740680038928986, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.10403173416852951, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09944429248571396, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06603679060935974, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05707656964659691, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05578989163041115, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05549166351556778, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03308970108628273, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.028737511485815048, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02841159887611866, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.026251301169395447, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.02606653794646263, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.0176161490380764, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01735866442322731, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.017196768894791603, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011594372801482677, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03308970108628273, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03308970108628273, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.4.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.2095576524734497, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.18695861101150513, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.17822471261024475, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.15851232409477234, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09566449373960495, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08714070916175842, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.11320783942937851, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10278883576393127, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09833946079015732, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08187927305698395, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.0775674432516098, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.0575646311044693, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.04935459420084953, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.046164318919181824, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04538174346089363, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.028994236141443253, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.024500075727701187, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.024127205833792686, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.021858954802155495, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.02137257158756256, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.015817200765013695, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.016173504292964935, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.014756884425878525, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.011794302612543106, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.04935459420084953, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.04935459420084953, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.5.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.07179942727088928, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.06272782385349274, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.0575268529355526, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.05125272274017334, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.03292565792798996, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.028575770556926727, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.04211081191897392, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.03832326829433441, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.03401795029640198, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.027597596868872643, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.02656625211238861, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.021354198455810547, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.01834231987595558, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.01593482308089733, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.015323616564273834, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.010706116445362568, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.008427944034337997, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.007987068966031075, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.007394589949399233, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.006993317045271397, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.005677962210029364, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.005734641570597887, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.004852258134633303, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.003964860457926989, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.05125272274017334, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.05125272274017334, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.5.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.06473594158887863, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.055526718497276306, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.049911461770534515, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.044354259967803955, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.02931882254779339, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.02469693124294281, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.03860938921570778, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.03517904505133629, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.030482374131679535, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.024252815172076225, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.023485815152525902, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.019544633105397224, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.016757169738411903, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.014120960608124733, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.01342977024614811, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.009770752862095833, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.0073308334685862064, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.006842531729489565, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.006326897535473108, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.005861466750502586, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.0050523970276117325, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.004929880145937204, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.004123569931834936, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.003137608990073204, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.049911461770534515, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.049911461770534515, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.5.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.20667532086372375, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.18412651121616364, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.17575180530548096, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.1560567021369934, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.09606868773698807, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.08750919252634048, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.11123546212911606, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.10240286588668823, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.09829515218734741, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.08090247958898544, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.07650333642959595, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.056504372507333755, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04887129366397858, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.046007949858903885, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.045349981635808945, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.0281996950507164, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.023354772478342056, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.022706395015120506, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.02021184004843235, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.019752709195017815, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.014485612511634827, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.013975773938000202, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.013443267904222012, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.008471165783703327, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04887129366397858, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04887129366397858, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.5.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.18959152698516846, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.17195124924182892, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.1653534173965454, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.14394891262054443, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.08860822021961212, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.0817338302731514, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10249451547861099, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.0936884954571724, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09002437442541122, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.0746944472193718, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07026924192905426, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05280518904328346, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.045185159891843796, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04280707612633705, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04224920645356178, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.026574712246656418, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.02261975221335888, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.022079620510339737, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.019902443513274193, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.019540734589099884, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.01446588896214962, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.014702496118843555, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.013650595210492611, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.010650047101080418, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05280518904328346, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05280518904328346, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.5.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.1762571483850479, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1653290092945099, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.16162365674972534, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.14647088944911957, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.08305193483829498, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07925103604793549, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.09263620525598526, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08550563454627991, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.08388134837150574, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.07411837577819824, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.07074187695980072, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.047173138707876205, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.040913499891757965, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.039821311831474304, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03956005722284317, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02358744852244854, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.020505355671048164, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.020246082916855812, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.018697958439588547, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.018533287569880486, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012398071587085724, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012418893165886402, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.012031331658363342, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.00827084295451641, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.047173138707876205, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.047173138707876205, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.5.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.2419440895318985, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.2276521474123001, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2230045050382614, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.20235870778560638, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.1143060028553009, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10936648398637772, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12692731618881226, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11716075241565704, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11531758308410645, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.10232201963663101, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09775715321302414, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06462647765874863, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.056016407907009125, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05473296344280243, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05443199723958969, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.032305825501680374, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02803993970155716, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.027713479474186897, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02559611015021801, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.025405537337064743, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016914283856749535, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016712086275219917, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01648643985390663, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010864359326660633, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.032305825501680374, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.032305825501680374, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.5.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.2036168873310089, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.1832730621099472, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.17470474541187286, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.1561361700296402, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.0934390053153038, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08529166877269745, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.111430324614048, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10100080072879791, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09583073854446411, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08062601834535599, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.07705391943454742, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05686961114406586, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.04845084622502327, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.04500170052051544, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04415525868535042, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.028722483664751053, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.02372308075428009, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.023303192108869553, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.021242249757051468, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.020712487399578094, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.015683121979236603, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.01551529485732317, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.014488616958260536, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.011047808453440666, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.04845084622502327, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.04845084622502327, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.6.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.06634723395109177, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.05762307345867157, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.051811300218105316, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.04625110328197479, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.030352430418133736, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.025681382045149803, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.04047377035021782, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.03665715083479881, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.03144051507115364, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.025375576689839363, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.02464967966079712, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.020561281591653824, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.017536645755171776, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.014713982120156288, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.013981547206640244, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.010300645604729652, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.007822458632290363, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.007315567694604397, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.006863356567919254, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.006390023976564407, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.0054395729675889015, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.005470580421388149, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.004440734162926674, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.003781738691031933, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.051811300218105316, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.051811300218105316, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.6.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.060605134814977646, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.051686182618141174, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.04550164192914963, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.04061912000179291, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.02736980840563774, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.022464901208877563, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.03753753378987312, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.034015703946352005, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.02854038216173649, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.022614534944295883, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.022116539999842644, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.019017454236745834, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.016205081716179848, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.013237089850008488, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.012449914589524269, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.009491239674389362, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.006952513940632343, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.006399805191904306, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.006020466797053814, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.005488865077495575, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.0049445740878582, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.004851828329265118, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.003889617044478655, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.0031672290060669184, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.051686182618141174, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.051686182618141174, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.6.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.18390421569347382, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.16390341520309448, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.1555459350347519, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.13815124332904816, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.0854179784655571, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.07736478745937347, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.10099665075540543, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.09283767640590668, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.08745920658111572, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.07208962738513947, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.06840716302394867, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.051350172609090805, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04431234672665596, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04098079353570938, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.040139734745025635, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.02561933919787407, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.020926428958773613, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.020221518352627754, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.01816825568675995, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.017614122480154037, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.013170411810278893, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.012800670228898525, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.011958121322095394, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.007893593981862068, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.051350172609090805, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.051350172609090805, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.6.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.18748553097248077, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.1659504473209381, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.15739239752292633, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.13692741096019745, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.08746771514415741, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.07871513068675995, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10398761928081512, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.09420737624168396, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.0894230529665947, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07223328202962875, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.06758613139390945, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.053499579429626465, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.045534636825323105, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04249802604317665, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.041758015751838684, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.026858501136302948, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.02269449643790722, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.02203347347676754, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.01975790224969387, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.019285235553979874, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.0145133500918746, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.015179971233010292, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.01350901834666729, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.011253130622208118, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.053499579429626465, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.053499579429626465, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.6.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.17114149034023285, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.16083428263664246, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.15720336139202118, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.14284665882587433, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.08073896169662476, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07706644386053085, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.09042980521917343, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08328168839216232, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.08152130991220474, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.07229864597320557, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06913543492555618, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.046180956065654755, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.039897676557302475, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03875955566763878, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03848888725042343, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02312663570046425, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.020065199583768845, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.019804779440164566, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.018367337062954903, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.018200399354100227, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012346301227807999, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012314205057919025, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011960572563111782, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008389642462134361, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.046180956065654755, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.046180956065654755, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.6.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.23557148873806, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.22183120250701904, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.21720033884048462, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.19756180047988892, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11134614795446396, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10655659437179565, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12412071228027344, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11433691531419754, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11234697699546814, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09990664571523666, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0955205112695694, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06329933553934097, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.054706476628780365, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05334842950105667, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05303625017404556, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03165601193904877, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02740606665611267, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.027081916108727455, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02507716789841652, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.024877533316612244, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016712261363863945, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016471730545163155, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.016263943165540695, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010853748768568039, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05334842950105667, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05334842950105667, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.6.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.21076911687850952, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.19050660729408264, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.18252255022525787, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.16333793103694916, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09705882519483566, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08929018676280975, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.11463408172130585, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10375270992517471, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09938328713178635, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08413700014352798, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08029516786336899, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05836555361747742, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.04982609674334526, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.04680825024843216, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.046071410179138184, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.029368113726377487, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.024785494431853294, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.024426598101854324, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.02229994349181652, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.02183464542031288, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.015965718775987625, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.016251832246780396, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.014962779358029366, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.011789047159254551, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.04982609674334526, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.04982609674334526, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.7.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.07253632694482803, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.06478405743837357, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.05837472155690193, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.05205954238772392, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.03348775953054428, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.02869037352502346, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.04561757668852806, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.04102073982357979, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.03436758741736412, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.028703801333904266, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.027966883033514023, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.023239726200699806, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.01973414234817028, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.01628398336470127, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.015371035784482956, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.011651421897113323, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.008739606477320194, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.008151110261678696, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.007829796522855759, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.007258750032633543, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.006187691818922758, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.006257256492972374, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.004955670330673456, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.004403706174343824, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.05205954238772392, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.05205954238772392, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.7.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.0626121312379837, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.05524720624089241, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.04855867102742195, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.043166615068912506, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.028583906590938568, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.023709077388048172, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.04002339765429497, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.03640666976571083, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.029457036405801773, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.02430696412920952, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.023732928559184074, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.02031564898788929, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.017455879598855972, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.013826129958033562, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.012836582958698273, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.010184375569224358, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.007291768211871386, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.006658272352069616, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.006462015211582184, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.0058193085715174675, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.005293699912726879, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.005209335591644049, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.00403254572302103, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.0033859771210700274, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.04855867102742195, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.04855867102742195, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.7.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.19899427890777588, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.17977093160152435, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.1722375452518463, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.15291020274162292, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.09298620373010635, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.0854603573679924, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.10868975520133972, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.09914804995059967, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.09474433958530426, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.07913115620613098, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.07504456490278244, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.055301252752542496, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04738381505012512, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04455778747797012, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04388604313135147, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.027598528191447258, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.022764069959521294, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.0221426859498024, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.019917359575629234, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.019456520676612854, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.014231402426958084, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.01377798430621624, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.013111609034240246, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.008610040880739689, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04738381505012512, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04738381505012512, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.7.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.1884087473154068, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.16933833062648773, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.16095305979251862, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.1393108069896698, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.08832570910453796, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.08029365539550781, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10636451095342636, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.09553088247776031, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09012182801961899, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07366209477186203, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.0699334368109703, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.054981034249067307, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.046462249010801315, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04323485121130943, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04246465116739273, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.027733784168958664, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.023538334295153618, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.022891977801918983, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.020789284259080887, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.020309122279286385, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.015384198166429996, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.01624443754553795, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.014347760006785393, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.012545468285679817, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.046462249010801315, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.046462249010801315, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.7.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.1666836440563202, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1567574441432953, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.1534167230129242, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.13940130174160004, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.0788261890411377, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07530619204044342, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08801373839378357, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08102373778820038, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07955755293369293, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.07058826088905334, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06748536229133606, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04497779533267021, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03889338672161102, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.0379004031419754, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03766704350709915, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02252550795674324, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.01975112594664097, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.019513443112373352, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01810702309012413, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.0179611723870039, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012069562450051308, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012280250899493694, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01174020767211914, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008592424914240837, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04497779533267021, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04497779533267021, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.7.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.23579122126102448, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.222219780087471, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.21776892244815826, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.1981118619441986, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.1117170974612236, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10700331628322601, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12449662387371063, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11445385217666626, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11269330978393555, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.10021616518497467, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09587167948484421, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06365256011486053, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05487082898616791, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.053646303713321686, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.053360287100076675, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03187926858663559, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.027765214443206787, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.0274557713419199, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.025439215824007988, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.02525932528078556, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.01705533266067505, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016955142840743065, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.016653813421726227, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.0115723367780447, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.053646303713321686, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.053646303713321686, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.7.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.20433014631271362, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.18405292928218842, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.1758449673652649, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.15770600736141205, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09402231127023697, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08595852553844452, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.11139664053916931, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10105358064174652, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09641910344362259, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08123823255300522, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.07780414819717407, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.056854426860809326, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.048471637070178986, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.04529405012726784, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.044517822563648224, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.028560878708958626, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.02386007271707058, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.023480109870433807, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.021384425461292267, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.020891698077321053, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.015477100387215614, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.015538507141172886, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.014423436485230923, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.011068885214626789, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.048471637070178986, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.048471637070178986, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.8.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.07229823619127274, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.0640784278512001, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.05915708839893341, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.05233281850814819, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.03329877555370331, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.029231058433651924, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.042107172310352325, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.038525331765413284, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.03417966887354851, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.028040915727615356, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.026857197284698486, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.0213544350117445, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.01839970052242279, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.01605963334441185, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.01547007542103529, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.010699591599404812, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.008419157937169075, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.007991536520421505, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.007386090699583292, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.00699745723977685, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.005629621911793947, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.005631897132843733, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.0048264265060424805, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.0037945793010294437, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.05233281850814819, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.05233281850814819, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.8.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.06344876438379288, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.05514335259795189, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.05007714778184891, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.044085338711738586, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.028834931552410126, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.024654246866703033, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.03750180825591087, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.034161586314439774, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.029770636931061745, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.023919442668557167, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.023006537929177284, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.019016969949007034, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.016308337450027466, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.013897260650992393, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.013273438438773155, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.009519466198980808, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.0072477711364626884, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.006803502328693867, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.006268806755542755, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.0058531141839921474, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.0049530318938195705, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.004873728379607201, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.004104185849428177, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.0031792030204087496, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.05007714778184891, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.05007714778184891, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.8.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.18240326642990112, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.1626574695110321, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.15496958792209625, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.13663387298583984, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.08455818891525269, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.07684142887592316, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.09996659308671951, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.09083674848079681, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.08648092299699783, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.07106637209653854, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.06717733293771744, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.05087890848517418, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04350433498620987, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04059627652168274, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.039900291711091995, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.025416448712348938, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.020873580127954483, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.020246481522917747, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.018127888441085815, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.017663544043898582, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.013154354877769947, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.012906396761536598, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.012026366777718067, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.008315572515130043, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.05087890848517418, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.05087890848517418, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.8.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.1977970451116562, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.18005962669849396, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.17297670245170593, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.15205296874046326, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.09264685213565826, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.0856722965836525, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10854403674602509, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.09851948916912079, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09423834830522537, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07904263585805893, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07453982532024384, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05574023351073265, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04742798954248428, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.044760387390851974, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.044106315821409225, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.027905823662877083, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.023507753387093544, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.022936217486858368, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.02082017809152603, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.020411867648363113, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.014840735122561455, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.015165221877396107, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.013953466899693012, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.010774858295917511, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04742798954248428, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04742798954248428, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.8.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.16268427670001984, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1532304435968399, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.14993759989738464, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.1363285779953003, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07694627344608307, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.0735417902469635, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08607260882854462, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07920283079147339, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07766101509332657, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06899941712617874, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0660800039768219, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.0439998134970665, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.037978317588567734, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.036970168352127075, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03672965243458748, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02201676554977894, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.01919376291334629, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.018961550667881966, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01760033145546913, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.017453495413064957, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.011735353618860245, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.011853777803480625, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011399183422327042, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008189890533685684, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.0439998134970665, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.0439998134970665, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.8.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.23203247785568237, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.21882356703281403, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2143489271402359, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.1949910670518875, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.10994546860456467, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10524184256792068, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.1226196438074112, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11273852735757828, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11088178306818008, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09857067465782166, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09438752382993698, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06263895332813263, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05398797243833542, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.052726712077856064, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.052423760294914246, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03131991624832153, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.027149271219968796, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.026830697432160378, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.024843666702508926, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.02466162107884884, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016555219888687134, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016400840133428574, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01612219400703907, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010934562422335148, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05398797243833542, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05398797243833542, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.8.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.21018268167972565, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.19030331075191498, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.18263329565525055, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.16296975314617157, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09686324745416641, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.0892181545495987, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.11363700777292252, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10320858657360077, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09913817793130875, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08380107581615448, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.0797918289899826, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05793079733848572, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.049579475075006485, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.046735674142837524, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04604290425777435, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.029255900532007217, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.024781428277492523, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02444753423333168, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.022280484437942505, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.021841231733560562, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.016100801527500153, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.016270168125629425, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.015168504789471626, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.01186355110257864, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.049579475075006485, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.049579475075006485, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.9.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.0923733115196228, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.08220721781253815, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.07622025161981583, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.06749071180820465, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.04258981719613075, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.037622444331645966, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.05377095192670822, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.04885927587747574, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.04363660514354706, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.0360131561756134, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.03449761122465134, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.02736286073923111, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.023438431322574615, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.02063322439789772, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.01992359384894371, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.013733101077377796, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.010986043140292168, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.010486569255590439, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.009727832861244678, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.009283306077122688, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.007337464485317469, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.007517325226217508, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.00634734658524394, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.00537061644718051, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.05377095192670822, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.05377095192670822, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.9.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.07935116440057755, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.06979311257600784, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.06384820491075516, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.05638270825147629, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.03625747188925743, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.031399913132190704, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.04671098664402962, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.04258308187127113, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.0373074896633625, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.030413977801799774, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.02923060953617096, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.023759830743074417, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.020337557420134544, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.017467139288783073, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.01673617959022522, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.011891154572367668, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.009111745283007622, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.008590858429670334, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.007955510169267654, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.007472165394574404, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.00619489885866642, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.006100549828261137, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.0051757474429905415, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.00400660140439868, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.04671098664402962, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.04671098664402962, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.9.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.19220881164073944, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.17436790466308594, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.16780315339565277, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.14817847311496735, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.0897936075925827, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.08307909220457077, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.10316525399684906, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.09467410296201706, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.09124304354190826, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.07631740719079971, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.07175160944461823, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.052423689514398575, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.045171648263931274, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04294484108686447, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04242084175348282, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.026141708716750145, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.021819069981575012, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02130700834095478, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.019036483019590378, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.01867504231631756, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.013404963538050652, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.012975042685866356, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.012573492713272572, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.007934489287436008, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.052423689514398575, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.052423689514398575, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.9.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.1993076205253601, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.18014933168888092, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.17243245244026184, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.15134650468826294, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.09344325959682465, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.08576633036136627, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10941915959119797, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.09956937283277512, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09505053609609604, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07860640436410904, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07455437630414963, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05619671940803528, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04785337299108505, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04504535347223282, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04437866061925888, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.028088334947824478, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.02348434552550316, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.022888371720910072, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.02051377296447754, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.020069656893610954, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.014821244403719902, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.014906519092619419, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.013882310129702091, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.01027657650411129, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04785337299108505, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04785337299108505, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.9.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.16206996142864227, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.15213730931282043, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.14868398010730743, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.13507655262947083, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07658286392688751, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07301879674196243, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08578245341777802, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07891295105218887, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07731891423463821, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.0684279277920723, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06548789143562317, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.043890975415706635, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03789806365966797, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.036841198801994324, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03659507632255554, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.021988868713378906, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.019250184297561646, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.019006384536623955, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.017626376822590828, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.017475441098213196, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.0117854755371809, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01205415092408657, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011435518972575665, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008514638058841228, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.043890975415706635, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.043890975415706635, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.9.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.22811272740364075, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.21440167725086212, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2097819298505783, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.19063790142536163, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.10790391266345978, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10301275551319122, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12053030729293823, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11082379519939423, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.10892365127801895, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.096433125436306, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09228646010160446, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06161956861615181, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.053128574043512344, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.051801811903715134, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05148930847644806, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.030858976766467094, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.026810333132743835, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.026479406282305717, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.024499699473381042, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.024301273748278618, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016448548063635826, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016406487673521042, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.016002364456653595, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011173042468726635, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.053128574043512344, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.053128574043512344, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.9.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.21688273549079895, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.195871502161026, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.18770845234394073, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.16686895489692688, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09991750866174698, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09185732901096344, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.11755452305078506, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10667914897203445, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10227162390947342, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08605503290891647, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.0815466120839119, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05971937254071236, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05112798139452934, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.048075463622808456, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04733084887266159, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.030013510957360268, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.02524554915726185, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.024882636964321136, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.022554337978363037, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.022082101553678513, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.016292694956064224, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.016276968643069267, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.015285375528037548, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.011505048722028732, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05112798139452934, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05112798139452934, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.10.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.08111102879047394, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.07254783809185028, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.06766746938228607, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.05942074581980705, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.037567462772130966, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.03346123546361923, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.04657381772994995, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.042521990835666656, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.03834857791662216, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.03160262852907181, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.030077604576945305, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.023668518289923668, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.020309679210186005, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.018092622980475426, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.017540713772177696, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.011844425462186337, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.00947027001529932, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.009059125557541847, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.008297783322632313, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.007934422232210636, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.006231253035366535, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.006245048716664314, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.00544535368680954, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.004236916080117226, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.04657381772994995, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.04657381772994995, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.10.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.07359469681978226, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.06516092270612717, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.06042276322841644, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.05277974158525467, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.033782899379730225, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.029807444661855698, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.042435433715581894, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.03851376846432686, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.034619253128767014, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.02816605381667614, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.026803895831108093, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.02149576134979725, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.018389325588941574, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.016235601156949997, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.01569240540266037, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.010735702700912952, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.008401831611990929, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.007987060584127903, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.007274224888533354, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.00690606702119112, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.0055682044476270676, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.005442801862955093, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.004773732740432024, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.0034874265547841787, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.05277974158525467, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.05277974158525467, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.10.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.17976276576519012, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.16189919412136078, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.15510153770446777, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.1357804536819458, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.08386819809675217, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.07699833065271378, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.09768102318048477, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.08885928243398666, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.08545619249343872, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.07042080163955688, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.06619781255722046, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.049674712121486664, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04243537038564682, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04017745330929756, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.03963658586144447, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.024822253733873367, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.020555876195430756, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02002415992319584, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.017803162336349487, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.017433637753129005, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.012822971679270267, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.012464363127946854, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.011917990632355213, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.007906206883490086, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.049674712121486664, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.049674712121486664, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.10.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.19702740013599396, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.17771965265274048, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.16960251331329346, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.1487310379743576, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.09191865473985672, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.08419815450906754, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10898296535015106, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.09889288991689682, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09361158311367035, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.0775689110159874, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07357574999332428, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.056102555245161057, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04769488424062729, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04453796520829201, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04377840459346771, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.028172001242637634, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.023658720776438713, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.02303505875170231, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.0208913441747427, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.020396705716848373, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.015303699299693108, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.01567230373620987, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.01426894124597311, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.01145181804895401, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04769488424062729, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04769488424062729, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.10.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.16228733956813812, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.15147458016872406, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.147598534822464, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.133662149310112, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07651983946561813, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07260174304246902, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08628086000680923, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07926963269710541, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07741113752126694, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06795590370893478, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0649128332734108, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04409165307879448, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03805132955312729, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03680633008480072, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03651457279920578, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.022104473784565926, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.019228477030992508, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.018950073048472404, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01753089763224125, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.0173508133739233, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.011845353990793228, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012083268724381924, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011417168192565441, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.00851461011916399, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04409165307879448, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04409165307879448, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.10.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.22261309623718262, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.20802195370197296, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.20303837954998016, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.18380849063396454, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.1050167977809906, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.0998120978474617, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.11781039834022522, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.10840636491775513, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.10619650781154633, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.0932191014289856, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0889882817864418, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06010079383850098, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.0518934428691864, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.050366226583719254, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.050000064074993134, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.030053550377488136, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.025963500142097473, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02559644542634487, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02357950061559677, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.023345347493886948, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.015806002542376518, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.015786699950695038, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.015273896045982838, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010550056584179401, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.0518934428691864, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.0518934428691864, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.10.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.22963909804821014, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.20756880939006805, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.19922778010368347, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.1766587495803833, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.1061212494969368, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.0978289544582367, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12449796497821808, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11283789575099945, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10863349586725235, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.09140748530626297, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08657213300466537, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06365633755922318, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.0542290173470974, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.051196929067373276, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.05046088993549347, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.03204844519495964, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.027043066918849945, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.026687270030379295, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.02419471926987171, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.023724334314465523, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.017471883445978165, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.017611542716622353, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.01644137129187584, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.012711179442703724, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.0542290173470974, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.0542290173470974, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.11.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.09471038728952408, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.08441315591335297, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.07877811789512634, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.06901639699935913, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.04399825632572174, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.03916407749056816, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.05446501821279526, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.04960593208670616, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.04493807256221771, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.03683320805430412, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.03501424565911293, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.02770739234983921, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.023780781775712967, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.021279659122228622, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.020652614533901215, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.013886299915611744, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.011258203536272049, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.010791515931487083, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.009872546419501305, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.009467078372836113, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.0073661478236317635, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.007558855228126049, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.006490133237093687, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.005331674125045538, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.04960593208670616, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.04960593208670616, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.11.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.0793822705745697, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.07023479789495468, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.06453816592693329, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.05635121092200279, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.03643607348203659, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03185543790459633, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.04654272645711899, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.04240185767412186, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.03729833662509918, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.03040153533220291, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.02904202975332737, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.023674670606851578, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.020249824970960617, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.01757160946726799, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.016896599903702736, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.011824156157672405, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.009196280501782894, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.008705075830221176, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.00800666119903326, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.007557692006230354, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.00619993731379509, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.006152571178972721, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.005265026353299618, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.004109062720090151, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.04654272645711899, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.04654272645711899, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.11.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.1938401460647583, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.17283737659454346, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.1647641360759735, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.14326342940330505, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.09012022614479065, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.08195556700229645, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.10631053894758224, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.0960053950548172, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.09189220517873764, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.07470855861902237, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.06997879594564438, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.054187141358852386, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04590385779738426, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04318470135331154, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04252967983484268, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.027100728824734688, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.022095059975981712, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.021475177258253098, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.01895921491086483, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.018507080152630806, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.013989497907459736, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.013476541265845299, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.012775829993188381, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.008517906069755554, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.054187141358852386, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.054187141358852386, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.11.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.19575975835323334, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.17775772511959076, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.16314177215099335, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.14397133886814117, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.09221747517585754, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.08105906844139099, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.12340552359819412, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.10978596657514572, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.0938209667801857, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.0790930762887001, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.0766202062368393, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.06391671299934387, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05313868075609207, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04506020247936249, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04295142740011215, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.032097749412059784, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.024306951090693474, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.022937802597880363, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.02180151827633381, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.020510662347078323, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.017426537349820137, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.01731007546186447, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.0146400835365057, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.012545672245323658, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05313868075609207, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05313868075609207, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.11.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.1656651645898819, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.15426109731197357, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.15001104772090912, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.13573534786701202, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.078199602663517, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07397285103797913, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08861343562602997, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08129531145095825, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07914450764656067, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06920995563268661, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06611698120832443, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04535221680998802, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03911295160651207, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03772561997175217, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03739975392818451, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.022767936810851097, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.019898805767297745, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.01959347166121006, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01814555749297142, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.017941074445843697, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012317722663283348, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012772978283464909, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011842994019389153, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.009278111159801483, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04535221680998802, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04535221680998802, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.11.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.2246875762939453, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.2094203233718872, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.20410379767417908, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.18452906608581543, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.10587392002344131, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10035202652215958, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.1192048043012619, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.10953209549188614, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.1070624440908432, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09369143098592758, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0893295407295227, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06066050007939339, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05242482200264931, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05077127739787102, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05038008093833923, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.030338531360030174, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.026141522452235222, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.025747839361429214, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02367529831826687, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.023426810279488564, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.0158806461840868, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.015857579186558723, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01529465802013874, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010516893118619919, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05242482200264931, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05242482200264931, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.11.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.22915378212928772, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.20730051398277283, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.19904541969299316, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.17664343118667603, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.10618459433317184, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09784796833992004, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12465621531009674, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.1128915399312973, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10870573669672012, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.09153155237436295, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08664526045322418, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06376823782920837, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05429444834589958, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05123336985707283, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.05048857629299164, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.03208930417895317, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.027049902826547623, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.026694169268012047, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.02420920506119728, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.023741841316223145, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.017339862883090973, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.017597826197743416, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.016272757202386856, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.012688130140304565, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05123336985707283, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05123336985707283, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.12.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.09936142712831497, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.0882679894566536, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.08140312135219574, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.07137010246515274, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.04610950127243996, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04047267511487007, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.059583116322755814, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.05331159383058548, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.04718722775578499, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.03861810266971588, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.03724299371242523, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.030456092208623886, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.025676630437374115, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.02238871343433857, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.021564612165093422, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.015369811095297337, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.0120000084862113, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.011409792117774487, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.010573736391961575, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.010039178654551506, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.008258509449660778, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.008328622207045555, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.007047239691019058, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.0059978412464261055, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.05331159383058548, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.05331159383058548, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.12.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.08464791625738144, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.07457271218299866, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.0683441236615181, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.05965745449066162, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.0388471893966198, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03373681753873825, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.050359148532152176, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.04534270614385605, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.039873044937849045, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.03232743963599205, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.030980568379163742, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.025655295699834824, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.0217230673879385, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.01878642849624157, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.018025169149041176, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.012857151217758656, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.009926670230925083, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.009392314590513706, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.008654155768454075, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.008158053271472454, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.006775897927582264, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.006758794654160738, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.005715447477996349, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.004645415581762791, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.050359148532152176, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.050359148532152176, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.12.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.19185824692249298, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.17256228625774384, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.16470789909362793, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.14446885883808136, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.08987101912498474, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.08207979798316956, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.10619068145751953, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.09596749395132065, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.09148573130369186, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.07533403486013412, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.07105202227830887, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.054195694625377655, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04592197388410568, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04309283569455147, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04241381585597992, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.027088407427072525, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.02205561101436615, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.021414682269096375, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.019084369763731956, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.01862308569252491, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.013995630666613579, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.01345471478998661, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.012836312875151634, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.008503880351781845, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.054195694625377655, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.054195694625377655, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.12.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.20962904393672943, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.18562078475952148, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.17620067298412323, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.15445590019226074, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.09794670343399048, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.08820542693138123, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.116356261074543, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.10519334673881531, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.10026030987501144, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.08127445727586746, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07676979154348373, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.06004529073834419, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05087818577885628, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04757343977689743, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04677799716591835, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.03023526445031166, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.025310833007097244, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.024546710774302483, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.022090625017881393, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.021575355902314186, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.01639850251376629, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.0168004110455513, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.01531671267002821, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.012325716204941273, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05087818577885628, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05087818577885628, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.12.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.16898709535598755, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1571710705757141, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.15290850400924683, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.1381736844778061, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07989440113306046, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07549551129341125, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.09060564637184143, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08289023488759995, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.08087866753339767, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.07055147737264633, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06759300827980042, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04641929268836975, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.039985645562410355, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.038659267127513885, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03834491968154907, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02337837591767311, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02056017331779003, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.020253784954547882, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.018753837794065475, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.018560145050287247, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012866825796663761, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.013408723287284374, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01244029775261879, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.009980638511478901, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04641929268836975, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04641929268836975, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.12.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.23236511647701263, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.21642330288887024, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2109871655702591, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.1904449611902237, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.10978308320045471, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10397175699472427, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12320615351200104, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11323282122612, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11103887110948563, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09688219428062439, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09235727041959763, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06292006373405457, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.054283592849969864, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.052717555314302444, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05233968421816826, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03146981820464134, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02724529057741165, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.0268524382263422, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.024647196754813194, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.02441091276705265, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016622863709926605, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016672734171152115, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01608199253678322, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.01129305362701416, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.054283592849969864, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.054283592849969864, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.12.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.23011666536331177, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.2070072591304779, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.19730734825134277, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.17449317872524261, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.1063273698091507, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09702161699533463, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12708328664302826, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11497606337070465, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10899266600608826, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.09107682853937149, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08619516342878342, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06467711925506592, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05527600273489952, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05132497847080231, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.05035177245736122, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.03264697268605232, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.02715372107923031, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02667497657239437, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.0242130346596241, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.023597614839673042, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.017836224287748337, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.017873873934149742, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.01653091423213482, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.012847579084336758, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05132497847080231, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05132497847080231, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.13.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.10213909298181534, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.09181688725948334, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.08669238537549973, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.07614187151193619, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.04754956066608429, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04299907013773918, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.057521212846040726, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.05237749591469765, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.0484396331012249, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.040061332285404205, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.03804078698158264, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.029256463050842285, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.02503294311463833, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.0228714719414711, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.022345155477523804, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.01463894173502922, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.0118982819840312, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.011480354703962803, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.010407092981040478, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.01006257627159357, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.007670809514820576, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.00764026353135705, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.006871089804917574, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.005136272870004177, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.05237749591469765, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.05237749591469765, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.13.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.08364420384168625, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.07518991827964783, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.07015972584486008, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.06166722998023033, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.03878398612141609, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03463580459356308, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.04810282588005066, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.043912373483181, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.039543092250823975, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.032745540142059326, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.031173640862107277, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.024396320804953575, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.020994562655687332, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.018659228459000587, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.018081730231642723, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.012200326658785343, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.009689881466329098, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.0092628113925457, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.008484462276101112, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.008097478188574314, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.006369702983647585, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.006301886402070522, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.00556590873748064, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.004146175924688578, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.04810282588005066, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.04810282588005066, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.13.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.20933467149734497, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.1882956326007843, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.18069924414157867, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.158315509557724, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.09775286912918091, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.08979347348213196, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.1138065904378891, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.10322064906358719, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.09948767721652985, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.08194328844547272, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.07698874920606613, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.05798124149441719, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04929068312048912, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04681167006492615, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04621042311191559, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.028967253863811493, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.02381446212530136, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02323269098997116, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.02055460587143898, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.02015010267496109, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.014868661761283875, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.014231311157345772, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.013744518160820007, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.008764226920902729, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04929068312048912, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04929068312048912, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.13.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.22750461101531982, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.2072417140007019, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.19912861287593842, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.17833483219146729, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.10650797188282013, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.09856891632080078, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.12446413189172745, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.11342708766460419, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.10829344391822815, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.09154056757688522, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.08819243311882019, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.06406870484352112, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.0546913668513298, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.05158976837992668, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.050834186375141144, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.032115962356328964, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.027341658249497414, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.026722058653831482, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.024411490187048912, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.023940661922097206, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.01732906885445118, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.017942391335964203, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.016332853585481644, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.013093994930386543, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.05158976837992668, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.05158976837992668, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.13.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.17158488929271698, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1596268117427826, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.15523813664913177, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.14031267166137695, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.08117655664682388, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07666884362697601, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.09198509156703949, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08441497385501862, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.08218464255332947, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.07170307636260986, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06856146454811096, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04721067100763321, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.04066522419452667, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.039182402193546295, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03883276879787445, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02365320362150669, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.020666969940066338, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.020345376804471016, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01881992071866989, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.018607188016176224, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012764004059135914, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.013277272693812847, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.012260274961590767, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.009646973572671413, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04721067100763321, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04721067100763321, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.13.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.23705653846263885, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.22088705003261566, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2152370661497116, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.19453756511211395, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.1120653972029686, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10619194060564041, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.1260979324579239, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11580414324998856, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11338284611701965, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09901244193315506, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0943223088979721, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06442520767450333, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.055486634373664856, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05379866436123848, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05340783670544624, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.032245948910713196, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02780304290354252, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02740365080535412, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.025173520669341087, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.024921493604779243, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.017092501744627953, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01700826920568943, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.016501493752002716, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011473393999040127, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05379866436123848, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05379866436123848, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.13.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.2439960241317749, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.21875041723251343, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.20916424691677094, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.18457379937171936, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.1127302423119545, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.1031016856431961, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.13287678360939026, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.1204565018415451, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.11572746932506561, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.09608486294746399, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.0907578244805336, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06803296506404877, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05786415562033653, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05433708801865578, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.05347524583339691, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.03423671796917915, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.0285765640437603, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.0281657837331295, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.02533121034502983, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.0247786957770586, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.018537364900112152, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.01849132962524891, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.017353331670165062, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.013140566647052765, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.05347524583339691, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.05347524583339691, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.14.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.10424488037824631, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.09404675662517548, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.08792061358690262, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.07763238996267319, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.048578958958387375, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.043544284999370575, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.06060922145843506, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.0549757182598114, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.04951776936650276, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04127885401248932, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.03949272260069847, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.030869731679558754, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.02632269635796547, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.023477280512452126, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.022763244807720184, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.015449536964297295, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.012360786087810993, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.011845728382468224, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.010933089070022106, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.010474280454218388, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.00815539713948965, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.00824003480374813, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.0071189384907484055, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.005706408992409706, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.04951776936650276, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.04951776936650276, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.14.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.0841318741440773, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.07603752613067627, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.0695098266005516, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.061464838683605194, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.038894351571798325, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03407863900065422, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.051064424216747284, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.04636864736676216, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.03965947777032852, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.033343810588121414, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03209308907389641, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.025806216523051262, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.022154830396175385, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.018763301894068718, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.017897984012961388, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.012900080531835556, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.009814596734941006, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.00923085305839777, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.00871579721570015, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.008151985704898834, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.006758652627468109, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.00665030675008893, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.005608730483800173, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.004370537586510181, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.051064424216747284, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.051064424216747284, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.14.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.2194114476442337, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.19817328453063965, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.1897161304950714, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.1665915846824646, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.10261544585227966, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.09422661364078522, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.12187033146619797, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.10919661074876785, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.10440018773078918, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.08657117933034897, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.08166251331567764, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06234155595302582, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.05223219096660614, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04920317605137825, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04848750680685043, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.0311729833483696, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.025239553302526474, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.024584274739027023, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.021982988342642784, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.021497495472431183, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.016114113852381706, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.015443004667758942, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.014659930020570755, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.009926020167768002, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.05223219096660614, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.05223219096660614, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.14.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.2451043426990509, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.21659700572490692, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.20483578741550446, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.17874456942081451, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.11447533965110779, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.10334701091051102, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.1379907876253128, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.12453643232584, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.11774825304746628, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.09409315139055252, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.0903477817773819, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.07104182243347168, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.06020486354827881, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.055550310760736465, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.054439641535282135, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.03559456765651703, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.029506895691156387, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.02864748425781727, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.025540543720126152, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.02481147274374962, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.019040733575820923, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.019608929753303528, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.01755474880337715, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.014199887402355671, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.03559456765651703, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.03559456765651703, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.14.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.1612621545791626, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.15026816725730896, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.14586375653743744, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.13207511603832245, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07646261900663376, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07212352752685547, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08729451894760132, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08010675758123398, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07737758010625839, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06777670234441757, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06485375761985779, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.044831451028585434, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03866015747189522, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.037003252655267715, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03661582991480827, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.022499021142721176, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.019655419513583183, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.019326208159327507, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.017969045788049698, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.01773681491613388, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012186964973807335, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01285128016024828, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011632228270173073, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.009495903737843037, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.044831451028585434, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.044831451028585434, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.14.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.2333511859178543, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.2181156873703003, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.21250203251838684, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.19235879182815552, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11058540642261505, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10485655814409256, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12495282292366028, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11465955525636673, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.1117902547121048, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09811998158693314, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09367595613002777, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06389470398426056, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.055031534284353256, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05319531261920929, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05276918783783913, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.0320056714117527, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02765602432191372, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02723834291100502, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02515741065144539, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.024881061166524887, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.01706412434577942, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.017189128324389458, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.016435060650110245, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.01187122892588377, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05319531261920929, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05319531261920929, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.14.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.22905723750591278, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.20517916977405548, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.19521762430667877, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.1729503870010376, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.10571648925542831, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09612210839986801, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12685558199882507, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11451590061187744, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.1084742546081543, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.09032357484102249, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08582853525876999, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06501206755638123, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05518336594104767, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.0513480007648468, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.0503263846039772, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.03308285027742386, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.02759529836475849, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.027087146416306496, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.024699050933122635, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.024051770567893982, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.01840297505259514, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.018716109916567802, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.01693183183670044, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.013982602395117283, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.0513480007648468, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.0513480007648468, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.15.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.11666414886713028, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.10602928698062897, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.09970837086439133, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.08854666352272034, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.05452314764261246, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.049299050122499466, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.06797266751527786, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.06117452681064606, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.055459439754486084, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04683902859687805, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.04488718509674072, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03467332944273949, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.02940622717142105, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.026408664882183075, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.025664405897259712, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.0174703486263752, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.014032304286956787, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.013494862243533134, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.01255026925355196, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.012077713385224342, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.009334007278084755, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.009463821537792683, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.008200091309845448, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.006772384513169527, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04683902859687805, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04683902859687805, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.15.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.08810259401798248, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08004835993051529, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.07284713536500931, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.06478346884250641, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04067221283912659, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03553685545921326, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.05450048670172691, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.04918598383665085, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.041490521281957626, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.03524171561002731, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03419729322195053, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.027716979384422302, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.023541107773780823, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.019659943878650665, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.01862768456339836, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.013893683440983295, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.010324222035706043, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.00965107511729002, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.009246932342648506, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.008591270074248314, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.007267958018928766, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.007098592352122068, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.005894139409065247, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.004685773514211178, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.04918598383665085, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.04918598383665085, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.15.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.23456385731697083, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.21531420946121216, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.20743320882320404, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.1857202649116516, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.11070002615451813, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.10285980999469757, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.1297006905078888, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.1173039972782135, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.11238881200551987, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.09601468592882156, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.09145043045282364, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06627190858125687, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.0561567060649395, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05312134325504303, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.05239717289805412, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.033112697303295135, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.027156980708241463, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.026501482352614403, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.02409745194017887, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.023611770942807198, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.01705765165388584, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.01640031486749649, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.015800446271896362, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.01031299214810133, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05312134325504303, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05312134325504303, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.15.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.21899835765361786, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.1978422850370407, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.19002032279968262, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.16452579200267792, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.10246901959180832, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.09439628571271896, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.11919937282800674, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.10794547945261002, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.10393045097589493, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.08561493456363678, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07929619401693344, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.06125788763165474, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.0518881231546402, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.0493588000535965, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.048741310834884644, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.03068169765174389, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.02574397437274456, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.025154128670692444, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.02237648144364357, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.021974915638566017, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.016339421272277832, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.01626567915081978, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.015499301254749298, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.011294243857264519, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.0518881231546402, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.0518881231546402, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.15.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.15770180523395538, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.14742374420166016, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.14333368837833405, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.13005302846431732, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.074795663356781, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07083833962678909, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08524028956890106, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07809461653232574, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07567718625068665, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06661991029977798, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06384902447462082, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04373669996857643, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03773334249854088, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03623216971755028, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03588201850652695, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.022020608186721802, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.01930447854101658, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.01900804601609707, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.017728151753544807, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.017513221129775047, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012091686949133873, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012669697403907776, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011600361205637455, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.00945026334375143, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04373669996857643, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04373669996857643, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.15.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.23564356565475464, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.2211351990699768, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.21591977775096893, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.19587956368923187, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11163628846406937, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10627896338701248, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12551924586296082, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11538129299879074, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11277883499860764, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09955240041017532, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09522084146738052, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06404978036880493, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05523828789591789, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.053552087396383286, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05315331742167473, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03202453628182411, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.027533670887351036, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.027150021865963936, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.025073280557990074, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.024819286540150642, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016862886026501656, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016634929925203323, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01628117449581623, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010960489511489868, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.053552087396383286, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.053552087396383286, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.15.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.22647976875305176, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.2024482637643814, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.1927223652601242, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.17141802608966827, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.10423047840595245, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.0946464091539383, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12394973635673523, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11255349963903427, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.1070900559425354, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08901431411504745, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08450054377317429, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06320696324110031, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05404765158891678, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05026623606681824, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04934534803032875, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.031886156648397446, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.02654283121228218, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.026087269186973572, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.023602349683642387, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.023010283708572388, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.017442844808101654, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.017400845885276794, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.01620781607925892, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.012444248422980309, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05404765158891678, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05404765158891678, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.16.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.11590727418661118, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.10605204105377197, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.10048915445804596, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.08960055559873581, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.05413498356938362, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.049528010189533234, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.06562458723783493, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.059830501675605774, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05495759844779968, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04686499014496803, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.04476265236735344, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.0334007665514946, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.028617089614272118, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.026046190410852432, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.025423923507332802, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.016733521595597267, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.01356329396367073, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.013093114830553532, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.012112772092223167, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.011715425178408623, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.00880417414009571, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.008751925081014633, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.007870012894272804, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.005899733863770962, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.05413498356938362, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.05413498356938362, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.16.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.09108318388462067, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08345703780651093, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.07700448483228683, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.06886369735002518, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04217318817973137, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.037533994764089584, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.054534588009119034, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.049892932176589966, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.042882587760686874, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.036832045763731, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.035446710884571075, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.02755543403327465, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.023793211206793785, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.020290987566113472, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.019393084570765495, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.013776133768260479, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.010540287010371685, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.00992585439234972, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.00948261097073555, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.008879275992512703, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.007188939023762941, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.007032295223325491, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.005993238650262356, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.004482063464820385, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.049892932176589966, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.049892932176589966, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.16.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.23983266949653625, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.22201016545295715, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.21503150463104248, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.1933378279209137, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.11326155811548233, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.10614102333784103, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.13152951002120972, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.11893036216497421, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.11476211249828339, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.09913219511508942, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.09454678744077682, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06720900535583496, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.05683869868516922, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05423944443464279, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.05360081046819687, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.03359147533774376, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.027577079832553864, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.027025625109672546, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.024652138352394104, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.024242153391242027, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.01724315993487835, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.01637602411210537, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.016067082062363625, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.010034640319645405, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05423944443464279, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05423944443464279, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.16.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.2320849895477295, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.20865845680236816, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.20059821009635925, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.1781105250120163, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.10818787664175034, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.099416084587574, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.12471873313188553, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.11383410543203354, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.11023689061403275, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.09156417101621628, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.08662991970777512, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.06421130895614624, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05503349006175995, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.05245098099112511, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.05181943252682686, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.0323946475982666, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.027934659272432327, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.027320686727762222, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.024720456451177597, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.02433253638446331, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.017744649201631546, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.01840244047343731, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.01691754162311554, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.013658495619893074, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.05245098099112511, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.05245098099112511, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.16.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.14805495738983154, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.13856996595859528, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.13454031944274902, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.12218108773231506, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07016174495220184, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.06640314310789108, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08033265918493271, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07361839711666107, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.0709780603647232, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06259211897850037, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06004016101360321, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04113738611340523, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03550390899181366, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03396492078900337, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.0335921049118042, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.020667053759098053, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.018040912225842476, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.01773693785071373, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.016575664281845093, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.0163434948772192, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.011241261847317219, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.011804170906543732, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.010727626271545887, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008715817704796791, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04113738611340523, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04113738611340523, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.16.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.22845053672790527, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.21470676362514496, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.20956233143806458, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.19028256833553314, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.1082024797797203, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.1030900627374649, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12179800122976303, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11202165484428406, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.10928066819906235, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09672294557094574, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09250153601169586, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06213308870792389, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05364133045077324, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.0519101582467556, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.051498766988515854, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.031056690961122513, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.026702728122472763, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.026321053504943848, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.024355579167604446, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.02410231903195381, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016307581216096878, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016161657869815826, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.015705900266766548, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010651995427906513, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05364133045077324, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05364133045077324, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.16.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.21916551887989044, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.1945718377828598, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.1843707263469696, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.16384552419185638, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.1004827544093132, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09058739244937897, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12038393318653107, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10945025086402893, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10355225205421448, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08526989817619324, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08110543340444565, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06147739291191101, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05259532108902931, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.04852879047393799, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04750606790184975, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.03106815181672573, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.025667477399110794, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02517629787325859, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.022724425420165062, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.022072019055485725, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.016971994191408157, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.01695694774389267, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.015614386647939682, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.012133860029280186, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05259532108902931, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05259532108902931, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.17.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.09702017903327942, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.08892075717449188, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.08326403051614761, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.0748206302523613, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.045153599232435226, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04079575836658478, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.05668080598115921, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.0516507513821125, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.04597777873277664, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.03949667513370514, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.03810432553291321, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.028803909197449684, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.02470206469297409, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.021792028099298477, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.021063625812530518, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.014429976232349873, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.011458257213234901, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.010961330495774746, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.01034674234688282, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.009877747856080532, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.0075792958959937096, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.007655511610209942, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.006535821128636599, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.005238090176135302, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.0516507513821125, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.0516507513821125, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.17.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.08191414177417755, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.0746629536151886, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.06833002716302872, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.061438750475645065, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.03774355724453926, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03322683274745941, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.05006025731563568, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.04548916965723038, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.038546204566955566, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.03305013105273247, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03211662173271179, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.025281812995672226, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.02165038511157036, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.018216287717223167, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.01730922982096672, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.012625445611774921, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.009527072310447693, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.008940786123275757, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.008605592884123325, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.008032230660319328, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.006586558651179075, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.006493933964520693, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.005386063363403082, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.0042536910623312, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.05006025731563568, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.05006025731563568, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.17.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.19983750581741333, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.18037481606006622, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.17052434384822845, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.15358620882034302, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.09235408902168274, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.08370831608772278, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.1147618442773819, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.10271640121936798, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.09462103247642517, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.08016729354858398, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.07711547613143921, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.058539628982543945, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04904281347990036, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.0443052276968956, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04314000904560089, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.029245130717754364, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.022769056260585785, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.021893257275223732, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.020283309742808342, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.01951954886317253, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.0150186438113451, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.01434130035340786, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.012997607700526714, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.009044409729540348, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04904281347990036, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04904281347990036, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.17.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.2257997989654541, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.20301946997642517, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.19423991441726685, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.17116889357566833, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.10530448704957962, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.09642394632101059, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.12401852011680603, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.1123625859618187, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.10746617615222931, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.08944348245859146, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.08399651944637299, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.0638689175248146, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.054264314472675323, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.05109007656574249, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.05034219101071358, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.032092299312353134, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.0271917674690485, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.026480693370103836, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.02412840910255909, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.02363264374434948, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.017424728721380234, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.018006546422839165, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.016407500952482224, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.013257740996778011, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.054264314472675323, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.054264314472675323, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.17.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.15382196009159088, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.14381498098373413, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.1396910846233368, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.12702886760234833, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07270950824022293, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.06881019473075867, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08308828622102737, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07620883733034134, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07357731461524963, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06495369225740433, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.062314413487911224, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.042544521391391754, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03672734647989273, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.035143643617630005, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03476990759372711, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.021390043199062347, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.018609371036291122, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.018298383802175522, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.017106514424085617, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.016874471679329872, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.011657150462269783, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012097058817744255, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011129972524940968, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008852720260620117, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.042544521391391754, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.042544521391391754, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.17.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.23629136383533478, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.221998929977417, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2167816162109375, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.1970624476671219, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11181584000587463, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10657082498073578, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12595754861831665, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11568234115839005, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.1129145696759224, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.10006839781999588, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0960417091846466, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06434372812509537, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05548098310828209, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05371786653995514, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05330934748053551, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.032316654920578, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.027896175161004066, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.027507085353136063, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.025547264143824577, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.025291267782449722, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.017312465235590935, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.017253439873456955, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.016725366935133934, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011848049238324165, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05371786653995514, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05371786653995514, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.17.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.23277296125888824, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.20573410391807556, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.1946689784526825, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.17207078635692596, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.10690053552389145, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09603912383317947, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12889841198921204, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11631854623556137, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.11022789776325226, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.09005008637905121, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.085513174533844, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06579466909170151, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05594460293650627, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05167718231678009, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.050617765635252, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.033234138041734695, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.027429264038801193, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.026894068345427513, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.024185629561543465, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.023510124534368515, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.0182279571890831, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.018219564110040665, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.0167915690690279, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.013175844214856625, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05167718231678009, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05167718231678009, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.18.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.11159697920084, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.10284176468849182, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.09691385924816132, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.08699440956115723, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.05219624936580658, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04757829010486603, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.06522989273071289, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.05899149924516678, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05303889140486717, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04585108160972595, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.04416908323764801, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.033286046236753464, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.028333332389593124, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.025280551984906197, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.02452143095433712, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.016733825206756592, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.013424481265246868, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.012912550009787083, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.012190655805170536, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.011723286472260952, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.008938467130064964, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.009078599512577057, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.00782180018723011, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.006471633445471525, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05303889140486717, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05303889140486717, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.18.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.09080873429775238, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.0825132355093956, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.074677474796772, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.06732600182294846, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04180391505360603, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03631275147199631, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.05631629005074501, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.0514843687415123, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.042789045721292496, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.03666194528341293, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03579147905111313, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.02847900614142418, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.024534741416573524, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.02019139565527439, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.019041620194911957, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.014246593229472637, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.010592938400804996, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.00984676368534565, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.009584269486367702, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.008834848180413246, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.007452117744833231, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.007347347680479288, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.005969054065644741, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.004772136453539133, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.0514843687415123, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.0514843687415123, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.18.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.21817098557949066, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.19805368781089783, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.18795980513095856, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.17003194987773895, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.10196997970342636, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.09285135567188263, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.12577274441719055, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.1121627613902092, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.10425809025764465, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.08868373185396194, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.08565255254507065, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.0645514577627182, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.053657468408346176, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.048984818160533905, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04783867299556732, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.03234352543950081, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.02512260526418686, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.024219023063778877, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.022392522543668747, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.021634038537740707, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.016693469136953354, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.015652792528271675, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.01456428226083517, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.009827886708080769, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.053657468408346176, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.053657468408346176, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.18.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.19611601531505585, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.17451535165309906, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.16691315174102783, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.14427968859672546, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.0910203754901886, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.08288411796092987, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10581089556217194, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.09621437638998032, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09244778007268906, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07538393139839172, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07042215764522552, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05457882955670357, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04667678102850914, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04434116557240486, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04377315938472748, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.027627406641840935, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.02394750341773033, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.023392587900161743, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.02099853940308094, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.020642446354031563, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.015439669601619244, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.016207506880164146, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.01468756515532732, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.012435068376362324, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04667678102850914, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04667678102850914, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.18.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.14265544712543488, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.13327494263648987, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.12921646237373352, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.11749128997325897, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.06741973012685776, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.06365444511175156, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.07725055515766144, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07099489867687225, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.06826197355985641, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.060155946761369705, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.05766630917787552, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.03952312469482422, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.0341607928276062, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.032549675554037094, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03216953203082085, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.019820544868707657, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.017160383984446526, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.01685040071606636, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.015736378729343414, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.015500363893806934, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.010679199360311031, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.011069446802139282, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01013621874153614, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.007981355302035809, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.03952312469482422, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.03952312469482422, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.18.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.22432978451251984, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.21059300005435944, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.20538759231567383, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.186675027012825, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.10603143274784088, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10088574141263962, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.11982519179582596, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.1101493239402771, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.10717980563640594, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09490736573934555, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09087194502353668, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.061201393604278564, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05278884619474411, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05094105750322342, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.050502434372901917, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.030659247189760208, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.026339607313275337, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02594776079058647, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02409440465271473, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.023816602304577827, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016307314857840538, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01617657206952572, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.015671780332922935, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010914131067693233, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05278884619474411, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05278884619474411, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.18.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.22135470807552338, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.19446787238121033, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.1830441653728485, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.16147533059120178, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.10138344764709473, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09030714631080627, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12372703105211258, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11153721064329147, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10473138839006424, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08512913435697556, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08101708441972733, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06333307176828384, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05381900072097778, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.0491536520421505, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04799271002411842, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.03215419501066208, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.026349464431405067, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.025791587308049202, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.023273451253771782, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.022537147626280785, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.017878437414765358, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.01791355572640896, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.016312770545482635, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.013245079666376114, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05381900072097778, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05381900072097778, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.19.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.11465656012296677, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.10600828379392624, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.09989749640226364, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.0902431458234787, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.053645115345716476, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.0489417165517807, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.06711427122354507, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.060852937400341034, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05449691414833069, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04747826233506203, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.045961130410432816, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03420788794755936, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.029157904908061028, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.025898372754454613, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.025087988004088402, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.017144372686743736, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.013635821640491486, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.013091490603983402, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.012428264133632183, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.011927100829780102, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.00912641640752554, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.009099568240344524, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.007961972616612911, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.006304722744971514, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.053645115345716476, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.053645115345716476, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.19.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.08965000510215759, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08232685178518295, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.07426565140485764, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.06724289059638977, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04138024151325226, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.0359480194747448, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.056460265070199966, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.051676955074071884, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.042223282158374786, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.0367119237780571, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03599132224917412, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.02862909622490406, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.02458825148642063, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.02000809647142887, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.018770568072795868, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.014324580319225788, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.010489951819181442, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.009705272503197193, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.009572271257638931, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.008795415051281452, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.007511377800256014, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.0073248897679150105, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.005949682090431452, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.004749493673443794, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.051676955074071884, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.051676955074071884, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.19.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.22832971811294556, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.2091153860092163, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.20005400478839874, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.18111038208007812, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.1068718433380127, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.09854882955551147, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.12875410914421082, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.11564058065414429, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.10896963626146317, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.09360790997743607, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.08994371443986893, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06586138904094696, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.05534180998802185, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05123921111226082, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.050247516483068466, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.032931022346019745, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.026108600199222565, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.025332389399409294, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.023362506181001663, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.022718077525496483, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.01687576062977314, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.015892764553427696, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.015032531693577766, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.00970155093818903, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05123921111226082, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05123921111226082, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.19.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.21137022972106934, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.1925043761730194, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.18610072135925293, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.1632358729839325, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.09859667718410492, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.09163976460695267, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.11444369703531265, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.10292671620845795, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09996694326400757, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.08387401700019836, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.0796528160572052, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.059008631855249405, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05010414123535156, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04815682768821716, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.047679923474788666, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.02986222133040428, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.026172997429966927, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.025721712037920952, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.02343345619738102, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.0231548510491848, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.016754725947976112, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.01785171777009964, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.016134297475218773, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.01397703867405653, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05010414123535156, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05010414123535156, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.19.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.13911974430084229, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.130160853266716, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.12640529870986938, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.11501024663448334, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.06569506973028183, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.06214374676346779, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.07496024668216705, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.06896717846393585, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.06651759892702103, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.05869591608643532, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.05628379061818123, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.03827868029475212, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.033143602311611176, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03168552368879318, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03133014962077141, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.019200792536139488, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.016632873564958572, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.01634667068719864, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.015251516364514828, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.015037299133837223, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.010289926081895828, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.010616136714816093, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.009803997352719307, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.007561687845736742, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.03827868029475212, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.03827868029475212, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.19.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.21969464421272278, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.20638231933116913, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.20135000348091125, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.18314434587955475, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.1038883775472641, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.09891800582408905, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.11729646474123001, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.10784762352705002, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.10504719614982605, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09307779371738434, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.089162178337574, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.05984938517212868, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05170333757996559, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.04993525519967079, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.04951624944806099, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.030045464634895325, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02594025991857052, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.025562485679984093, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02376994490623474, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.02350855991244316, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016147594898939133, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01608770340681076, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.015559244900941849, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011065609753131866, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05170333757996559, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05170333757996559, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.19.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.20882654190063477, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.18293501436710358, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.1708936095237732, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.1502760797739029, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09569812566041946, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08449085056781769, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.11814698576927185, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10681642591953278, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09894713014364243, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.07988262176513672, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.0761018916964531, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06059737503528595, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05178295448422432, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.046530455350875854, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04524950683116913, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.03089858964085579, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.025215450674295425, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02453891932964325, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.02227173186838627, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.02141343243420124, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.017463907599449158, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.017549382522702217, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.015780964866280556, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.013132983818650246, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05178295448422432, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05178295448422432, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.20.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.11444326490163803, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.1055014580488205, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.09825535863637924, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.0891423374414444, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.053389087319374084, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04801574721932411, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.06906714290380478, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.06236892566084862, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05436888337135315, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.0473749116063118, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.04616570472717285, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03520072624087334, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.02987632527947426, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.0258669164031744, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.024836190044879913, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.01766885630786419, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.01373202446848154, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.013081732206046581, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.012574264779686928, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.011948813684284687, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.009422007016837597, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.009440392255783081, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.007976274937391281, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.006634380668401718, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.053389087319374084, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.053389087319374084, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.20.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.09047260880470276, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08232685923576355, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.07258374243974686, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.06598429381847382, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04149811714887619, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.035029057413339615, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.0590379424393177, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.053789906203746796, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04254136607050896, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.0367581807076931, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.036301515996456146, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.029818270355463028, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.025577928870916367, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.020057296380400658, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.018573004752397537, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.014916702173650265, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.010609252378344536, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.009675871580839157, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.009685897268354893, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.008738551288843155, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.00780309597030282, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.00764507194980979, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.005905516445636749, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.004975953139364719, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.053789906203746796, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.053789906203746796, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.20.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.22678525745868683, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.2049216777086258, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.19451342523097992, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.1758013367652893, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.10528506338596344, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.09566793590784073, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.12972676753997803, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.11574698984622955, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.10772284865379333, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.09136522561311722, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.08834898471832275, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06667500734329224, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.05533298850059509, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05051372945308685, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.049303170293569565, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.03336772322654724, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.025823956355452538, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.024896109476685524, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.02298101596534252, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.02219715341925621, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.0171296838670969, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.016026947647333145, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.014846895821392536, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.00993427261710167, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05051372945308685, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05051372945308685, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.20.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.22984875738620758, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.2021685689687729, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.19181154668331146, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.16082192957401276, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.10723299533128738, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.0965796411037445, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.12582704424858093, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.11445076018571854, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.10925163328647614, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.08504299819469452, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.079674132168293, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.06471829116344452, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05526352301239967, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.051943499594926834, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.05112643912434578, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.032539453357458115, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.027630910277366638, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.02689254656434059, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.023370549082756042, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.022835325449705124, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.017570000141859055, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.018266256898641586, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.016515854746103287, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.013427401892840862, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.051943499594926834, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.051943499594926834, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.20.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.1456148475408554, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1367599219083786, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.13323040306568146, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.12116726487874985, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.06882210820913315, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.06545256078243256, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.07771433144807816, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07165767252445221, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.06958616524934769, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06164563074707985, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.05905027315020561, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.03964614123106003, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03439237177371979, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.033128201961517334, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.032827407121658325, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.019848518073558807, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.017282966524362564, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.017022754997015, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.015853779390454292, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.015668600797653198, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.010559497401118279, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.010853266343474388, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01013503223657608, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.007590027526021004, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.03964614123106003, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.03964614123106003, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.20.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.22867681086063385, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.21522530913352966, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2104169726371765, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.19131822884082794, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.10803673416376114, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10315576940774918, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12112215906381607, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11150986701250076, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.10908965766429901, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09691968560218811, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09275484830141068, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.061633262783288956, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05334612727165222, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05178083851933479, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05141318216919899, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.030911065638065338, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.026616882532835007, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.026269732043147087, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.024352600798010826, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.02411971427500248, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.01639235019683838, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016056003049016, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.015872187912464142, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010563229210674763, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05334612727165222, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05334612727165222, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.20.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.20761169493198395, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.18257753551006317, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.17165516316890717, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.15128041803836823, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09508275985717773, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08455172181129456, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.11502408981323242, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10470683872699738, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09812071174383163, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.07946012169122696, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.07559830695390701, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05888964235782623, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05035287141799927, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.04591505974531174, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.044816240668296814, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.029812291264533997, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.024339772760868073, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02380193956196308, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.021334176883101463, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.020627474412322044, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.016428286209702492, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.016223173588514328, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.014982105232775211, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.011625177226960659, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05035287141799927, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05035287141799927, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.21.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.10928948223590851, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.10006524622440338, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.09303509443998337, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.08444321900606155, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.050899043679237366, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04556712508201599, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.0658453032374382, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.05945511534810066, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05192425101995468, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.044911228120326996, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.0437929630279541, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03360670059919357, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.0284200981259346, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.024586215615272522, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.023609735071659088, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.016865352168679237, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.012933803722262383, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.012287275865674019, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.011757158674299717, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.011147325858473778, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.009004323743283749, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.008747897110879421, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.007616496179252863, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.005942426156252623, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05192425101995468, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05192425101995468, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.21.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.09122080355882645, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.0827295333147049, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.07359453290700912, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.06687337160110474, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04202055186033249, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03561447188258171, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.058935899287462234, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.053484562784433365, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04306158050894737, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.03705049678683281, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03659228980541229, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.02994987927377224, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.0255876611918211, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.02033252641558647, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.01890396699309349, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.014943400397896767, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.010692759416997433, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.009786752983927727, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.009723183698952198, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.008815012872219086, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.007780096028000116, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.00760724488645792, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.005941588431596756, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.004923895932734013, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.053484562784433365, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.053484562784433365, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.21.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.21679089963436127, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.19406531751155853, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.18209117650985718, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.16445887088775635, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.1000591441988945, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.08938778936862946, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.12616044282913208, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.11263936012983322, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.1027204692363739, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.08627913892269135, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.08350113779306412, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06491731852293015, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.053830377757549286, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04797002300620079, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04648280143737793, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.03252711519598961, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.024591311812400818, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02347838319838047, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.021786458790302277, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.020803524181246758, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.01671263948082924, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.015561647713184357, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.014025588519871235, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.00958433747291565, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.053830377757549286, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.053830377757549286, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.21.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.2303474247455597, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.200810506939888, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.19055958092212677, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.16070528328418732, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.10687829554080963, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.09497074782848358, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.12309370189905167, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.11327061057090759, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.10911892354488373, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.08520635962486267, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07752732932567596, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.06319588422775269, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.054441068321466446, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.05147664248943329, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.05077638849616051, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.03168822452425957, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.026984339579939842, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.026227418333292007, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.02272414043545723, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.022253308445215225, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.01691633090376854, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.017261743545532227, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.01593218930065632, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.012141730636358261, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.05147664248943329, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.05147664248943329, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.21.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.14371661841869354, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.13515472412109375, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.13193932175636292, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.12005634605884552, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.06801272928714752, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.0648375153541565, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.07643576711416245, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07048289477825165, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.06867460906505585, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.060972027480602264, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.058374855667352676, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.03906713053584099, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03381593897938728, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03271247446537018, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.032450467348098755, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.01957164704799652, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.017039747908711433, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.016797924414277077, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.015634218230843544, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.015469217672944069, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.010480672121047974, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.010633102618157864, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01012298185378313, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.007406961638480425, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.03906713053584099, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.03906713053584099, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.21.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.23519429564476013, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.22171618044376373, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.21705223619937897, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.19742456078529358, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.1113191619515419, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10647161304950714, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12421892583370209, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11450593173503876, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11228305846452713, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09994170069694519, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0955977588891983, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06342466175556183, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05481244996190071, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.053373727947473526, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05303028225898743, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03174586966633797, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.0274603720754385, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02712080255150795, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.025146547704935074, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.024931227788329124, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.01685454323887825, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016577906906604767, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01638275384902954, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010981686413288116, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.053373727947473526, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.053373727947473526, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.21.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.2052726000547409, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.18048812448978424, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.16935867071151733, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.15033024549484253, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.0936426892876625, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08314555883407593, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.11404795944690704, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10393796861171722, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09687253832817078, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.07892251014709473, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.07514623552560806, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.057932473719120026, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.049950361251831055, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.04526190087199211, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.044099144637584686, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.029377024620771408, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.02403027005493641, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.023432869464159012, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.021204207092523575, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.020439133048057556, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.016207829117774963, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.016114214435219765, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.014700859785079956, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.011512459255754948, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.049950361251831055, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.049950361251831055, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.22.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.10957325994968414, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.09994599968194962, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.09271310269832611, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.08412954956293106, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.050933849066495895, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.045396242290735245, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.06604039669036865, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.05968862771987915, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.052117422223091125, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.044849272817373276, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.04372585937380791, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03375091031193733, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.02858457900583744, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.024624032899737358, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.023603098466992378, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.01696127839386463, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.012966522015631199, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.012304143980145454, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.01177222654223442, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.011138373054564, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.009057406336069107, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.008805310353636742, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.007635244634002447, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.005993344821035862, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.052117422223091125, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.052117422223091125, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.22.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.09877453744411469, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08956434577703476, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.07973670959472656, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.0725289136171341, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.045579712837934494, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03869170323014259, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.06360765546560287, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.05781104788184166, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04672587662935257, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.040120042860507965, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03957948833703995, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.032331496477127075, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.027652788907289505, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.02208142727613449, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.020574579015374184, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.016180474311113358, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.011671765707433224, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.010717642493546009, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.010614946484565735, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.009668298065662384, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.008505980484187603, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.00833604484796524, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.006594098638743162, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.0055103180930018425, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04672587662935257, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04672587662935257, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.22.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.2146129459142685, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.19117869436740875, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.1782289296388626, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.16099302470684052, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.09880749136209488, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.08744488656520844, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.12621107697486877, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.11253862082958221, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.10159089416265488, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.08476818352937698, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.08218155801296234, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06464135646820068, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.053708694875240326, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04738851636648178, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04576430097222328, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.032327935099601746, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.024311577901244164, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.023111021146178246, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.02147120237350464, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.02039506658911705, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.01654907315969467, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.01550542563199997, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.013749765232205391, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.009554313495755196, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.053708694875240326, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.053708694875240326, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.22.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.21571846306324005, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.18611060082912445, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.17531917989253998, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.14945052564144135, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.1008707731962204, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.09003767371177673, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.11816170811653137, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.10780710726976395, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.1029682606458664, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07973866164684296, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.0729612410068512, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.06060759350657463, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05197511985898018, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04882344603538513, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04804620519280434, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.030374085530638695, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.02580353058874607, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.025104764848947525, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.021735278889536858, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.02121789939701557, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.016415836289525032, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.016898665577173233, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.015385137870907784, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.0122162364423275, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05197511985898018, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05197511985898018, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.22.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.15724799036979675, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.14801055192947388, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.14456278085708618, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.13154108822345734, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07440970838069916, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07095904648303986, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08344663679599762, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07703141123056412, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07512904703617096, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06670434027910233, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06390923261642456, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.0426163449883461, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.036944154649972916, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.035757336765527725, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.035480011254549026, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.021316412836313248, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.01857135072350502, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.018309658393263817, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.017029915004968643, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.01685103215277195, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.011337440460920334, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01150318793952465, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.010952526703476906, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.007916001603007317, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.0426163449883461, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.0426163449883461, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.22.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.23904350399971008, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.2253643125295639, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2206437885761261, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.20078004896640778, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11306136846542358, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10818864405155182, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12597690522670746, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11627401411533356, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11410269141197205, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.1015218049287796, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09717012196779251, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06427565962076187, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05563081428408623, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05418301373720169, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05384523794054985, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.0321015864610672, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.0277982447296381, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02745252102613449, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.025441676378250122, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.02522149682044983, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016887299716472626, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01666787452995777, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.016404885798692703, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010895858518779278, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05418301373720169, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05384523794054985, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.22.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.21165591478347778, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.18738839030265808, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.17656877636909485, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.1568971425294876, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.0969102680683136, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.0867493748664856, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.11748203635215759, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.1068744957447052, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09991614520549774, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08211133629083633, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.07816799730062485, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05971162021160126, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05125107616186142, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.0467614084482193, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04563642665743828, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.0302292350679636, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.024670062586665154, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02412590943276882, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.021808689460158348, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.021095577627420425, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.016624247655272484, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.01631348580121994, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.015167895704507828, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.011537257581949234, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05125107616186142, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05125107616186142, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.23.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.11144105345010757, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.10212317854166031, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.09526549279689789, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.0864814817905426, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.05190658941864967, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04660457372665405, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.06652797758579254, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.06018252298235893, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.0529869981110096, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04579433798789978, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.044573768973350525, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.033835429698228836, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.028793832287192345, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.025053590536117554, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.02409978397190571, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.016952916979789734, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.013133362866938114, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.012500988319516182, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.011926903389394283, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.011330543085932732, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.00897444412112236, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.008796442300081253, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.007630597334355116, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.005924706347286701, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.0529869981110096, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.0529869981110096, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.23.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.09834861755371094, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08901286870241165, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.08058246970176697, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.07312973588705063, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04527607560157776, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03916103392839432, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.06106571853160858, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.0556943379342556, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04643521085381508, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.039720963686704636, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03891979157924652, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.03095146454870701, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.02656635455787182, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.02190260961651802, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.020674949511885643, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.015520544722676277, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.011497283354401588, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.010694186203181744, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.010401714593172073, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.00962082203477621, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.008115554228425026, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.00796891562640667, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.006500585936009884, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.005228078458458185, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04643521085381508, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04643521085381508, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.23.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.22936303913593292, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.20820878446102142, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.19763407111167908, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.17858092486858368, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.106652170419693, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.0970284715294838, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.1297520250082016, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.11741314083337784, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.10894081741571426, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.09268859773874283, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.0893661305308342, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06626816838979721, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.056016065180301666, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05111228674650192, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04990869015455246, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.033119283616542816, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.026141654700040817, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.025206558406352997, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.023295940831303596, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.02249949984252453, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.016994085162878036, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.016197707504034042, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.014908474870026112, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.01002773828804493, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05111228674650192, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05111228674650192, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.23.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.2108493149280548, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.18377168476581573, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.17388375103473663, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.14398780465126038, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.0971338301897049, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.08699032664299011, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.11393299698829651, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.10438046604394913, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09974299371242523, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07697572559118271, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07006049156188965, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05831246078014374, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05016935244202614, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04685966670513153, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04606541618704796, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.029204130172729492, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.024611441418528557, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.023848094046115875, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.02069205790758133, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.02015066333115101, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.015600213780999184, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.015930181369185448, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.014527887105941772, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.011241447180509567, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05016935244202614, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05016935244202614, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.23.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.16388827562332153, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1541646271944046, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.15059228241443634, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.13702112436294556, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07755082100629807, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.0739508718252182, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08709289133548737, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08020004630088806, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07829032838344574, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06949421763420105, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0665753185749054, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04442601650953293, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.038472916930913925, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.037278834730386734, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03699669614434242, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02231348305940628, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.019385285675525665, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.01911500096321106, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.0177652295678854, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.017593974247574806, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012034471146762371, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012025880627334118, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01165598165243864, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008325144648551941, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04442601650953293, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04442601650953293, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.23.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.24376733601093292, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.22961558401584625, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2248132824897766, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.20449243485927582, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11534594744443893, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.11028263717889786, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12855149805545807, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11855839192867279, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11639999598264694, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.10343199223279953, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09889744967222214, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06556588411331177, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05673935264348984, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05528217926621437, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05493824928998947, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03280350938439369, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.028353199362754822, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.027995241805911064, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.025915885344147682, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.025698309764266014, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.017327969893813133, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016974076628684998, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01683942787349224, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011072498746216297, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03280350938439369, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03280350938439369, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.23.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.21763044595718384, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.1923941969871521, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.18110594153404236, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.16124969720840454, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09962042421102524, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08897606283426285, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.120120570063591, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11001227796077728, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10273993015289307, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.0843755230307579, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08043208718299866, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.061254099011421204, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05274949595332146, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.0480387881398201, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04686591029167175, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.030863473191857338, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.025283673778176308, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02469497174024582, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.022322557866573334, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.021569600328803062, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.01666448265314102, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.01665089838206768, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.015107650309801102, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.011665080673992634, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05274949595332146, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05274949595332146, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.24.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.11982377618551254, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.1102212518453598, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.10303369909524918, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.09354404360055923, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.05602928623557091, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.05050589516758919, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.07108020782470703, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.06468871235847473, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05703223869204521, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.049530621618032455, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.048145826905965805, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.036255158483982086, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.030983267351984978, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.02705051563680172, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.026059577241539955, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.018155114725232124, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.014207431115210056, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.013546726666390896, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.012927585281431675, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.012312479317188263, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.009571581147611141, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.009526669979095459, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.008186751045286655, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.006474317982792854, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.049530621618032455, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.049530621618032455, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.24.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.105857253074646, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.09681237488985062, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.08756309002637863, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.07963361591100693, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.049108684062957764, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.0425257682800293, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.0668167695403099, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.060869138687849045, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.050196968019008636, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.04343812167644501, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.04267967864871025, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.03392903879284859, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.029076386243104935, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.0237272996455431, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.022276148200035095, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.016949808225035667, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.012385053560137749, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.011477666907012463, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.011267125606536865, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.010363537818193436, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.008854063227772713, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.008587241172790527, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.007035992108285427, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.005521143786609173, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.050196968019008636, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.050196968019008636, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.24.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.23770567774772644, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.21765241026878357, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.2083035111427307, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.1884317398071289, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.11104048788547516, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.1023450717329979, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.13261345028877258, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.12016896903514862, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.11312233656644821, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.09735787659883499, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.09344839304685593, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06752979010343552, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.05735821649432182, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.053194575011730194, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.05217171087861061, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.03375744819641113, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.027144575491547585, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02631070837378502, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.02430320531129837, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.023617621511220932, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.017297660931944847, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.016562573611736298, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.015533704310655594, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.01016708742827177, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.053194575011730194, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.053194575011730194, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.24.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.1822643131017685, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.1650846302509308, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.15686260163784027, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.1310717761516571, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.0854109600186348, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.0780898779630661, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10173016786575317, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.09326981008052826, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.08658351749181747, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07047121971845627, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.0642717108130455, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05195440351963043, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04468100890517235, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04109637439250946, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04020283371210098, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.025976350530982018, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.021295061334967613, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.020598160102963448, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.018290936946868896, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.017697649076581, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.013676178641617298, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.013509802520275116, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.012464023195207119, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.00896008126437664, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05195440351963043, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05195440351963043, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.24.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.1644883006811142, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1546863466501236, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.15115080773830414, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.13738568127155304, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07785758376121521, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07422785460948944, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08722874522209167, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08050209283828735, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07861840724945068, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06969896703958511, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06669887900352478, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.044481806457042694, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03859532251954079, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03740832582116127, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.037124037742614746, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02226800099015236, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.01938902959227562, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.01912275142967701, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.017748164013028145, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.017569800838828087, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.011835074983537197, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01194112841039896, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01145770400762558, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008156314492225647, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.044481806457042694, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.044481806457042694, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.24.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.24699078500270844, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.23270951211452484, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2278493493795395, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.20718933641910553, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11691351979970932, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.1118030995130539, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.13060861825942993, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.12017872929573059, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11802347749471664, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.10484223067760468, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.1002972275018692, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06657490879297256, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.057536207139492035, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05607863888144493, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.055738404393196106, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.033394187688827515, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.028808271512389183, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02845500037074089, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.026330342516303062, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.026110786944627762, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.01778838224709034, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01731196418404579, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01731100305914879, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011376766487956047, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.033394187688827515, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.033394187688827515, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.24.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.2215447723865509, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.19618546962738037, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.1850007325410843, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.16475047171115875, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.1013823002576828, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09081171452999115, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12227802723646164, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11167708784341812, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10460495948791504, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08610530942678452, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.0819515660405159, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06198311969637871, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05348736792802811, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.04883664473891258, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04767434298992157, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.031283020973205566, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.025610540062189102, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02503371611237526, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.022622372955083847, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.02187088504433632, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.016980202868580818, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.016727423295378685, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.01545341033488512, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.0115788159891963, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05348736792802811, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05348736792802811, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.25.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.12556305527687073, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.1159956231713295, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.10893036425113678, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.09871136397123337, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.0588366873562336, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.05336252599954605, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.07406643033027649, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.06730064749717712, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.059843894094228745, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.052128471434116364, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.05057583749294281, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03771689906716347, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.03223186358809471, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.02836601994931698, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.027398614212870598, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.018879787996411324, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.014800465665757656, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.01413826085627079, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.013466855511069298, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.012852605432271957, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.00989446323364973, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.00976674072444439, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.00851232185959816, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.006496401038020849, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.052128471434116364, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.052128471434116364, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.25.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.10687704384326935, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.09802205115556717, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.08918632566928864, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.08103649318218231, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.049645498394966125, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.043320197612047195, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.0663713738322258, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.060791224241256714, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.05062929913401604, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.04395134374499321, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.04291936010122299, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.03364581987261772, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.02906634286046028, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.02394874580204487, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.022595783695578575, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.016815830022096634, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.012461531907320023, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.011590085923671722, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.01133390050381422, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.010473296977579594, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.008740531280636787, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.00853308942168951, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.007042492739856243, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.00542002497240901, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.05062929913401604, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.05062929913401604, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.25.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.239474356174469, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.21975542604923248, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.2106943130493164, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.1905088871717453, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.11187077313661575, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.10350967943668365, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.1340796798467636, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.12072005122900009, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.11383438855409622, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.0981183797121048, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.09425124526023865, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06833413988351822, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.0576237253844738, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.053602997213602066, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.052614402025938034, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.034064799547195435, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.02730974368751049, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02648700773715973, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.024464676156640053, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.023826181888580322, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.01750899851322174, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.016577132046222687, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.01571711339056492, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.010110314004123211, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.053602997213602066, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.053602997213602066, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.25.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.19715285301208496, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.17291495203971863, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.1650565266609192, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.136726513504982, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.09188529849052429, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.08323013037443161, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10596141219139099, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.09645688533782959, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09369111061096191, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07316645234823227, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.06495869904756546, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05389346927404404, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04622245207428932, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04414921998977661, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04363996908068657, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.027014337480068207, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.022814135998487473, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.02226768620312214, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.01903679035604, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.018694404512643814, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.014341474510729313, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.014087200164794922, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.013615787029266357, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.009435413405299187, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05389346927404404, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05389346927404404, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.25.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.16994985938072205, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.15970930457115173, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.15601615607738495, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.14184722304344177, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.0805244967341423, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07675040513277054, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.0902675986289978, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08324528485536575, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.08130747824907303, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.07200834900140762, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06889088451862335, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.046129852533340454, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03994375467300415, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03870568424463272, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03841806575655937, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.023116545751690865, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.020098570734262466, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.019830169156193733, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.018395040184259415, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.01821562834084034, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.01236339844763279, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012440857477486134, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011966140940785408, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008563359268009663, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.046129852533340454, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.046129852533340454, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.25.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.25007277727127075, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.23540708422660828, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2304924726486206, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.209419846534729, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11847733706235886, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.11326760798692703, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.13172279298305511, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.12175863236188889, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11957496404647827, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.10608039051294327, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.10131092369556427, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06711410731077194, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05823390185832977, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05675370618700981, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.056398868560791016, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03351031243801117, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.029027877375483513, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02866833098232746, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.026475690305233, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.026250038295984268, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.017450524494051933, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.017264489084482193, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.016953660175204277, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011098220013082027, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03351031243801117, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03351031243801117, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.25.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.22341962158679962, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.19779229164123535, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.18608558177947998, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.16591614484786987, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.10212691873311996, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09126339107751846, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12343217432498932, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11322055757045746, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10547378659248352, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08701349794864655, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08286767452955246, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06288120150566101, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.054267920553684235, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.049180008471012115, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04791262000799179, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.03156855329871178, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.02575099654495716, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.025119254365563393, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.02278914488852024, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.021966073662042618, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.0168871209025383, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.01683790795505047, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.015216715633869171, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.0115489661693573, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.054267920553684235, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.054267920553684235, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.26.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.12053371965885162, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.11076760292053223, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.10318152606487274, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.093513622879982, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.056388288736343384, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.05059025064110756, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.07188374549150467, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.06559693813323975, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05744076520204544, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04981626942753792, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.048390377312898636, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03664573282003403, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.031455881893634796, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.027232738211750984, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.026162054389715195, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.018347304314374924, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.014282265678048134, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.013571036979556084, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.012987343594431877, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.012313530780375004, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.00958429928869009, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.009602971374988556, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.008095034398138523, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.00645834906026721, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04981626942753792, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04981626942753792, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.26.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.10353386402130127, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.0944262221455574, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.08451580256223679, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.07670380175113678, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04793551564216614, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.04102899506688118, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.06642840802669525, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.06052868440747261, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04905562102794647, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.042328137904405594, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.041505519300699234, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.0335739329457283, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.02885543182492256, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.02319573424756527, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.021646760404109955, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.016795940697193146, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.012149459682404995, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.011178635992109776, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.011039599776268005, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.010068141855299473, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.008812550455331802, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.008530905470252037, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.006894449237734079, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.005481294821947813, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04905562102794647, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04905562102794647, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.26.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.23001907765865326, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.20809012651443481, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.1968235969543457, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.17770859599113464, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.1067451760172844, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.09663660824298859, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.13227549195289612, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.11854508519172668, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.10938778519630432, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.09268905967473984, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.08947689831256866, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06777966022491455, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.05658711493015289, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05123372748494148, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.049881577491760254, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.033908549696207047, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.026182422414422035, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02514859102666378, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.023289721459150314, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.02240009233355522, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.017397766932845116, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.016305483877658844, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.014977026730775833, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.009975658729672432, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05123372748494148, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05123372748494148, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.26.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.18919837474822998, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.17131733894348145, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.16575421392917633, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.14084136486053467, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.08916810154914856, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.08256521075963974, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10014063864946365, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.09214363992214203, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09038402140140533, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07386420667171478, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.06727684289216995, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.051175300031900406, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.044151365756988525, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04279990494251251, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04248662292957306, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.025581825524568558, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.02205626666545868, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.021664096042513847, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.018968136981129646, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.018748996779322624, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.013436826877295971, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.013422160409390926, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.012991671450436115, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.008961319923400879, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.051175300031900406, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.051175300031900406, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.26.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.17453797161579132, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.16400179266929626, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.16019724309444427, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.14550961554050446, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.08278516680002213, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07890249043703079, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.09273967146873474, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08559832721948624, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.0836155116558075, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.0739915668964386, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.07065751403570175, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04734564572572708, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.041043464094400406, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03977876156568527, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.039477959275245667, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.023681458085775375, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.020611200481653214, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.020329318940639496, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.018839513882994652, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.018649715930223465, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012530026957392693, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012689477764070034, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01211528293788433, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008654211647808552, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04734564572572708, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04734564572572708, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.26.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.2512197196483612, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.23645330965518951, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.23150409758090973, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.21027915179729462, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11918137967586517, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.11390683054924011, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.13286268711090088, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.12249982357025146, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.12028618156909943, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.10664454847574234, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.10184229910373688, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06776771694421768, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.0586634986102581, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.057152822613716125, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.0567997507750988, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.033883705735206604, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.029367459937930107, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.029003577306866646, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.026795823127031326, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.026569750159978867, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.01788564771413803, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01764919050037861, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01738482527434826, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011606301181018353, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.033883705735206604, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.033883705735206604, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.26.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.22692427039146423, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.20162063837051392, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.19002698361873627, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.1692453920841217, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.10395738482475281, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09318237006664276, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12549404799938202, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11499937623739243, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10729505121707916, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08877211809158325, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08449645340442657, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06395233422517776, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.0551777146756649, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05010344088077545, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04885312542319298, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.03233486786484718, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.026354603469371796, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.025724049657583237, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.023392533883452415, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.022584237158298492, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.017627809196710587, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.017340756952762604, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.015992915257811546, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.012071800418198109, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05010344088077545, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05010344088077545, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.27.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.12452154606580734, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.11453695595264435, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.10809440165758133, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.09805474430322647, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.05838008224964142, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.053114622831344604, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.07246813923120499, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.06587137281894684, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05941365286707878, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.05149584263563156, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.0498381033539772, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03697812929749489, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.031559258699417114, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.028150558471679688, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.027310607954859734, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.01852261647582054, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.014705331064760685, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.01410877425223589, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.013334398157894611, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.012790704146027565, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.009739626199007034, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.009637013077735901, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.008491975255310535, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.0064606983214616776, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.05149584263563156, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.05149584263563156, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.27.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.10733161121606827, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.0989459753036499, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.09235842525959015, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.0837026834487915, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.05014695227146149, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.04509536176919937, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.06361188739538193, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.05822722986340523, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.05106978118419647, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.04435816779732704, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.04289370775222778, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.03225512057542801, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.02784596011042595, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.024184204638004303, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.023253321647644043, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.016180627048015594, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.012574711814522743, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.011957300826907158, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.011417492292821407, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.010832133702933788, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.008471757173538208, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.008316163904964924, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.007201797794550657, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.005438519176095724, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.05106978118419647, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.05106978118419647, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.27.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.22618617117404938, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.20783376693725586, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.19920216500759125, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.18024316430091858, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.10609455406665802, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.09800407290458679, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.125311478972435, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.11450386047363281, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.10792074352502823, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.09308404475450516, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.08924183249473572, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06385636329650879, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.05472434312105179, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05085771530866623, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04990743100643158, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.031863462179899216, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.02589772827923298, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02509617619216442, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.023191772401332855, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.022561104968190193, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.01634899154305458, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.015685463324189186, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.014880356378853321, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.009533638134598732, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05085771530866623, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05085771530866623, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.27.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.20709311962127686, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.18371646106243134, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.1756037026643753, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.14739514887332916, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.09714093059301376, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.08835341036319733, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.11171499639749527, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.10243193060159683, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09894679486751556, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07798339426517487, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07212778180837631, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.057404011487960815, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04943528771400452, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04704694449901581, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.046497926115989685, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.02893695794045925, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.02502412721514702, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.02444585971534252, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.021348239853978157, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.020963555201888084, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.015841927379369736, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.016442058607935905, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.01509474590420723, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.012123816646635532, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04943528771400452, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04943528771400452, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.27.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.1696636974811554, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.15925085544586182, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.1554262638092041, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.14101669192314148, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.0805472731590271, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07662193477153778, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.09046194702386856, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.0834740698337555, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.08137940615415573, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.07186915725469589, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06864246726036072, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04620400816202164, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.04008869826793671, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.038752127438783646, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.038441333919763565, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02313356287777424, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02013331465423107, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.019838061183691025, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01839279942214489, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.018199166283011436, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012288039550185204, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012490564025938511, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011849011294543743, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.00861911941319704, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04620400816202164, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04620400816202164, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.27.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.243917778134346, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.22943468391895294, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.22448599338531494, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.20380203425884247, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.1157817617058754, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.11056102067232132, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12916427850723267, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11916787177324295, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11689942330121994, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.1034947857260704, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09887633472681046, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06598954647779465, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05710281804203987, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.055572204291820526, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05520958453416824, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.0330083966255188, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.028569214046001434, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.0281955786049366, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02604973316192627, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.025813542306423187, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.017437513917684555, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.017210720106959343, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.016923565417528152, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011341173201799393, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.0330083966255188, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.0330083966255188, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.27.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.22700971364974976, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.20089219510555267, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.18851624429225922, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.16792292892932892, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.10385089367628098, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09249718487262726, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.1275073140859604, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11588882654905319, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10735159367322922, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08841566741466522, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.0842674970626831, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06474127620458603, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.055522285401821136, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05008291080594063, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04873017966747284, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.032673366367816925, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.026347137987613678, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.025668693706393242, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.023331694304943085, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.022461935877799988, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.017774414271116257, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.017407815903425217, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.015970418229699135, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.012096922844648361, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05008291080594063, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05008291080594063, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.28.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.11572492867708206, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.10626456886529922, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.09801332652568817, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.08891233056783676, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.054125506430864334, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04799175262451172, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.07131495326757431, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.0644075870513916, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05519085004925728, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04779224097728729, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.04679208621382713, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03645377233624458, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.030863337218761444, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.02618774026632309, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.024978477507829666, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.01823790743947029, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.013802701607346535, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.013025574386119843, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.012560619041323662, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.011811941862106323, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.009593253023922443, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.009462372399866581, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.007903127931058407, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.0064032673835754395, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.054125506430864334, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.054125506430864334, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.28.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.09731120616197586, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08851464837789536, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.07734818011522293, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.07040656358003616, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04492749646306038, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.037374380975961685, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.06498654931783676, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.059081148356199265, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04605111479759216, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.039760228246450424, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03935680538415909, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.03285251930356026, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.02815963886678219, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.021789396181702614, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.020010769367218018, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.016488082706928253, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.011497395113110542, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.01039078924804926, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.010493731126189232, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.009369997307658195, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.008600966073572636, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.008353759534657001, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.006392995826900005, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.005371517036110163, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04605111479759216, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04605111479759216, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.28.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.23069420456886292, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.2078019231557846, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.19543907046318054, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.17634005844593048, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.10710982978343964, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.09611407667398453, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.1343490481376648, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.1203572005033493, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.10976240783929825, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.0927569717168808, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.0896335318684578, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06881999224424362, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.05757840350270271, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05147475376725197, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.049937356263399124, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.034555140882730484, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.026490207761526108, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02535529062151909, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.023588579148054123, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.022594979032874107, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.017789918929338455, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.01691047102212906, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.015112156048417091, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.010707507841289043, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05147475376725197, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05147475376725197, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.28.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.21962280571460724, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.18841341137886047, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.1776753067970276, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.15232819318771362, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.10183048993349075, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.09018845111131668, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.1187899112701416, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.10850027948617935, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.10426682978868484, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07932937145233154, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07580279558897018, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.06088311970233917, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.052138980478048325, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04907821863889694, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04832329601049423, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.03049134463071823, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.025779221206903458, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.024988699704408646, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.02140943519771099, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.020911216735839844, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.016389766708016396, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.01662471890449524, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.015413561835885048, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.011755919083952904, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.052138980478048325, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.052138980478048325, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.28.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.15797848999500275, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1480463296175003, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.14414793252944946, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.1307307481765747, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07497101277112961, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07113786041736603, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08464109897613525, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.0781075581908226, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07579217851161957, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.0668020099401474, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06375230103731155, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04328674077987671, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03749806433916092, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03608153015375137, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.035743407905101776, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02166181430220604, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.018741797655820847, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.01844456046819687, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01710067130625248, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.01689102128148079, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.011502068489789963, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.011650791391730309, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011037035845220089, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008009687066078186, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04328674077987671, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04328674077987671, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.28.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.22683678567409515, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.2130274772644043, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2081146389245987, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.18886259198188782, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.10766664892435074, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10252943634986877, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.1204918920993805, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11119615286588669, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.10873465240001678, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09609802812337875, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09165717661380768, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.061596956104040146, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.053289253264665604, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.0517064668238163, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.0513235367834568, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03083229809999466, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.0266769677400589, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02631176821887493, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.024330344051122665, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.02409081533551216, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.01633727364242077, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01625257357954979, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.015798402950167656, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010893873870372772, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.053289253264665604, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.053289253264665604, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.28.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.20614174008369446, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.1810261756181717, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.168910950422287, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.14987972378730774, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09427877515554428, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08322176337242126, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.1160864531993866, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10599948465824127, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09758887439966202, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.07945822179317474, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.07577288150787354, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05936186760663986, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.051070112735033035, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.045723121613264084, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04437762871384621, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.030095098540186882, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.024472499266266823, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.023798203095793724, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.021657794713974, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.020795727148652077, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.01661045290529728, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.01673768274486065, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.014861950650811195, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.012166278436779976, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.051070112735033035, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.051070112735033035, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.29.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.11367612332105637, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.10432964563369751, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.09678502380847931, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.08786752074956894, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.053042247891426086, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04731567203998566, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.07015542685985565, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.06234882026910782, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05415944382548332, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04688425362110138, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.04583379998803139, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.035805508494377136, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.029868802055716515, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.025623805820941925, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.024538956582546234, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.01795237883925438, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.013443049974739552, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.012744519859552383, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.01222216710448265, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.011546935886144638, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.009424896910786629, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.009090684354305267, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.007751536555588245, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.006098092067986727, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05415944382548332, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05415944382548332, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.29.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.0973631888628006, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08995675295591354, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.08168809115886688, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.07412026077508926, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04534174129366875, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.039626553654670715, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.06180315464735031, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.05597155541181564, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04617545008659363, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.040325410664081573, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.039512503892183304, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.03140662983059883, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.02674219384789467, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.021911388263106346, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.020605646073818207, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.01570458523929119, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.01145564578473568, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.010617943480610847, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.010464434511959553, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.009630531072616577, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.00819715578109026, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.007936128415167332, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.006453374866396189, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.005081410985440016, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04617545008659363, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04617545008659363, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.29.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.24874486029148102, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.2292284071445465, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.22003290057182312, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.1993403136730194, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.11686625331640244, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.10830976068973541, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.13931411504745483, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.1263684183359146, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.11889489740133286, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.10296541452407837, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.09905707836151123, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.07132161408662796, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.06042494252324104, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05609932541847229, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.055037982761859894, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.035800572484731674, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.028698571026325226, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.027871983125805855, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.025843419134616852, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.025148339569568634, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.01840903051197529, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.01761748641729355, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.016524985432624817, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.011030805297195911, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.035800572484731674, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.035800572484731674, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.29.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.18839126825332642, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.16278888285160065, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.15342433750629425, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.13441592454910278, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.08790618926286697, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.07792135328054428, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10402284562587738, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.09350671619176865, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.08984015882015228, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.06998612731695175, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.06666959822177887, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05339683219790459, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04496820643544197, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.042426321655511856, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04180406033992767, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.026778001338243484, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.022253278642892838, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.021577540785074234, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.018739059567451477, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.01832115463912487, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.014231531880795956, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.014290078543126583, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.013338725082576275, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.01006705779582262, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05339683219790459, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05339683219790459, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.29.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.15341390669345856, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.14361542463302612, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.13970118761062622, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.1266121119260788, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07276876270771027, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.06895560771226883, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08237628638744354, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07598202675580978, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07361187040805817, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06476320326328278, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.061901893466711044, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04211081936955452, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03650331497192383, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03506097197532654, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03471751883625984, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.021085144951939583, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.018305864185094833, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.018002627417445183, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.016710253432393074, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.016491355374455452, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.01124162133783102, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.011525582522153854, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01075770240277052, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008063158951699734, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04211081936955452, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04211081936955452, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.29.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.20209452509880066, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1895570456981659, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.184932142496109, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.1678933948278427, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.09669799357652664, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.09197380393743515, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.10844431817531586, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.10006734728813171, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.09762997180223465, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.08626969903707504, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.08249043673276901, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.056075092405080795, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.04892962425947189, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.047404706478118896, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.047041989862918854, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.028243333101272583, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.0261384230107069, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.025827614590525627, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02423454262316227, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.024021128192543983, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.015970636159181595, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.018166454508900642, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01549572590738535, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.014622834511101246, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.04892962425947189, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.04892962425947189, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.29.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.20571546256542206, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.1815090775489807, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.17045757174491882, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.1500616818666458, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09432607889175415, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08405863493680954, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.1150156781077385, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10463142395019531, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09726862609386444, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.07922500371932983, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.07495211809873581, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05831105634570122, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05018393322825432, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.045558180660009384, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04440474137663841, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.029313212260603905, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.02412106655538082, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02354668453335762, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.02121189422905445, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.02046799287199974, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.01579117961227894, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.016077060252428055, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.014262380078434944, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.01144026592373848, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05018393322825432, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05018393322825432, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.30.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.1095728725194931, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.100243479013443, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.09128393232822418, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.08260037004947662, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.051102083176374435, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04463138431310654, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.06921995431184769, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.06240910664200783, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05219741538167, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04499661177396774, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.04423733800649643, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.0353984534740448, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.02993728592991829, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.02478559873998165, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.023427501320838928, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.017751535400748253, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.013142452575266361, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.01227540336549282, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.011954843997955322, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.011102452874183655, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.00936879962682724, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.009230590425431728, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.0075102150440216064, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.006275800988078117, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05219741538167, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05219741538167, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.30.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.09547897428274155, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08686858415603638, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.07536287605762482, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.0684126615524292, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04400666058063507, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03634760528802872, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.06484340876340866, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.058620940893888474, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04517835006117821, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.03889043256640434, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03865814954042435, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.032974984496831894, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.027953792363405228, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.021396392956376076, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.019549990072846413, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.016563745215535164, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.01135834027081728, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.01020627561956644, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.010375534184277058, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.009200098924338818, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.008627797476947308, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.008383037522435188, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.006264181341975927, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.005445764400064945, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04517835006117821, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04517835006117821, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.30.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.22474946081638336, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.20137616991996765, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.18781088292598724, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.16936330497264862, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.10393724590539932, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.09239887446165085, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.13313470780849457, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.11901592463254929, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.10685907304286957, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.08965485543012619, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.08724135905504227, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.0685306265950203, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.05686498433351517, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05003635585308075, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04826974868774414, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.03444301709532738, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.025873634964227676, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02458743378520012, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.022994350641965866, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.021843496710062027, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.01781383715569973, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.016799015924334526, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.01471610739827156, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.010716233402490616, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05003635585308075, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05003635585308075, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.30.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.16379593312740326, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.14769664406776428, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.1425313651561737, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.12055926024913788, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.07679182291030884, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.07108142971992493, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.08731945604085922, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.07948681712150574, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.07775185257196426, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.06287302076816559, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.05801357328891754, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.04490472376346588, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.03835875540971756, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.03710135444998741, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.036819204688072205, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.0225103497505188, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.01961367391049862, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.019267059862613678, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.01692737452685833, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.016717446967959404, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.012195341289043427, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.012644082307815552, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.011801408603787422, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.009256135672330856, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.04490472376346588, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.04490472376346588, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.30.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.15331622958183289, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.14363162219524384, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.1398492157459259, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.12690052390098572, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07327380031347275, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.0694596990942955, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08264171332120895, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07623498886823654, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07402262091636658, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06524883955717087, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06245270371437073, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04276455193758011, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03727343678474426, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03593238443136215, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.035613540560007095, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02160659246146679, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.019766774028539658, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.019496247172355652, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01829102821648121, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.018106089904904366, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012209240347146988, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.013711347244679928, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011786255054175854, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.01097308099269867, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04276455193758011, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04276455193758011, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.30.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.161906898021698, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.15187588334083557, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.14817042648792267, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.1344291865825653, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07737308740615845, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07358581572771072, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.0868956670165062, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08015749603509903, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07815074175596237, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06899557262659073, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06600962579250336, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04486967250704765, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.039062194526195526, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.037813249975442886, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03752044960856438, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.022616958245635033, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.020649300888180733, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02039312571287155, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01909191906452179, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.01891881786286831, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012667782604694366, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.014124766923487186, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.012267973273992538, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011174408718943596, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04486967250704765, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04486967250704765, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.30.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.17595866322517395, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.15610191226005554, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.1475788950920105, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.12861225008964539, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.08129768073558807, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.07320835441350937, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.099279023706913, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.0883098840713501, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.08329994976520538, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.06792747229337692, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.06408126652240753, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05014503374695778, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.042950693517923355, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.03983841836452484, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.03902662917971611, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.025673769414424896, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.021892568096518517, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02151186391711235, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.019457319751381874, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.01897076703608036, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.014794301241636276, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.015404149889945984, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.013819348998367786, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.012092744931578636, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05014503374695778, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05014503374695778, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.31.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.1061895564198494, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.0968628078699112, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.0882173702120781, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.07984215021133423, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.04949919879436493, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.043128401041030884, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.066900834441185, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.060333918780088425, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05060521513223648, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04350423440337181, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.04272889345884323, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03416188061237335, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.028950683772563934, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.02398574724793434, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.022681990638375282, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.017103267833590508, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.01269147265702486, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.011865245178341866, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.011522394604980946, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.010716697201132774, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.009003915823996067, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.008881181478500366, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.007230670191347599, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.006025675218552351, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05060521513223648, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05060521513223648, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.31.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.09517841041088104, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08642280846834183, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.07565269619226456, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.06861930340528488, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04396772384643555, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.036606211215257645, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.06458213925361633, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.05770289897918701, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.045150693506002426, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.038752928376197815, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.038485389202833176, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.03274136036634445, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.027539081871509552, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.02137724496424198, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.019634725525975227, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.016404755413532257, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.01133318804204464, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.010291951708495617, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.010328191332519054, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.009264237247407436, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.00861305184662342, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.00828433409333229, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.006385622546076775, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.005472538061439991, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.045150693506002426, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.045150693506002426, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.31.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.2369954138994217, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.20976445078849792, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.19394990801811218, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.17515923082828522, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.10949000716209412, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.09581846743822098, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.140801802277565, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.12658214569091797, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.11301255971193314, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.09350759536027908, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.09111417084932327, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.07231248170137405, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.06061223894357681, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.052722275257110596, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.05070376768708229, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.03639683127403259, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.027204643934965134, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.025729410350322723, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.023973781615495682, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.02264346554875374, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.018712390214204788, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.01772727072238922, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.015440641902387142, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.01116263773292303, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.052722275257110596, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.052722275257110596, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.31.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.1016402393579483, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.09097951650619507, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.08753809332847595, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.07567872107028961, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.0461217425763607, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04225469008088112, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.054787587374448776, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.04818283021450043, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.04686923325061798, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.0379980094730854, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.03609740734100342, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.02759561501443386, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.024853995069861412, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.023935209959745407, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.02371959760785103, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.014634456485509872, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.014866461046040058, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.014681817963719368, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.013609597459435463, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.013487079180777073, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.009156208485364914, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.011941846460103989, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.00888765323907137, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.010768837295472622, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.04818283021450043, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.04818283021450043, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.31.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.14864933490753174, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.139508455991745, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.13613034784793854, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.12340245395898819, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07056599110364914, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.06713946163654327, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.07937955111265182, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07310084253549576, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07123634964227676, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06288063526153564, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06006962060928345, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.040562666952610016, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.035055696964263916, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.0339076891541481, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03362968564033508, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02027907967567444, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.01753162033855915, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.017274271696805954, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.015979034826159477, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.01580420695245266, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.010695081204175949, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.010762694291770458, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.010303606279194355, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.007279439829289913, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.040562666952610016, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.040562666952610016, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.31.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.10389047116041183, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.09726250171661377, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.09485975652933121, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.08593438565731049, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.04926689341664314, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.04681713879108429, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.05551265552639961, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.05106944218277931, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.049770206212997437, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.043839775025844574, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.041905198246240616, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.028441954404115677, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.024597933515906334, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.023779118433594704, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.02358659729361534, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.014284290373325348, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.012519893236458302, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.012341500259935856, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.011453012935817242, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.01133302878588438, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.007762065157294273, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.007997272536158562, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.0074990964494645596, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.005783628206700087, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.05106944218277931, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.05106944218277931, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + }, + { + "key": "model.layers.31.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.10926377028226852, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.09788458794355392, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.09272480010986328, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.08060410618782043, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.050845175981521606, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.04602187126874924, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.06256718933582306, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.05570872128009796, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.05201271176338196, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.042947325855493546, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.040726907551288605, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.032257549464702606, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.027281779795885086, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.025067172944545746, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.024508720263838768, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.016622059047222137, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.014052143320441246, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.01378138829022646, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.012646831572055817, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.01232102606445551, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.009740615263581276, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.010195295326411724, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.009029214270412922, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.008230828680098057, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ], + "best_option_max": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.05201271176338196, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + "best_option": { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.05201271176338196, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + } + } + ], + "base_perplexity": 6.128923067853044, + "q_last_module_idx": 65 +} \ No newline at end of file diff --git a/measurement.json b/measurement.json new file mode 100644 index 0000000000000000000000000000000000000000..8803c8019cd053fd58be4b77c537d8ff4624a28e --- /dev/null +++ b/measurement.json @@ -0,0 +1,95878 @@ +{ + "measurement": [ + { + "key": "model.layers.0.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.011168720200657845, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.009599272161722183, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.004790821112692356, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.005077867768704891, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.0050776307471096516, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.0020578650292009115, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.01079169474542141, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.009518211707472801, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.005259049125015736, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.004644699394702911, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.0048562814481556416, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.0050615472719073296, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.004643067717552185, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.0027538840658962727, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.002120051998645067, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.0026784869842231274, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.001900350907817483, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.001615153276361525, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.0018467125482857227, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.0015654281014576554, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.0017823567613959312, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.0018463897285982966, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.0014250442618504167, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.001544640981592238, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.0.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.011349556967616081, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.00980227068066597, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.004773853346705437, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.005033744964748621, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.005033500958234072, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.001928701065480709, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.01110624335706234, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.009691532701253891, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.005209838971495628, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.004562461748719215, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.0047759003937244415, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.005032561253756285, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.004560141358524561, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.002655236516147852, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.00194216996897012, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.0026120387483388186, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.0016779541037976742, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.001346124685369432, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.0016128767747431993, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.0012812796048820019, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.001617392641492188, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.0016132771270349622, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.0012200467754155397, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.0012539472663775086, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.0.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.11578802764415741, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.07042404264211655, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.043504536151885986, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.050045717507600784, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.0499984472990036, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.024094626307487488, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.07798509299755096, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.06458660215139389, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.0545671209692955, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.03145388513803482, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03755158931016922, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.042070720344781876, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.031096184626221657, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.02426334097981453, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.022416630759835243, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.02171494998037815, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.01284871157258749, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.011072240769863129, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.009356767870485783, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.007840869016945362, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.011130761355161667, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.009300582110881805, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.007214382756501436, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.006086327601224184, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.0.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.11313514411449432, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.0732056200504303, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.05039582401514053, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.04979194700717926, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.048297327011823654, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.027966083958745003, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.07069999724626541, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.06314126402139664, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.053515732288360596, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.031263019889593124, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.03367261961102486, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03614491969347, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.030515408143401146, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.024038290604948997, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.022272668778896332, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.0183316171169281, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.013834427110850811, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.012738020159304142, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.011082837358117104, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.009938649833202362, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.01025769766420126, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.010982340201735497, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.00825857650488615, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.0086810402572155, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.0.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.11327356100082397, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1058005541563034, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.10340237617492676, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.09472587704658508, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.051241014152765274, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.04909159988164902, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.05705980956554413, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.052599381655454636, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.051703788340091705, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.046595051884651184, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.04498507082462311, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.029025588184595108, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.02526126801967621, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.024675775319337845, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.02453121356666088, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.01461299229413271, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.013298182748258114, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.013174419291317463, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.012456698343157768, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.012386062182486057, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.008203603327274323, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.008870589546859264, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.008029122836887836, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.006887989118695259, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.0.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.13404832780361176, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.12670819461345673, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.12447718530893326, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.114055335521698, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.06098993122577667, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.05894749239087105, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.06735212355852127, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.06225910410284996, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.06142910569906235, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.055878788232803345, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.05380960926413536, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.03398489952087402, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.029508553445339203, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.02898118458688259, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.02886049821972847, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.016938773915171623, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.01486075110733509, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.014734474010765553, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01386601198464632, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.013790595345199108, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.008871518075466156, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.008892863988876343, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.008702414110302925, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.005870731081813574, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.0.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.07670606672763824, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.06584762781858444, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.06038632243871689, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.05355939269065857, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.03438195586204529, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.029740920290350914, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.04498032480478287, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.03952185437083244, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.03574554622173309, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.028567753732204437, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.02737824246287346, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.022440247237682343, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.019085902720689774, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.01691751927137375, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.016391701996326447, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.011539027094841003, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.00958372000604868, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.009348219260573387, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.008578285574913025, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.008279938250780106, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.006880487315356731, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.00712636299431324, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.00626949081197381, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.0057810521684587, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.1.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.019663630053400993, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.013406879268586636, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.007660592906177044, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.008349855430424213, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.00809794757515192, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.0036465248558670282, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.013933522626757622, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.012625516392290592, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.0092203663662076, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.006169123109430075, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.006603498011827469, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.007054396439343691, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.006034728605300188, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.0040495740249753, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.003436762373894453, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.0035601980052888393, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.0023379670456051826, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.0019536123145371675, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.0020181608851999044, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.001642401795834303, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.0019508769037202, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.0019979930948466063, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.0013339562574401498, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.0014734480064362288, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.1.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.017518868669867516, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.01219908520579338, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.006678944453597069, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.007290045265108347, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.0070999860763549805, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.0029782995115965605, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.013018052093684673, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.011648166924715042, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.008138769306242466, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.005624793935567141, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.006064086686819792, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.0065219649113714695, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.005529758520424366, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.0035434754099696875, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.0028902385383844376, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.003280432429164648, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.0020198565907776356, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.0016353140817955136, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.0017707019578665495, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.0013762976741418242, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.0017502045957371593, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.0017561280401423573, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.001087298383936286, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.0012416673125699162, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.1.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.13320785760879517, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08857599645853043, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.06414425373077393, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.062488194555044174, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.057051729410886765, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.034113503992557526, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.08158433437347412, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.07376103848218918, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.0631725862622261, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.03857894614338875, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.039744194597005844, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.04156964644789696, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.03522882238030434, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.027669740840792656, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.02555888704955578, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.020771265029907227, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.014558087103068829, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.012766730040311813, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.011037899181246758, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.009418193250894547, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.010778137482702732, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.010443516075611115, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.008073403500020504, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.006722630932927132, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.1.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.15270331501960754, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.12318339943885803, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.11086562275886536, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.09390351921319962, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.07004237920045853, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.05806812644004822, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.08862987905740738, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.07844515144824982, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.07241526991128922, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.05164502561092377, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.049597982317209244, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.04550931602716446, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.0382709726691246, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.034586384892463684, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.033655546605587006, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.02309839054942131, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.01936962828040123, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.01856229268014431, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.01617896556854248, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.015599473379552364, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.013060690835118294, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.014155326411128044, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.011925097554922104, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.011372439563274384, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.1.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.1567811667919159, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1485101878643036, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.14568355679512024, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.13279618322849274, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.0726742148399353, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07009439915418625, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.0808241069316864, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.0739935040473938, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07315933704376221, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06607392430305481, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06361325085163116, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04134838655591011, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03595521301031113, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03534533828496933, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03519531339406967, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.021018361672759056, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.019455352798104286, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.019302669912576675, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01826331578195095, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.01818668842315674, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012167708948254585, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.013381781987845898, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.012010962702333927, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010793034918606281, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.1.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.19255827367305756, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.18272824585437775, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.17977085709571838, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.16407370567321777, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.08971576392650604, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.08672579377889633, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.098969966173172, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.09122589230537415, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.09027554839849472, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.08172681927680969, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0785735547542572, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.050545584410429, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.043932583183050156, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.04328686743974686, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.04312015324831009, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.025338059291243553, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02311822585761547, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02295522764325142, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02161826193332672, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.02152479812502861, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.013921397738158703, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.015096379444003105, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.013726180419325829, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011499397456645966, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.1.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.019199078902602196, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.01878763735294342, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.006921666674315929, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.00646975776180625, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.005650496110320091, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.0034061234910041094, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.018811136484146118, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.018254732713103294, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.005731276702135801, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.005354026332497597, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.0053440118208527565, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.005248498171567917, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.005061940755695105, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.0026619865093380213, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.0026766916271299124, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.002273819176480174, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.0023983637802302837, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.001029878854751587, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.0023623292800039053, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.000951776746660471, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.0023449964355677366, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.0023288021329790354, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.0007354797562584281, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.0007976787746883929, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.2.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.05501016974449158, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.043334461748600006, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.03665383905172348, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.03312649205327034, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.024345438927412033, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.018490733578801155, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.03337150067090988, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.03038576804101467, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.025987472385168076, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.018868857994675636, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.018617024645209312, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.01694329082965851, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.014517929404973984, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.011795316822826862, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.011069130152463913, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.008465803228318691, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.00624634325504303, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.005706323776394129, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.005192603450268507, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.004691623616963625, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.00441138306632638, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.004422945436090231, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.0034645930863916874, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.0029624789021909237, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.2.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.05672017112374306, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.04279809445142746, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.03405015915632248, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.03148718550801277, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.024548310786485672, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.017191600054502487, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.03530892729759216, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.032226257026195526, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.026598580181598663, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.01862349361181259, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.018713288009166718, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.017903584986925125, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.015325364656746387, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.011925559490919113, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.010991599410772324, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.008935020305216312, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.0063719660975039005, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.005681759677827358, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.0052579911425709724, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.004604979418218136, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.004677099175751209, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.004683797247707844, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.0035144537687301636, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.00315110944211483, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.2.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.16881753504276276, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.1410287767648697, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.12958964705467224, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.11464769393205643, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.07729382812976837, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.06546717882156372, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.09526020288467407, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.08619674295186996, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.08046350628137589, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.061131030321121216, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.058465443551540375, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.04843978211283684, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.041259944438934326, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.03719085827469826, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.03618834540247917, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.02420763485133648, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.01911242865025997, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.018178023397922516, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.015842380002141, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.015133141539990902, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.012576712295413017, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.012094818986952305, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.011064727790653706, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.0076539963483810425, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.2.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.156541645526886, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.13858985900878906, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.12967531383037567, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.11246905475854874, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.07320044189691544, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.06499434262514114, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.09032202512025833, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.08170590549707413, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.07497423142194748, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.06085894629359245, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.05692308768630028, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.04705378785729408, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.0400383360683918, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.036137260496616364, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.035172898322343826, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.024017736315727234, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.020189229398965836, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.019475512206554413, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.0179683156311512, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.017397213727235794, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.013736758381128311, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.014681994915008545, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.01252848468720913, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.01172161940485239, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.2.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.19975252449512482, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.18823857605457306, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.1846926212310791, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.16731983423233032, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.09383014589548111, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.09011030942201614, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.10378500074148178, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.09576664119958878, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.09457574784755707, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.08421718329191208, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.08041726052761078, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.05310473218560219, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.04596571624279022, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.04511168226599693, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.04491891711950302, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02657609060406685, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.023638196289539337, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.023406749591231346, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.021765321493148804, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.021637681871652603, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.014316429384052753, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.014850785955786705, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.014048966579139233, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010670391842722893, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.2.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.23413872718811035, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.22090375423431396, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.21686704456806183, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.19659093022346497, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.10993479937314987, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10565081238746643, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12136806547641754, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.1120128333568573, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11077161878347397, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09870945662260056, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0940827876329422, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06181109696626663, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05351909250020981, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05259128287434578, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.052355699241161346, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03086814470589161, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02699517272412777, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02673065848648548, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.024733295664191246, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.024595467373728752, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016258612275123596, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016147766262292862, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.015950508415699005, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010682974942028522, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.2.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.1790362149477005, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.15777480602264404, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.14935097098350525, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.13222238421440125, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.08091238141059875, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.07288438826799393, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.0961962640285492, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.08775460720062256, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.083379827439785, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.06838435679674149, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.06484240293502808, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.04880445823073387, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.04199089854955673, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.03893505781888962, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.03818897530436516, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.02450370043516159, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.020580368116497993, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02021637000143528, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.018184015527367592, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.017703499644994736, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.013272203505039215, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.01354095060378313, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.012265851721167564, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.009716490283608437, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.3.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.04273678734898567, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.03584158048033714, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.03184032440185547, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.028262462466955185, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.01939507946372032, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.015999650582671165, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.025632822886109352, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.02331860177218914, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.020244425162672997, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.01563546620309353, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.015160813927650452, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.013055362738668919, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.011152705177664757, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.009398764930665493, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.008941682986915112, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.006538650952279568, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.004987573716789484, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.004653621930629015, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.004263903480023146, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.003960269037634134, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.003473845310509205, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.0034712685737758875, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.002868041628971696, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.0023902307730168104, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.3.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.0428481288254261, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.03501201048493385, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.029858678579330444, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.026616744697093964, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.019089410081505775, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.01491191703826189, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.02664991095662117, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.024140529334545135, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.020131420344114304, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.015170253813266754, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.014985787682235241, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.01351439580321312, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.011503629386425018, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.00922862533479929, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.008605791255831718, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.006766557227820158, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.0048356736078858376, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.0043977960012853146, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.004073855467140675, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.0036462252028286457, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.0035126814618706703, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.0034059712197631598, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.0026915359776467085, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.002188342157751322, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.3.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.1756085604429245, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.14955584704875946, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.13890667259693146, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.12239907681941986, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.08067318052053452, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.06992373615503311, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.09721731394529343, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.08924295753240585, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.0834602639079094, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.06486669182777405, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.06175591051578522, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.04936389625072479, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04259105026721954, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.03869667276740074, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.03773403912782669, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.02467101626098156, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.01980268396437168, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.018941247835755348, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.016586411744356155, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.015916477888822556, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.012710344977676868, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.012334806844592094, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.011317485012114048, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.007636150810867548, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.3.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.17265617847442627, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.15349774062633514, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.14630544185638428, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.125789612531662, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.08070380240678787, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.07379523664712906, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.09464309364557266, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.0860721543431282, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.08211905509233475, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.0660165399312973, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.06170710548758507, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.0487380288541317, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.041771307587623596, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.039406705647706985, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.038827069103717804, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.024621980264782906, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.021481908857822418, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.02098594233393669, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.018773801624774933, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.018404876813292503, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.013682577759027481, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.014801764860749245, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.012944573536515236, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.011539943516254425, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.3.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.209043487906456, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.19700421392917633, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.19321338832378387, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.17542071640491486, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.09851204603910446, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.09449568390846252, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.1090313121676445, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.10056618601083755, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.0993163213133812, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.08842168003320694, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.08447795361280441, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.05569034069776535, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.04819665849208832, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.04729839414358139, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.04708410054445267, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02784634381532669, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.024610033258795738, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.024359652772545815, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02261015959084034, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.022482460364699364, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.014900386333465576, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.015216012485325336, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.014612419530749321, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010649471543729305, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.3.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.24484506249427795, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.23094336688518524, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.22673256695270538, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.2060040980577469, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11548089236021042, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.11090388149023056, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.1278538554906845, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11774861067533493, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11639799177646637, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.1037989929318428, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09926366060972214, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.0653105154633522, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05634693056344986, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05533489212393761, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05509016662836075, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.032666780054569244, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.028497371822595596, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.028219513595104218, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.026142287999391556, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.025994718074798584, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.0174561757594347, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.017186559736728668, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01712212525308132, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011524848639965057, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.3.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.20301590859889984, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.17993180453777313, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.17123068869113922, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.15232975780963898, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09230129420757294, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08366638422012329, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.10861063003540039, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.09921830892562866, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09496082365512848, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.07847336679697037, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.07448621094226837, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05513899773359299, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.04751267284154892, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.044402044266462326, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04363902285695076, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.027674056589603424, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.023362280800938606, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.022996846586465836, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.020694028586149216, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.020208235830068588, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.014936521649360657, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.015182916074991226, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.013920888304710388, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.010784035548567772, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.4.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.05948399379849434, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.05045810714364052, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.0450628362596035, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.04024699702858925, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.02709878981113434, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.022555913776159286, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.03599774092435837, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.03251658007502556, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.02822723612189293, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.022175561636686325, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.02156623639166355, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.01832883432507515, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.015611917711794376, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.013163863681256771, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.012529414147138596, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.009210369549691677, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.007052582688629627, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.006607340648770332, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.006121970247477293, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.00571369007229805, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.004913266748189926, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.004976646043360233, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.0040607331320643425, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.003532960545271635, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.4.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.05663783848285675, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.04680328816175461, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.04078899696469307, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.036554399877786636, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.025308020412921906, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.020324628800153732, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.03483179956674576, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.031312715262174606, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.026685133576393127, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.02047596499323845, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.020128708332777023, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.01776907965540886, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.014995941892266273, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.01226199883967638, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.011548623442649841, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.008920381776988506, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.006504755932837725, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.0060025774873793125, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.005570911802351475, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.005091105587780476, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.004694020375609398, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.0045917523093521595, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.0037039704620838165, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.003107720520347357, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.4.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.1917179524898529, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.16577664017677307, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.15542399883270264, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.1376524418592453, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.08840838074684143, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.07796114683151245, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.1050894558429718, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.09659653156995773, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.09124546498060226, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.07238545268774033, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.06868880242109299, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.05346288904547691, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.046208061277866364, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04242615029215813, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.041495393961668015, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.026723310351371765, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.021805057302117348, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02098340354859829, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.01855577528476715, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.0179296862334013, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.013873790390789509, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.013583481311798096, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.012519973330199718, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.008629865944385529, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.4.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.16939057409763336, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.1513446867465973, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.14464770257472992, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.12346041202545166, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.079004667699337, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.07215382903814316, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.09365367144346237, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.08410616219043732, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.08046706020832062, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.06545904278755188, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.0601811446249485, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.04818735271692276, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.040927812457084656, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.03869343549013138, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.038167115300893784, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.024444689974188805, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.021173864603042603, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.020702725276350975, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.018659424036741257, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.018337540328502655, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.013722981326282024, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.01465285662561655, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.013013369403779507, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.011514890007674694, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.4.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.1883140653371811, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.17655447125434875, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.1725977212190628, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.15641066431999207, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.0888027548789978, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.08470439165830612, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.09940768033266068, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.0913398340344429, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.08968131244182587, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.07924304902553558, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.07571203261613846, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.05076218023896217, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.04382229968905449, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.042683910578489304, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.042410291731357574, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.025482339784502983, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02221999131143093, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02194603905081749, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.020312026143074036, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.020150791853666306, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.013768038712441921, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.013788512907922268, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.013400079682469368, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.009614584967494011, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.4.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.246357262134552, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.23156091570854187, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.22673998773097992, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.20575161278247833, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11632581055164337, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.11125487089157104, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12949900329113007, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11917395889759064, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11740680038928986, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.10403173416852951, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09944429248571396, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06603679060935974, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05707656964659691, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05578989163041115, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05549166351556778, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03308970108628273, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.028737511485815048, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02841159887611866, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.026251301169395447, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.02606653794646263, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.0176161490380764, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01735866442322731, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.017196768894791603, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011594372801482677, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.4.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.2095576524734497, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.18695861101150513, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.17822471261024475, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.15851232409477234, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09566449373960495, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08714070916175842, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.11320783942937851, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10278883576393127, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09833946079015732, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08187927305698395, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.0775674432516098, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.0575646311044693, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.04935459420084953, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.046164318919181824, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04538174346089363, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.028994236141443253, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.024500075727701187, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.024127205833792686, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.021858954802155495, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.02137257158756256, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.015817200765013695, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.016173504292964935, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.014756884425878525, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.011794302612543106, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.5.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.07179942727088928, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.06272782385349274, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.0575268529355526, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.05125272274017334, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.03292565792798996, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.028575770556926727, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.04211081191897392, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.03832326829433441, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.03401795029640198, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.027597596868872643, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.02656625211238861, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.021354198455810547, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.01834231987595558, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.01593482308089733, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.015323616564273834, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.010706116445362568, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.008427944034337997, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.007987068966031075, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.007394589949399233, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.006993317045271397, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.005677962210029364, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.005734641570597887, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.004852258134633303, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.003964860457926989, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.5.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.06473594158887863, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.055526718497276306, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.049911461770534515, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.044354259967803955, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.02931882254779339, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.02469693124294281, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.03860938921570778, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.03517904505133629, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.030482374131679535, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.024252815172076225, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.023485815152525902, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.019544633105397224, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.016757169738411903, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.014120960608124733, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.01342977024614811, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.009770752862095833, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.0073308334685862064, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.006842531729489565, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.006326897535473108, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.005861466750502586, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.0050523970276117325, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.004929880145937204, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.004123569931834936, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.003137608990073204, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.5.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.20667532086372375, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.18412651121616364, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.17575180530548096, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.1560567021369934, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.09606868773698807, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.08750919252634048, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.11123546212911606, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.10240286588668823, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.09829515218734741, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.08090247958898544, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.07650333642959595, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.056504372507333755, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04887129366397858, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.046007949858903885, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.045349981635808945, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.0281996950507164, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.023354772478342056, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.022706395015120506, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.02021184004843235, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.019752709195017815, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.014485612511634827, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.013975773938000202, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.013443267904222012, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.008471165783703327, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.5.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.18959152698516846, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.17195124924182892, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.1653534173965454, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.14394891262054443, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.08860822021961212, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.0817338302731514, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10249451547861099, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.0936884954571724, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09002437442541122, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.0746944472193718, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07026924192905426, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05280518904328346, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.045185159891843796, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04280707612633705, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04224920645356178, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.026574712246656418, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.02261975221335888, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.022079620510339737, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.019902443513274193, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.019540734589099884, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.01446588896214962, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.014702496118843555, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.013650595210492611, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.010650047101080418, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.5.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.1762571483850479, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1653290092945099, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.16162365674972534, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.14647088944911957, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.08305193483829498, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07925103604793549, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.09263620525598526, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08550563454627991, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.08388134837150574, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.07411837577819824, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.07074187695980072, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.047173138707876205, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.040913499891757965, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.039821311831474304, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03956005722284317, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02358744852244854, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.020505355671048164, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.020246082916855812, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.018697958439588547, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.018533287569880486, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012398071587085724, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012418893165886402, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.012031331658363342, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.00827084295451641, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.5.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.2419440895318985, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.2276521474123001, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2230045050382614, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.20235870778560638, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.1143060028553009, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10936648398637772, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12692731618881226, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11716075241565704, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11531758308410645, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.10232201963663101, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09775715321302414, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06462647765874863, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.056016407907009125, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05473296344280243, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05443199723958969, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.032305825501680374, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02803993970155716, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.027713479474186897, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02559611015021801, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.025405537337064743, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016914283856749535, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016712086275219917, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01648643985390663, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010864359326660633, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.5.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.2036168873310089, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.1832730621099472, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.17470474541187286, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.1561361700296402, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.0934390053153038, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08529166877269745, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.111430324614048, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10100080072879791, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09583073854446411, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08062601834535599, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.07705391943454742, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05686961114406586, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.04845084622502327, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.04500170052051544, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04415525868535042, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.028722483664751053, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.02372308075428009, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.023303192108869553, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.021242249757051468, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.020712487399578094, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.015683121979236603, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.01551529485732317, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.014488616958260536, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.011047808453440666, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.6.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.06634723395109177, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.05762307345867157, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.051811300218105316, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.04625110328197479, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.030352430418133736, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.025681382045149803, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.04047377035021782, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.03665715083479881, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.03144051507115364, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.025375576689839363, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.02464967966079712, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.020561281591653824, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.017536645755171776, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.014713982120156288, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.013981547206640244, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.010300645604729652, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.007822458632290363, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.007315567694604397, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.006863356567919254, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.006390023976564407, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.0054395729675889015, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.005470580421388149, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.004440734162926674, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.003781738691031933, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.6.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.060605134814977646, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.051686182618141174, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.04550164192914963, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.04061912000179291, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.02736980840563774, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.022464901208877563, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.03753753378987312, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.034015703946352005, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.02854038216173649, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.022614534944295883, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.022116539999842644, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.019017454236745834, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.016205081716179848, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.013237089850008488, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.012449914589524269, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.009491239674389362, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.006952513940632343, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.006399805191904306, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.006020466797053814, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.005488865077495575, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.0049445740878582, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.004851828329265118, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.003889617044478655, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.0031672290060669184, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.6.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.18390421569347382, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.16390341520309448, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.1555459350347519, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.13815124332904816, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.0854179784655571, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.07736478745937347, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.10099665075540543, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.09283767640590668, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.08745920658111572, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.07208962738513947, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.06840716302394867, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.051350172609090805, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04431234672665596, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04098079353570938, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.040139734745025635, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.02561933919787407, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.020926428958773613, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.020221518352627754, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.01816825568675995, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.017614122480154037, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.013170411810278893, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.012800670228898525, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.011958121322095394, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.007893593981862068, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.6.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.18748553097248077, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.1659504473209381, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.15739239752292633, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.13692741096019745, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.08746771514415741, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.07871513068675995, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10398761928081512, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.09420737624168396, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.0894230529665947, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07223328202962875, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.06758613139390945, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.053499579429626465, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.045534636825323105, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04249802604317665, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.041758015751838684, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.026858501136302948, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.02269449643790722, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.02203347347676754, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.01975790224969387, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.019285235553979874, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.0145133500918746, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.015179971233010292, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.01350901834666729, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.011253130622208118, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.6.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.17114149034023285, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.16083428263664246, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.15720336139202118, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.14284665882587433, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.08073896169662476, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07706644386053085, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.09042980521917343, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08328168839216232, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.08152130991220474, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.07229864597320557, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06913543492555618, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.046180956065654755, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.039897676557302475, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03875955566763878, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03848888725042343, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02312663570046425, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.020065199583768845, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.019804779440164566, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.018367337062954903, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.018200399354100227, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012346301227807999, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012314205057919025, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011960572563111782, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008389642462134361, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.6.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.23557148873806, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.22183120250701904, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.21720033884048462, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.19756180047988892, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11134614795446396, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10655659437179565, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12412071228027344, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11433691531419754, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11234697699546814, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09990664571523666, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0955205112695694, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06329933553934097, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.054706476628780365, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05334842950105667, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05303625017404556, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03165601193904877, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02740606665611267, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.027081916108727455, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02507716789841652, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.024877533316612244, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016712261363863945, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016471730545163155, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.016263943165540695, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010853748768568039, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.6.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.21076911687850952, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.19050660729408264, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.18252255022525787, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.16333793103694916, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09705882519483566, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08929018676280975, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.11463408172130585, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10375270992517471, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09938328713178635, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08413700014352798, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08029516786336899, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05836555361747742, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.04982609674334526, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.04680825024843216, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.046071410179138184, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.029368113726377487, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.024785494431853294, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.024426598101854324, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.02229994349181652, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.02183464542031288, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.015965718775987625, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.016251832246780396, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.014962779358029366, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.011789047159254551, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.7.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.07253632694482803, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.06478405743837357, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.05837472155690193, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.05205954238772392, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.03348775953054428, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.02869037352502346, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.04561757668852806, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.04102073982357979, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.03436758741736412, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.028703801333904266, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.027966883033514023, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.023239726200699806, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.01973414234817028, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.01628398336470127, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.015371035784482956, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.011651421897113323, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.008739606477320194, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.008151110261678696, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.007829796522855759, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.007258750032633543, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.006187691818922758, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.006257256492972374, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.004955670330673456, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.004403706174343824, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.7.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.0626121312379837, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.05524720624089241, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.04855867102742195, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.043166615068912506, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.028583906590938568, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.023709077388048172, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.04002339765429497, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.03640666976571083, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.029457036405801773, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.02430696412920952, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.023732928559184074, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.02031564898788929, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.017455879598855972, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.013826129958033562, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.012836582958698273, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.010184375569224358, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.007291768211871386, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.006658272352069616, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.006462015211582184, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.0058193085715174675, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.005293699912726879, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.005209335591644049, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.00403254572302103, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.0033859771210700274, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.7.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.19899427890777588, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.17977093160152435, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.1722375452518463, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.15291020274162292, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.09298620373010635, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.0854603573679924, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.10868975520133972, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.09914804995059967, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.09474433958530426, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.07913115620613098, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.07504456490278244, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.055301252752542496, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04738381505012512, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04455778747797012, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04388604313135147, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.027598528191447258, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.022764069959521294, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.0221426859498024, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.019917359575629234, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.019456520676612854, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.014231402426958084, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.01377798430621624, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.013111609034240246, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.008610040880739689, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.7.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.1884087473154068, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.16933833062648773, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.16095305979251862, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.1393108069896698, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.08832570910453796, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.08029365539550781, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10636451095342636, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.09553088247776031, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09012182801961899, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07366209477186203, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.0699334368109703, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.054981034249067307, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.046462249010801315, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04323485121130943, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04246465116739273, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.027733784168958664, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.023538334295153618, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.022891977801918983, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.020789284259080887, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.020309122279286385, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.015384198166429996, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.01624443754553795, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.014347760006785393, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.012545468285679817, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.7.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.1666836440563202, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1567574441432953, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.1534167230129242, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.13940130174160004, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.0788261890411377, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07530619204044342, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08801373839378357, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08102373778820038, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07955755293369293, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.07058826088905334, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06748536229133606, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04497779533267021, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03889338672161102, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.0379004031419754, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03766704350709915, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02252550795674324, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.01975112594664097, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.019513443112373352, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01810702309012413, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.0179611723870039, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012069562450051308, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012280250899493694, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01174020767211914, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008592424914240837, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.7.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.23579122126102448, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.222219780087471, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.21776892244815826, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.1981118619441986, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.1117170974612236, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10700331628322601, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12449662387371063, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11445385217666626, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11269330978393555, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.10021616518497467, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09587167948484421, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06365256011486053, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05487082898616791, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.053646303713321686, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.053360287100076675, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03187926858663559, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.027765214443206787, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.0274557713419199, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.025439215824007988, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.02525932528078556, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.01705533266067505, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016955142840743065, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.016653813421726227, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.0115723367780447, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.7.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.20433014631271362, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.18405292928218842, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.1758449673652649, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.15770600736141205, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09402231127023697, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08595852553844452, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.11139664053916931, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10105358064174652, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09641910344362259, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08123823255300522, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.07780414819717407, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.056854426860809326, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.048471637070178986, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.04529405012726784, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.044517822563648224, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.028560878708958626, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.02386007271707058, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.023480109870433807, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.021384425461292267, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.020891698077321053, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.015477100387215614, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.015538507141172886, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.014423436485230923, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.011068885214626789, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.8.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.07229823619127274, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.0640784278512001, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.05915708839893341, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.05233281850814819, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.03329877555370331, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.029231058433651924, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.042107172310352325, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.038525331765413284, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.03417966887354851, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.028040915727615356, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.026857197284698486, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.0213544350117445, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.01839970052242279, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.01605963334441185, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.01547007542103529, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.010699591599404812, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.008419157937169075, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.007991536520421505, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.007386090699583292, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.00699745723977685, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.005629621911793947, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.005631897132843733, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.0048264265060424805, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.0037945793010294437, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.8.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.06344876438379288, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.05514335259795189, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.05007714778184891, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.044085338711738586, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.028834931552410126, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.024654246866703033, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.03750180825591087, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.034161586314439774, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.029770636931061745, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.023919442668557167, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.023006537929177284, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.019016969949007034, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.016308337450027466, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.013897260650992393, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.013273438438773155, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.009519466198980808, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.0072477711364626884, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.006803502328693867, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.006268806755542755, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.0058531141839921474, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.0049530318938195705, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.004873728379607201, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.004104185849428177, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.0031792030204087496, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.8.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.18240326642990112, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.1626574695110321, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.15496958792209625, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.13663387298583984, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.08455818891525269, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.07684142887592316, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.09996659308671951, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.09083674848079681, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.08648092299699783, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.07106637209653854, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.06717733293771744, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.05087890848517418, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04350433498620987, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04059627652168274, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.039900291711091995, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.025416448712348938, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.020873580127954483, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.020246481522917747, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.018127888441085815, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.017663544043898582, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.013154354877769947, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.012906396761536598, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.012026366777718067, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.008315572515130043, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.8.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.1977970451116562, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.18005962669849396, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.17297670245170593, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.15205296874046326, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.09264685213565826, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.0856722965836525, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10854403674602509, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.09851948916912079, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09423834830522537, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07904263585805893, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07453982532024384, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05574023351073265, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04742798954248428, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.044760387390851974, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.044106315821409225, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.027905823662877083, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.023507753387093544, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.022936217486858368, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.02082017809152603, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.020411867648363113, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.014840735122561455, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.015165221877396107, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.013953466899693012, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.010774858295917511, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.8.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.16268427670001984, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1532304435968399, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.14993759989738464, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.1363285779953003, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07694627344608307, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.0735417902469635, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08607260882854462, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07920283079147339, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07766101509332657, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06899941712617874, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0660800039768219, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.0439998134970665, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.037978317588567734, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.036970168352127075, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03672965243458748, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02201676554977894, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.01919376291334629, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.018961550667881966, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01760033145546913, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.017453495413064957, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.011735353618860245, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.011853777803480625, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011399183422327042, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008189890533685684, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.8.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.23203247785568237, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.21882356703281403, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2143489271402359, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.1949910670518875, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.10994546860456467, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10524184256792068, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.1226196438074112, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11273852735757828, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11088178306818008, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09857067465782166, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09438752382993698, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06263895332813263, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05398797243833542, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.052726712077856064, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.052423760294914246, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03131991624832153, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.027149271219968796, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.026830697432160378, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.024843666702508926, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.02466162107884884, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016555219888687134, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016400840133428574, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01612219400703907, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010934562422335148, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.8.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.21018268167972565, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.19030331075191498, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.18263329565525055, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.16296975314617157, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09686324745416641, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.0892181545495987, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.11363700777292252, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10320858657360077, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09913817793130875, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08380107581615448, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.0797918289899826, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05793079733848572, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.049579475075006485, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.046735674142837524, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04604290425777435, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.029255900532007217, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.024781428277492523, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02444753423333168, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.022280484437942505, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.021841231733560562, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.016100801527500153, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.016270168125629425, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.015168504789471626, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.01186355110257864, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.9.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.0923733115196228, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.08220721781253815, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.07622025161981583, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.06749071180820465, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.04258981719613075, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.037622444331645966, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.05377095192670822, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.04885927587747574, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.04363660514354706, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.0360131561756134, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.03449761122465134, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.02736286073923111, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.023438431322574615, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.02063322439789772, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.01992359384894371, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.013733101077377796, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.010986043140292168, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.010486569255590439, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.009727832861244678, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.009283306077122688, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.007337464485317469, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.007517325226217508, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.00634734658524394, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.00537061644718051, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.9.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.07935116440057755, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.06979311257600784, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.06384820491075516, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.05638270825147629, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.03625747188925743, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.031399913132190704, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.04671098664402962, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.04258308187127113, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.0373074896633625, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.030413977801799774, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.02923060953617096, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.023759830743074417, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.020337557420134544, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.017467139288783073, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.01673617959022522, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.011891154572367668, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.009111745283007622, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.008590858429670334, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.007955510169267654, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.007472165394574404, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.00619489885866642, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.006100549828261137, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.0051757474429905415, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.00400660140439868, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.9.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.19220881164073944, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.17436790466308594, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.16780315339565277, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.14817847311496735, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.0897936075925827, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.08307909220457077, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.10316525399684906, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.09467410296201706, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.09124304354190826, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.07631740719079971, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.07175160944461823, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.052423689514398575, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.045171648263931274, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04294484108686447, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04242084175348282, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.026141708716750145, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.021819069981575012, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02130700834095478, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.019036483019590378, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.01867504231631756, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.013404963538050652, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.012975042685866356, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.012573492713272572, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.007934489287436008, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.9.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.1993076205253601, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.18014933168888092, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.17243245244026184, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.15134650468826294, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.09344325959682465, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.08576633036136627, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10941915959119797, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.09956937283277512, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09505053609609604, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07860640436410904, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07455437630414963, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05619671940803528, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04785337299108505, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04504535347223282, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04437866061925888, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.028088334947824478, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.02348434552550316, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.022888371720910072, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.02051377296447754, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.020069656893610954, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.014821244403719902, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.014906519092619419, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.013882310129702091, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.01027657650411129, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.9.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.16206996142864227, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.15213730931282043, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.14868398010730743, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.13507655262947083, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07658286392688751, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07301879674196243, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08578245341777802, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07891295105218887, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07731891423463821, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.0684279277920723, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06548789143562317, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.043890975415706635, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03789806365966797, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.036841198801994324, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03659507632255554, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.021988868713378906, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.019250184297561646, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.019006384536623955, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.017626376822590828, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.017475441098213196, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.0117854755371809, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01205415092408657, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011435518972575665, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008514638058841228, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.9.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.22811272740364075, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.21440167725086212, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2097819298505783, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.19063790142536163, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.10790391266345978, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10301275551319122, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12053030729293823, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11082379519939423, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.10892365127801895, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.096433125436306, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09228646010160446, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06161956861615181, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.053128574043512344, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.051801811903715134, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05148930847644806, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.030858976766467094, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.026810333132743835, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.026479406282305717, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.024499699473381042, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.024301273748278618, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016448548063635826, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016406487673521042, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.016002364456653595, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011173042468726635, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.9.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.21688273549079895, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.195871502161026, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.18770845234394073, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.16686895489692688, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09991750866174698, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09185732901096344, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.11755452305078506, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10667914897203445, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10227162390947342, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08605503290891647, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.0815466120839119, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05971937254071236, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05112798139452934, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.048075463622808456, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04733084887266159, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.030013510957360268, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.02524554915726185, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.024882636964321136, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.022554337978363037, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.022082101553678513, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.016292694956064224, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.016276968643069267, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.015285375528037548, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.011505048722028732, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.10.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.08111102879047394, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.07254783809185028, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.06766746938228607, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.05942074581980705, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.037567462772130966, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.03346123546361923, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.04657381772994995, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.042521990835666656, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.03834857791662216, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.03160262852907181, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.030077604576945305, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.023668518289923668, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.020309679210186005, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.018092622980475426, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.017540713772177696, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.011844425462186337, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.00947027001529932, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.009059125557541847, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.008297783322632313, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.007934422232210636, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.006231253035366535, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.006245048716664314, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.00544535368680954, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.004236916080117226, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.10.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.07359469681978226, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.06516092270612717, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.06042276322841644, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.05277974158525467, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.033782899379730225, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.029807444661855698, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.042435433715581894, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.03851376846432686, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.034619253128767014, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.02816605381667614, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.026803895831108093, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.02149576134979725, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.018389325588941574, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.016235601156949997, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.01569240540266037, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.010735702700912952, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.008401831611990929, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.007987060584127903, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.007274224888533354, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.00690606702119112, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.0055682044476270676, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.005442801862955093, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.004773732740432024, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.0034874265547841787, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.10.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.17976276576519012, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.16189919412136078, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.15510153770446777, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.1357804536819458, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.08386819809675217, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.07699833065271378, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.09768102318048477, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.08885928243398666, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.08545619249343872, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.07042080163955688, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.06619781255722046, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.049674712121486664, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04243537038564682, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04017745330929756, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.03963658586144447, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.024822253733873367, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.020555876195430756, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02002415992319584, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.017803162336349487, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.017433637753129005, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.012822971679270267, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.012464363127946854, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.011917990632355213, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.007906206883490086, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.10.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.19702740013599396, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.17771965265274048, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.16960251331329346, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.1487310379743576, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.09191865473985672, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.08419815450906754, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10898296535015106, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.09889288991689682, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09361158311367035, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.0775689110159874, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07357574999332428, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.056102555245161057, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04769488424062729, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04453796520829201, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04377840459346771, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.028172001242637634, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.023658720776438713, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.02303505875170231, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.0208913441747427, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.020396705716848373, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.015303699299693108, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.01567230373620987, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.01426894124597311, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.01145181804895401, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.10.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.16228733956813812, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.15147458016872406, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.147598534822464, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.133662149310112, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07651983946561813, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07260174304246902, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08628086000680923, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07926963269710541, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07741113752126694, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06795590370893478, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0649128332734108, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04409165307879448, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03805132955312729, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03680633008480072, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03651457279920578, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.022104473784565926, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.019228477030992508, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.018950073048472404, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01753089763224125, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.0173508133739233, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.011845353990793228, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012083268724381924, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011417168192565441, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.00851461011916399, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.10.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.22261309623718262, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.20802195370197296, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.20303837954998016, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.18380849063396454, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.1050167977809906, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.0998120978474617, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.11781039834022522, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.10840636491775513, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.10619650781154633, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.0932191014289856, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0889882817864418, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06010079383850098, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.0518934428691864, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.050366226583719254, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.050000064074993134, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.030053550377488136, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.025963500142097473, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02559644542634487, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02357950061559677, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.023345347493886948, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.015806002542376518, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.015786699950695038, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.015273896045982838, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010550056584179401, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.10.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.22963909804821014, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.20756880939006805, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.19922778010368347, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.1766587495803833, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.1061212494969368, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.0978289544582367, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12449796497821808, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11283789575099945, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10863349586725235, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.09140748530626297, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08657213300466537, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06365633755922318, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.0542290173470974, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.051196929067373276, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.05046088993549347, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.03204844519495964, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.027043066918849945, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.026687270030379295, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.02419471926987171, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.023724334314465523, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.017471883445978165, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.017611542716622353, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.01644137129187584, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.012711179442703724, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.11.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.09471038728952408, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.08441315591335297, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.07877811789512634, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.06901639699935913, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.04399825632572174, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.03916407749056816, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.05446501821279526, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.04960593208670616, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.04493807256221771, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.03683320805430412, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.03501424565911293, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.02770739234983921, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.023780781775712967, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.021279659122228622, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.020652614533901215, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.013886299915611744, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.011258203536272049, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.010791515931487083, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.009872546419501305, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.009467078372836113, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.0073661478236317635, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.007558855228126049, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.006490133237093687, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.005331674125045538, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.11.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.0793822705745697, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.07023479789495468, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.06453816592693329, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.05635121092200279, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.03643607348203659, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03185543790459633, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.04654272645711899, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.04240185767412186, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.03729833662509918, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.03040153533220291, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.02904202975332737, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.023674670606851578, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.020249824970960617, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.01757160946726799, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.016896599903702736, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.011824156157672405, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.009196280501782894, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.008705075830221176, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.00800666119903326, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.007557692006230354, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.00619993731379509, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.006152571178972721, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.005265026353299618, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.004109062720090151, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.11.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.1938401460647583, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.17283737659454346, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.1647641360759735, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.14326342940330505, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.09012022614479065, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.08195556700229645, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.10631053894758224, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.0960053950548172, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.09189220517873764, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.07470855861902237, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.06997879594564438, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.054187141358852386, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04590385779738426, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04318470135331154, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04252967983484268, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.027100728824734688, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.022095059975981712, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.021475177258253098, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.01895921491086483, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.018507080152630806, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.013989497907459736, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.013476541265845299, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.012775829993188381, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.008517906069755554, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.11.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.19575975835323334, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.17775772511959076, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.16314177215099335, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.14397133886814117, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.09221747517585754, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.08105906844139099, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.12340552359819412, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.10978596657514572, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.0938209667801857, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.0790930762887001, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.0766202062368393, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.06391671299934387, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05313868075609207, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04506020247936249, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04295142740011215, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.032097749412059784, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.024306951090693474, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.022937802597880363, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.02180151827633381, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.020510662347078323, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.017426537349820137, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.01731007546186447, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.0146400835365057, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.012545672245323658, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.11.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.1656651645898819, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.15426109731197357, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.15001104772090912, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.13573534786701202, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.078199602663517, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07397285103797913, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08861343562602997, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08129531145095825, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07914450764656067, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06920995563268661, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06611698120832443, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04535221680998802, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03911295160651207, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03772561997175217, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03739975392818451, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.022767936810851097, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.019898805767297745, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.01959347166121006, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01814555749297142, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.017941074445843697, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012317722663283348, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012772978283464909, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011842994019389153, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.009278111159801483, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.11.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.2246875762939453, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.2094203233718872, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.20410379767417908, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.18452906608581543, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.10587392002344131, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10035202652215958, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.1192048043012619, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.10953209549188614, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.1070624440908432, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09369143098592758, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0893295407295227, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06066050007939339, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05242482200264931, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05077127739787102, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05038008093833923, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.030338531360030174, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.026141522452235222, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.025747839361429214, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02367529831826687, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.023426810279488564, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.0158806461840868, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.015857579186558723, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01529465802013874, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010516893118619919, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.11.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.22915378212928772, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.20730051398277283, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.19904541969299316, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.17664343118667603, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.10618459433317184, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09784796833992004, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12465621531009674, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.1128915399312973, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10870573669672012, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.09153155237436295, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08664526045322418, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06376823782920837, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05429444834589958, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05123336985707283, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.05048857629299164, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.03208930417895317, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.027049902826547623, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.026694169268012047, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.02420920506119728, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.023741841316223145, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.017339862883090973, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.017597826197743416, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.016272757202386856, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.012688130140304565, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.12.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.09936142712831497, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.0882679894566536, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.08140312135219574, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.07137010246515274, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.04610950127243996, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04047267511487007, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.059583116322755814, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.05331159383058548, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.04718722775578499, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.03861810266971588, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.03724299371242523, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.030456092208623886, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.025676630437374115, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.02238871343433857, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.021564612165093422, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.015369811095297337, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.0120000084862113, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.011409792117774487, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.010573736391961575, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.010039178654551506, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.008258509449660778, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.008328622207045555, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.007047239691019058, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.0059978412464261055, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.12.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.08464791625738144, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.07457271218299866, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.0683441236615181, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.05965745449066162, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.0388471893966198, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03373681753873825, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.050359148532152176, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.04534270614385605, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.039873044937849045, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.03232743963599205, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.030980568379163742, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.025655295699834824, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.0217230673879385, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.01878642849624157, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.018025169149041176, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.012857151217758656, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.009926670230925083, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.009392314590513706, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.008654155768454075, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.008158053271472454, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.006775897927582264, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.006758794654160738, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.005715447477996349, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.004645415581762791, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.12.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.19185824692249298, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.17256228625774384, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.16470789909362793, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.14446885883808136, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.08987101912498474, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.08207979798316956, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.10619068145751953, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.09596749395132065, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.09148573130369186, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.07533403486013412, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.07105202227830887, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.054195694625377655, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04592197388410568, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04309283569455147, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04241381585597992, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.027088407427072525, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.02205561101436615, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.021414682269096375, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.019084369763731956, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.01862308569252491, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.013995630666613579, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.01345471478998661, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.012836312875151634, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.008503880351781845, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.12.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.20962904393672943, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.18562078475952148, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.17620067298412323, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.15445590019226074, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.09794670343399048, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.08820542693138123, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.116356261074543, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.10519334673881531, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.10026030987501144, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.08127445727586746, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07676979154348373, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.06004529073834419, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05087818577885628, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04757343977689743, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04677799716591835, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.03023526445031166, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.025310833007097244, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.024546710774302483, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.022090625017881393, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.021575355902314186, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.01639850251376629, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.0168004110455513, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.01531671267002821, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.012325716204941273, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.12.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.16898709535598755, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1571710705757141, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.15290850400924683, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.1381736844778061, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07989440113306046, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07549551129341125, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.09060564637184143, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08289023488759995, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.08087866753339767, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.07055147737264633, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06759300827980042, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04641929268836975, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.039985645562410355, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.038659267127513885, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03834491968154907, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02337837591767311, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02056017331779003, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.020253784954547882, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.018753837794065475, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.018560145050287247, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012866825796663761, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.013408723287284374, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01244029775261879, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.009980638511478901, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.12.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.23236511647701263, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.21642330288887024, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2109871655702591, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.1904449611902237, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.10978308320045471, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10397175699472427, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12320615351200104, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11323282122612, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11103887110948563, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09688219428062439, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09235727041959763, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06292006373405457, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.054283592849969864, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.052717555314302444, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05233968421816826, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03146981820464134, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02724529057741165, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.0268524382263422, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.024647196754813194, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.02441091276705265, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016622863709926605, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016672734171152115, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01608199253678322, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.01129305362701416, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.12.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.23011666536331177, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.2070072591304779, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.19730734825134277, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.17449317872524261, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.1063273698091507, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09702161699533463, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12708328664302826, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11497606337070465, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10899266600608826, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.09107682853937149, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08619516342878342, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06467711925506592, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05527600273489952, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05132497847080231, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.05035177245736122, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.03264697268605232, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.02715372107923031, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02667497657239437, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.0242130346596241, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.023597614839673042, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.017836224287748337, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.017873873934149742, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.01653091423213482, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.012847579084336758, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.13.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.10213909298181534, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.09181688725948334, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.08669238537549973, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.07614187151193619, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.04754956066608429, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04299907013773918, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.057521212846040726, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.05237749591469765, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.0484396331012249, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.040061332285404205, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.03804078698158264, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.029256463050842285, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.02503294311463833, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.0228714719414711, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.022345155477523804, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.01463894173502922, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.0118982819840312, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.011480354703962803, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.010407092981040478, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.01006257627159357, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.007670809514820576, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.00764026353135705, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.006871089804917574, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.005136272870004177, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.13.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.08364420384168625, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.07518991827964783, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.07015972584486008, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.06166722998023033, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.03878398612141609, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03463580459356308, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.04810282588005066, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.043912373483181, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.039543092250823975, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.032745540142059326, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.031173640862107277, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.024396320804953575, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.020994562655687332, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.018659228459000587, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.018081730231642723, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.012200326658785343, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.009689881466329098, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.0092628113925457, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.008484462276101112, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.008097478188574314, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.006369702983647585, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.006301886402070522, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.00556590873748064, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.004146175924688578, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.13.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.20933467149734497, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.1882956326007843, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.18069924414157867, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.158315509557724, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.09775286912918091, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.08979347348213196, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.1138065904378891, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.10322064906358719, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.09948767721652985, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.08194328844547272, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.07698874920606613, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.05798124149441719, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04929068312048912, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04681167006492615, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04621042311191559, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.028967253863811493, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.02381446212530136, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02323269098997116, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.02055460587143898, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.02015010267496109, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.014868661761283875, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.014231311157345772, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.013744518160820007, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.008764226920902729, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.13.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.22750461101531982, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.2072417140007019, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.19912861287593842, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.17833483219146729, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.10650797188282013, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.09856891632080078, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.12446413189172745, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.11342708766460419, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.10829344391822815, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.09154056757688522, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.08819243311882019, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.06406870484352112, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.0546913668513298, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.05158976837992668, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.050834186375141144, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.032115962356328964, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.027341658249497414, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.026722058653831482, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.024411490187048912, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.023940661922097206, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.01732906885445118, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.017942391335964203, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.016332853585481644, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.013093994930386543, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.13.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.17158488929271698, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1596268117427826, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.15523813664913177, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.14031267166137695, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.08117655664682388, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07666884362697601, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.09198509156703949, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08441497385501862, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.08218464255332947, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.07170307636260986, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06856146454811096, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04721067100763321, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.04066522419452667, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.039182402193546295, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03883276879787445, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02365320362150669, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.020666969940066338, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.020345376804471016, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01881992071866989, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.018607188016176224, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012764004059135914, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.013277272693812847, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.012260274961590767, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.009646973572671413, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.13.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.23705653846263885, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.22088705003261566, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2152370661497116, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.19453756511211395, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.1120653972029686, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10619194060564041, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.1260979324579239, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11580414324998856, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11338284611701965, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09901244193315506, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0943223088979721, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06442520767450333, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.055486634373664856, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05379866436123848, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05340783670544624, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.032245948910713196, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02780304290354252, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02740365080535412, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.025173520669341087, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.024921493604779243, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.017092501744627953, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01700826920568943, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.016501493752002716, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011473393999040127, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.13.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.2439960241317749, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.21875041723251343, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.20916424691677094, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.18457379937171936, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.1127302423119545, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.1031016856431961, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.13287678360939026, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.1204565018415451, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.11572746932506561, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.09608486294746399, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.0907578244805336, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06803296506404877, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05786415562033653, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05433708801865578, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.05347524583339691, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.03423671796917915, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.0285765640437603, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.0281657837331295, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.02533121034502983, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.0247786957770586, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.018537364900112152, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.01849132962524891, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.017353331670165062, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.013140566647052765, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.14.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.10424488037824631, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.09404675662517548, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.08792061358690262, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.07763238996267319, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.048578958958387375, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.043544284999370575, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.06060922145843506, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.0549757182598114, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.04951776936650276, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04127885401248932, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.03949272260069847, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.030869731679558754, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.02632269635796547, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.023477280512452126, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.022763244807720184, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.015449536964297295, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.012360786087810993, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.011845728382468224, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.010933089070022106, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.010474280454218388, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.00815539713948965, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.00824003480374813, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.0071189384907484055, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.005706408992409706, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.14.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.0841318741440773, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.07603752613067627, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.0695098266005516, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.061464838683605194, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.038894351571798325, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03407863900065422, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.051064424216747284, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.04636864736676216, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.03965947777032852, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.033343810588121414, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03209308907389641, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.025806216523051262, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.022154830396175385, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.018763301894068718, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.017897984012961388, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.012900080531835556, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.009814596734941006, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.00923085305839777, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.00871579721570015, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.008151985704898834, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.006758652627468109, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.00665030675008893, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.005608730483800173, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.004370537586510181, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.14.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.2194114476442337, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.19817328453063965, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.1897161304950714, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.1665915846824646, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.10261544585227966, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.09422661364078522, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.12187033146619797, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.10919661074876785, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.10440018773078918, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.08657117933034897, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.08166251331567764, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06234155595302582, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.05223219096660614, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04920317605137825, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04848750680685043, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.0311729833483696, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.025239553302526474, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.024584274739027023, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.021982988342642784, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.021497495472431183, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.016114113852381706, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.015443004667758942, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.014659930020570755, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.009926020167768002, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.14.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.2451043426990509, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.21659700572490692, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.20483578741550446, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.17874456942081451, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.11447533965110779, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.10334701091051102, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.1379907876253128, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.12453643232584, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.11774825304746628, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.09409315139055252, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.0903477817773819, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.07104182243347168, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.06020486354827881, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.055550310760736465, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.054439641535282135, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.03559456765651703, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.029506895691156387, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.02864748425781727, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.025540543720126152, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.02481147274374962, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.019040733575820923, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.019608929753303528, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.01755474880337715, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.014199887402355671, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.14.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.1612621545791626, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.15026816725730896, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.14586375653743744, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.13207511603832245, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07646261900663376, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07212352752685547, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08729451894760132, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08010675758123398, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07737758010625839, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06777670234441757, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06485375761985779, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.044831451028585434, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03866015747189522, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.037003252655267715, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03661582991480827, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.022499021142721176, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.019655419513583183, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.019326208159327507, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.017969045788049698, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.01773681491613388, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012186964973807335, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01285128016024828, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011632228270173073, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.009495903737843037, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.14.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.2333511859178543, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.2181156873703003, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.21250203251838684, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.19235879182815552, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11058540642261505, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10485655814409256, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12495282292366028, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11465955525636673, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.1117902547121048, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09811998158693314, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09367595613002777, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06389470398426056, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.055031534284353256, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05319531261920929, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05276918783783913, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.0320056714117527, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02765602432191372, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02723834291100502, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02515741065144539, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.024881061166524887, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.01706412434577942, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.017189128324389458, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.016435060650110245, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.01187122892588377, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.14.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.22905723750591278, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.20517916977405548, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.19521762430667877, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.1729503870010376, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.10571648925542831, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09612210839986801, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12685558199882507, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11451590061187744, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.1084742546081543, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.09032357484102249, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08582853525876999, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06501206755638123, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05518336594104767, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.0513480007648468, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.0503263846039772, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.03308285027742386, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.02759529836475849, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.027087146416306496, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.024699050933122635, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.024051770567893982, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.01840297505259514, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.018716109916567802, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.01693183183670044, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.013982602395117283, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.15.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.11666414886713028, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.10602928698062897, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.09970837086439133, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.08854666352272034, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.05452314764261246, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.049299050122499466, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.06797266751527786, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.06117452681064606, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.055459439754486084, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04683902859687805, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.04488718509674072, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03467332944273949, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.02940622717142105, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.026408664882183075, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.025664405897259712, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.0174703486263752, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.014032304286956787, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.013494862243533134, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.01255026925355196, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.012077713385224342, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.009334007278084755, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.009463821537792683, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.008200091309845448, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.006772384513169527, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.15.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.08810259401798248, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08004835993051529, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.07284713536500931, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.06478346884250641, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04067221283912659, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03553685545921326, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.05450048670172691, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.04918598383665085, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.041490521281957626, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.03524171561002731, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03419729322195053, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.027716979384422302, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.023541107773780823, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.019659943878650665, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.01862768456339836, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.013893683440983295, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.010324222035706043, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.00965107511729002, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.009246932342648506, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.008591270074248314, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.007267958018928766, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.007098592352122068, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.005894139409065247, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.004685773514211178, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.15.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.23456385731697083, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.21531420946121216, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.20743320882320404, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.1857202649116516, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.11070002615451813, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.10285980999469757, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.1297006905078888, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.1173039972782135, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.11238881200551987, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.09601468592882156, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.09145043045282364, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06627190858125687, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.0561567060649395, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05312134325504303, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.05239717289805412, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.033112697303295135, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.027156980708241463, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.026501482352614403, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.02409745194017887, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.023611770942807198, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.01705765165388584, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.01640031486749649, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.015800446271896362, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.01031299214810133, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.15.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.21899835765361786, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.1978422850370407, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.19002032279968262, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.16452579200267792, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.10246901959180832, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.09439628571271896, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.11919937282800674, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.10794547945261002, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.10393045097589493, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.08561493456363678, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07929619401693344, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.06125788763165474, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.0518881231546402, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.0493588000535965, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.048741310834884644, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.03068169765174389, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.02574397437274456, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.025154128670692444, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.02237648144364357, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.021974915638566017, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.016339421272277832, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.01626567915081978, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.015499301254749298, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.011294243857264519, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.15.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.15770180523395538, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.14742374420166016, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.14333368837833405, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.13005302846431732, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.074795663356781, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07083833962678909, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08524028956890106, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07809461653232574, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07567718625068665, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06661991029977798, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06384902447462082, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04373669996857643, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03773334249854088, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03623216971755028, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03588201850652695, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.022020608186721802, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.01930447854101658, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.01900804601609707, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.017728151753544807, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.017513221129775047, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012091686949133873, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012669697403907776, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011600361205637455, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.00945026334375143, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.15.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.23564356565475464, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.2211351990699768, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.21591977775096893, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.19587956368923187, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11163628846406937, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10627896338701248, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12551924586296082, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11538129299879074, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11277883499860764, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09955240041017532, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09522084146738052, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06404978036880493, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05523828789591789, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.053552087396383286, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05315331742167473, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03202453628182411, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.027533670887351036, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.027150021865963936, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.025073280557990074, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.024819286540150642, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016862886026501656, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016634929925203323, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01628117449581623, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010960489511489868, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.15.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.22647976875305176, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.2024482637643814, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.1927223652601242, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.17141802608966827, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.10423047840595245, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.0946464091539383, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12394973635673523, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11255349963903427, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.1070900559425354, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08901431411504745, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08450054377317429, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06320696324110031, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05404765158891678, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05026623606681824, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04934534803032875, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.031886156648397446, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.02654283121228218, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.026087269186973572, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.023602349683642387, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.023010283708572388, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.017442844808101654, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.017400845885276794, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.01620781607925892, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.012444248422980309, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.16.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.11590727418661118, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.10605204105377197, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.10048915445804596, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.08960055559873581, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.05413498356938362, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.049528010189533234, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.06562458723783493, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.059830501675605774, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05495759844779968, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04686499014496803, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.04476265236735344, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.0334007665514946, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.028617089614272118, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.026046190410852432, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.025423923507332802, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.016733521595597267, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.01356329396367073, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.013093114830553532, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.012112772092223167, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.011715425178408623, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.00880417414009571, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.008751925081014633, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.007870012894272804, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.005899733863770962, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.16.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.09108318388462067, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08345703780651093, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.07700448483228683, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.06886369735002518, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04217318817973137, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.037533994764089584, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.054534588009119034, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.049892932176589966, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.042882587760686874, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.036832045763731, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.035446710884571075, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.02755543403327465, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.023793211206793785, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.020290987566113472, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.019393084570765495, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.013776133768260479, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.010540287010371685, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.00992585439234972, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.00948261097073555, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.008879275992512703, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.007188939023762941, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.007032295223325491, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.005993238650262356, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.004482063464820385, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.16.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.23983266949653625, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.22201016545295715, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.21503150463104248, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.1933378279209137, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.11326155811548233, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.10614102333784103, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.13152951002120972, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.11893036216497421, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.11476211249828339, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.09913219511508942, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.09454678744077682, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06720900535583496, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.05683869868516922, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05423944443464279, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.05360081046819687, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.03359147533774376, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.027577079832553864, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.027025625109672546, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.024652138352394104, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.024242153391242027, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.01724315993487835, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.01637602411210537, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.016067082062363625, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.010034640319645405, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.16.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.2320849895477295, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.20865845680236816, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.20059821009635925, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.1781105250120163, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.10818787664175034, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.099416084587574, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.12471873313188553, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.11383410543203354, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.11023689061403275, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.09156417101621628, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.08662991970777512, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.06421130895614624, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05503349006175995, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.05245098099112511, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.05181943252682686, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.0323946475982666, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.027934659272432327, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.027320686727762222, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.024720456451177597, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.02433253638446331, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.017744649201631546, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.01840244047343731, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.01691754162311554, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.013658495619893074, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.16.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.14805495738983154, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.13856996595859528, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.13454031944274902, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.12218108773231506, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07016174495220184, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.06640314310789108, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08033265918493271, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07361839711666107, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.0709780603647232, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06259211897850037, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06004016101360321, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04113738611340523, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03550390899181366, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03396492078900337, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.0335921049118042, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.020667053759098053, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.018040912225842476, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.01773693785071373, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.016575664281845093, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.0163434948772192, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.011241261847317219, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.011804170906543732, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.010727626271545887, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008715817704796791, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.16.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.22845053672790527, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.21470676362514496, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.20956233143806458, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.19028256833553314, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.1082024797797203, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.1030900627374649, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12179800122976303, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11202165484428406, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.10928066819906235, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09672294557094574, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09250153601169586, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06213308870792389, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05364133045077324, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.0519101582467556, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.051498766988515854, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.031056690961122513, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.026702728122472763, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.026321053504943848, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.024355579167604446, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.02410231903195381, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016307581216096878, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016161657869815826, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.015705900266766548, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010651995427906513, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.16.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.21916551887989044, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.1945718377828598, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.1843707263469696, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.16384552419185638, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.1004827544093132, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09058739244937897, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12038393318653107, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10945025086402893, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10355225205421448, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08526989817619324, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08110543340444565, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06147739291191101, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05259532108902931, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.04852879047393799, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04750606790184975, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.03106815181672573, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.025667477399110794, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02517629787325859, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.022724425420165062, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.022072019055485725, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.016971994191408157, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.01695694774389267, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.015614386647939682, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.012133860029280186, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.17.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.09702017903327942, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.08892075717449188, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.08326403051614761, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.0748206302523613, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.045153599232435226, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04079575836658478, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.05668080598115921, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.0516507513821125, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.04597777873277664, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.03949667513370514, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.03810432553291321, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.028803909197449684, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.02470206469297409, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.021792028099298477, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.021063625812530518, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.014429976232349873, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.011458257213234901, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.010961330495774746, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.01034674234688282, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.009877747856080532, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.0075792958959937096, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.007655511610209942, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.006535821128636599, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.005238090176135302, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.17.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.08191414177417755, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.0746629536151886, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.06833002716302872, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.061438750475645065, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.03774355724453926, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03322683274745941, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.05006025731563568, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.04548916965723038, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.038546204566955566, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.03305013105273247, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03211662173271179, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.025281812995672226, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.02165038511157036, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.018216287717223167, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.01730922982096672, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.012625445611774921, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.009527072310447693, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.008940786123275757, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.008605592884123325, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.008032230660319328, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.006586558651179075, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.006493933964520693, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.005386063363403082, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.0042536910623312, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.17.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.19983750581741333, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.18037481606006622, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.17052434384822845, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.15358620882034302, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.09235408902168274, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.08370831608772278, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.1147618442773819, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.10271640121936798, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.09462103247642517, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.08016729354858398, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.07711547613143921, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.058539628982543945, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.04904281347990036, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.0443052276968956, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04314000904560089, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.029245130717754364, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.022769056260585785, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.021893257275223732, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.020283309742808342, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.01951954886317253, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.0150186438113451, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.01434130035340786, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.012997607700526714, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.009044409729540348, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.17.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.2257997989654541, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.20301946997642517, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.19423991441726685, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.17116889357566833, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.10530448704957962, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.09642394632101059, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.12401852011680603, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.1123625859618187, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.10746617615222931, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.08944348245859146, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.08399651944637299, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.0638689175248146, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.054264314472675323, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.05109007656574249, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.05034219101071358, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.032092299312353134, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.0271917674690485, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.026480693370103836, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.02412840910255909, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.02363264374434948, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.017424728721380234, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.018006546422839165, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.016407500952482224, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.013257740996778011, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.17.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.15382196009159088, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.14381498098373413, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.1396910846233368, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.12702886760234833, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07270950824022293, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.06881019473075867, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08308828622102737, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07620883733034134, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07357731461524963, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06495369225740433, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.062314413487911224, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.042544521391391754, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03672734647989273, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.035143643617630005, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03476990759372711, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.021390043199062347, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.018609371036291122, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.018298383802175522, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.017106514424085617, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.016874471679329872, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.011657150462269783, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012097058817744255, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011129972524940968, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008852720260620117, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.17.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.23629136383533478, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.221998929977417, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2167816162109375, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.1970624476671219, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11181584000587463, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10657082498073578, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12595754861831665, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11568234115839005, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.1129145696759224, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.10006839781999588, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0960417091846466, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06434372812509537, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05548098310828209, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05371786653995514, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05330934748053551, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.032316654920578, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.027896175161004066, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.027507085353136063, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.025547264143824577, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.025291267782449722, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.017312465235590935, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.017253439873456955, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.016725366935133934, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011848049238324165, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.17.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.23277296125888824, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.20573410391807556, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.1946689784526825, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.17207078635692596, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.10690053552389145, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09603912383317947, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12889841198921204, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11631854623556137, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.11022789776325226, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.09005008637905121, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.085513174533844, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06579466909170151, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05594460293650627, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05167718231678009, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.050617765635252, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.033234138041734695, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.027429264038801193, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.026894068345427513, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.024185629561543465, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.023510124534368515, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.0182279571890831, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.018219564110040665, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.0167915690690279, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.013175844214856625, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.18.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.11159697920084, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.10284176468849182, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.09691385924816132, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.08699440956115723, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.05219624936580658, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04757829010486603, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.06522989273071289, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.05899149924516678, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05303889140486717, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04585108160972595, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.04416908323764801, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.033286046236753464, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.028333332389593124, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.025280551984906197, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.02452143095433712, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.016733825206756592, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.013424481265246868, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.012912550009787083, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.012190655805170536, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.011723286472260952, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.008938467130064964, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.009078599512577057, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.00782180018723011, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.006471633445471525, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.18.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.09080873429775238, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.0825132355093956, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.074677474796772, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.06732600182294846, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04180391505360603, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03631275147199631, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.05631629005074501, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.0514843687415123, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.042789045721292496, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.03666194528341293, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03579147905111313, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.02847900614142418, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.024534741416573524, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.02019139565527439, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.019041620194911957, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.014246593229472637, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.010592938400804996, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.00984676368534565, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.009584269486367702, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.008834848180413246, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.007452117744833231, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.007347347680479288, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.005969054065644741, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.004772136453539133, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.18.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.21817098557949066, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.19805368781089783, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.18795980513095856, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.17003194987773895, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.10196997970342636, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.09285135567188263, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.12577274441719055, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.1121627613902092, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.10425809025764465, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.08868373185396194, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.08565255254507065, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.0645514577627182, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.053657468408346176, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.048984818160533905, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04783867299556732, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.03234352543950081, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.02512260526418686, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.024219023063778877, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.022392522543668747, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.021634038537740707, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.016693469136953354, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.015652792528271675, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.01456428226083517, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.009827886708080769, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.18.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.19611601531505585, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.17451535165309906, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.16691315174102783, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.14427968859672546, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.0910203754901886, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.08288411796092987, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10581089556217194, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.09621437638998032, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09244778007268906, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07538393139839172, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07042215764522552, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05457882955670357, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04667678102850914, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04434116557240486, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04377315938472748, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.027627406641840935, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.02394750341773033, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.023392587900161743, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.02099853940308094, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.020642446354031563, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.015439669601619244, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.016207506880164146, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.01468756515532732, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.012435068376362324, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.18.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.14265544712543488, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.13327494263648987, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.12921646237373352, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.11749128997325897, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.06741973012685776, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.06365444511175156, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.07725055515766144, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07099489867687225, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.06826197355985641, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.060155946761369705, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.05766630917787552, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.03952312469482422, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.0341607928276062, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.032549675554037094, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03216953203082085, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.019820544868707657, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.017160383984446526, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.01685040071606636, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.015736378729343414, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.015500363893806934, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.010679199360311031, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.011069446802139282, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01013621874153614, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.007981355302035809, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.18.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.22432978451251984, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.21059300005435944, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.20538759231567383, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.186675027012825, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.10603143274784088, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10088574141263962, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.11982519179582596, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.1101493239402771, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.10717980563640594, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09490736573934555, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09087194502353668, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.061201393604278564, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05278884619474411, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05094105750322342, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.050502434372901917, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.030659247189760208, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.026339607313275337, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02594776079058647, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02409440465271473, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.023816602304577827, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016307314857840538, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01617657206952572, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.015671780332922935, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010914131067693233, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.18.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.22135470807552338, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.19446787238121033, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.1830441653728485, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.16147533059120178, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.10138344764709473, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09030714631080627, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12372703105211258, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11153721064329147, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10473138839006424, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08512913435697556, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08101708441972733, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06333307176828384, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05381900072097778, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.0491536520421505, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04799271002411842, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.03215419501066208, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.026349464431405067, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.025791587308049202, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.023273451253771782, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.022537147626280785, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.017878437414765358, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.01791355572640896, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.016312770545482635, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.013245079666376114, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.19.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.11465656012296677, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.10600828379392624, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.09989749640226364, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.0902431458234787, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.053645115345716476, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.0489417165517807, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.06711427122354507, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.060852937400341034, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05449691414833069, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04747826233506203, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.045961130410432816, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03420788794755936, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.029157904908061028, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.025898372754454613, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.025087988004088402, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.017144372686743736, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.013635821640491486, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.013091490603983402, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.012428264133632183, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.011927100829780102, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.00912641640752554, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.009099568240344524, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.007961972616612911, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.006304722744971514, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.19.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.08965000510215759, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08232685178518295, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.07426565140485764, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.06724289059638977, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04138024151325226, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.0359480194747448, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.056460265070199966, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.051676955074071884, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.042223282158374786, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.0367119237780571, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03599132224917412, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.02862909622490406, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.02458825148642063, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.02000809647142887, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.018770568072795868, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.014324580319225788, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.010489951819181442, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.009705272503197193, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.009572271257638931, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.008795415051281452, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.007511377800256014, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.0073248897679150105, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.005949682090431452, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.004749493673443794, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.19.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.22832971811294556, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.2091153860092163, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.20005400478839874, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.18111038208007812, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.1068718433380127, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.09854882955551147, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.12875410914421082, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.11564058065414429, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.10896963626146317, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.09360790997743607, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.08994371443986893, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06586138904094696, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.05534180998802185, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05123921111226082, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.050247516483068466, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.032931022346019745, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.026108600199222565, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.025332389399409294, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.023362506181001663, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.022718077525496483, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.01687576062977314, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.015892764553427696, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.015032531693577766, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.00970155093818903, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.19.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.21137022972106934, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.1925043761730194, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.18610072135925293, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.1632358729839325, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.09859667718410492, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.09163976460695267, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.11444369703531265, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.10292671620845795, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09996694326400757, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.08387401700019836, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.0796528160572052, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.059008631855249405, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05010414123535156, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04815682768821716, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.047679923474788666, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.02986222133040428, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.026172997429966927, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.025721712037920952, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.02343345619738102, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.0231548510491848, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.016754725947976112, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.01785171777009964, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.016134297475218773, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.01397703867405653, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.19.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.13911974430084229, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.130160853266716, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.12640529870986938, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.11501024663448334, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.06569506973028183, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.06214374676346779, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.07496024668216705, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.06896717846393585, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.06651759892702103, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.05869591608643532, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.05628379061818123, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.03827868029475212, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.033143602311611176, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03168552368879318, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03133014962077141, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.019200792536139488, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.016632873564958572, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.01634667068719864, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.015251516364514828, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.015037299133837223, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.010289926081895828, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.010616136714816093, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.009803997352719307, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.007561687845736742, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.19.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.21969464421272278, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.20638231933116913, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.20135000348091125, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.18314434587955475, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.1038883775472641, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.09891800582408905, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.11729646474123001, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.10784762352705002, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.10504719614982605, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09307779371738434, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.089162178337574, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.05984938517212868, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05170333757996559, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.04993525519967079, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.04951624944806099, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.030045464634895325, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02594025991857052, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.025562485679984093, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02376994490623474, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.02350855991244316, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016147594898939133, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01608770340681076, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.015559244900941849, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011065609753131866, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.19.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.20882654190063477, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.18293501436710358, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.1708936095237732, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.1502760797739029, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09569812566041946, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08449085056781769, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.11814698576927185, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10681642591953278, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09894713014364243, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.07988262176513672, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.0761018916964531, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06059737503528595, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05178295448422432, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.046530455350875854, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04524950683116913, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.03089858964085579, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.025215450674295425, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02453891932964325, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.02227173186838627, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.02141343243420124, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.017463907599449158, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.017549382522702217, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.015780964866280556, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.013132983818650246, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.20.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.11444326490163803, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.1055014580488205, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.09825535863637924, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.0891423374414444, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.053389087319374084, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04801574721932411, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.06906714290380478, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.06236892566084862, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05436888337135315, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.0473749116063118, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.04616570472717285, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03520072624087334, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.02987632527947426, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.0258669164031744, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.024836190044879913, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.01766885630786419, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.01373202446848154, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.013081732206046581, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.012574264779686928, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.011948813684284687, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.009422007016837597, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.009440392255783081, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.007976274937391281, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.006634380668401718, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.20.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.09047260880470276, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08232685923576355, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.07258374243974686, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.06598429381847382, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04149811714887619, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.035029057413339615, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.0590379424393177, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.053789906203746796, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04254136607050896, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.0367581807076931, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.036301515996456146, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.029818270355463028, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.025577928870916367, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.020057296380400658, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.018573004752397537, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.014916702173650265, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.010609252378344536, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.009675871580839157, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.009685897268354893, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.008738551288843155, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.00780309597030282, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.00764507194980979, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.005905516445636749, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.004975953139364719, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.20.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.22678525745868683, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.2049216777086258, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.19451342523097992, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.1758013367652893, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.10528506338596344, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.09566793590784073, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.12972676753997803, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.11574698984622955, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.10772284865379333, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.09136522561311722, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.08834898471832275, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06667500734329224, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.05533298850059509, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05051372945308685, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.049303170293569565, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.03336772322654724, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.025823956355452538, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.024896109476685524, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.02298101596534252, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.02219715341925621, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.0171296838670969, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.016026947647333145, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.014846895821392536, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.00993427261710167, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.20.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.22984875738620758, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.2021685689687729, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.19181154668331146, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.16082192957401276, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.10723299533128738, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.0965796411037445, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.12582704424858093, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.11445076018571854, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.10925163328647614, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.08504299819469452, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.079674132168293, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.06471829116344452, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05526352301239967, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.051943499594926834, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.05112643912434578, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.032539453357458115, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.027630910277366638, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.02689254656434059, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.023370549082756042, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.022835325449705124, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.017570000141859055, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.018266256898641586, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.016515854746103287, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.013427401892840862, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.20.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.1456148475408554, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1367599219083786, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.13323040306568146, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.12116726487874985, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.06882210820913315, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.06545256078243256, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.07771433144807816, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07165767252445221, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.06958616524934769, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06164563074707985, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.05905027315020561, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.03964614123106003, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03439237177371979, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.033128201961517334, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.032827407121658325, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.019848518073558807, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.017282966524362564, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.017022754997015, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.015853779390454292, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.015668600797653198, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.010559497401118279, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.010853266343474388, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01013503223657608, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.007590027526021004, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.20.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.22867681086063385, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.21522530913352966, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2104169726371765, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.19131822884082794, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.10803673416376114, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10315576940774918, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12112215906381607, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11150986701250076, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.10908965766429901, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09691968560218811, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09275484830141068, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.061633262783288956, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05334612727165222, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05178083851933479, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05141318216919899, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.030911065638065338, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.026616882532835007, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.026269732043147087, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.024352600798010826, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.02411971427500248, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.01639235019683838, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016056003049016, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.015872187912464142, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010563229210674763, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.20.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.20761169493198395, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.18257753551006317, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.17165516316890717, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.15128041803836823, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09508275985717773, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08455172181129456, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.11502408981323242, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10470683872699738, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09812071174383163, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.07946012169122696, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.07559830695390701, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05888964235782623, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05035287141799927, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.04591505974531174, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.044816240668296814, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.029812291264533997, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.024339772760868073, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02380193956196308, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.021334176883101463, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.020627474412322044, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.016428286209702492, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.016223173588514328, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.014982105232775211, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.011625177226960659, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.21.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.10928948223590851, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.10006524622440338, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.09303509443998337, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.08444321900606155, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.050899043679237366, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04556712508201599, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.0658453032374382, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.05945511534810066, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05192425101995468, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.044911228120326996, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.0437929630279541, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03360670059919357, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.0284200981259346, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.024586215615272522, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.023609735071659088, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.016865352168679237, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.012933803722262383, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.012287275865674019, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.011757158674299717, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.011147325858473778, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.009004323743283749, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.008747897110879421, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.007616496179252863, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.005942426156252623, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.21.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.09122080355882645, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.0827295333147049, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.07359453290700912, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.06687337160110474, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04202055186033249, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03561447188258171, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.058935899287462234, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.053484562784433365, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04306158050894737, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.03705049678683281, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03659228980541229, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.02994987927377224, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.0255876611918211, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.02033252641558647, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.01890396699309349, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.014943400397896767, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.010692759416997433, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.009786752983927727, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.009723183698952198, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.008815012872219086, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.007780096028000116, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.00760724488645792, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.005941588431596756, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.004923895932734013, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.21.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.21679089963436127, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.19406531751155853, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.18209117650985718, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.16445887088775635, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.1000591441988945, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.08938778936862946, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.12616044282913208, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.11263936012983322, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.1027204692363739, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.08627913892269135, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.08350113779306412, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06491731852293015, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.053830377757549286, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04797002300620079, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04648280143737793, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.03252711519598961, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.024591311812400818, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02347838319838047, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.021786458790302277, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.020803524181246758, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.01671263948082924, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.015561647713184357, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.014025588519871235, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.00958433747291565, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.21.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.2303474247455597, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.200810506939888, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.19055958092212677, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.16070528328418732, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.10687829554080963, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.09497074782848358, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.12309370189905167, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.11327061057090759, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.10911892354488373, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.08520635962486267, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07752732932567596, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.06319588422775269, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.054441068321466446, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.05147664248943329, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.05077638849616051, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.03168822452425957, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.026984339579939842, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.026227418333292007, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.02272414043545723, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.022253308445215225, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.01691633090376854, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.017261743545532227, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.01593218930065632, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.012141730636358261, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.21.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.14371661841869354, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.13515472412109375, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.13193932175636292, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.12005634605884552, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.06801272928714752, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.0648375153541565, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.07643576711416245, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07048289477825165, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.06867460906505585, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.060972027480602264, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.058374855667352676, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.03906713053584099, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03381593897938728, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03271247446537018, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.032450467348098755, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.01957164704799652, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.017039747908711433, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.016797924414277077, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.015634218230843544, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.015469217672944069, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.010480672121047974, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.010633102618157864, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01012298185378313, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.007406961638480425, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.21.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.23519429564476013, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.22171618044376373, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.21705223619937897, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.19742456078529358, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.1113191619515419, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10647161304950714, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12421892583370209, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11450593173503876, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11228305846452713, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09994170069694519, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0955977588891983, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06342466175556183, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05481244996190071, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.053373727947473526, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05303028225898743, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03174586966633797, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.0274603720754385, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02712080255150795, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.025146547704935074, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.024931227788329124, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.01685454323887825, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016577906906604767, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01638275384902954, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010981686413288116, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.21.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.2052726000547409, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.18048812448978424, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.16935867071151733, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.15033024549484253, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.0936426892876625, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08314555883407593, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.11404795944690704, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10393796861171722, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09687253832817078, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.07892251014709473, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.07514623552560806, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.057932473719120026, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.049950361251831055, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.04526190087199211, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.044099144637584686, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.029377024620771408, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.02403027005493641, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.023432869464159012, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.021204207092523575, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.020439133048057556, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.016207829117774963, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.016114214435219765, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.014700859785079956, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.011512459255754948, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.22.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.10957325994968414, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.09994599968194962, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.09271310269832611, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.08412954956293106, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.050933849066495895, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.045396242290735245, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.06604039669036865, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.05968862771987915, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.052117422223091125, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.044849272817373276, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.04372585937380791, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03375091031193733, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.02858457900583744, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.024624032899737358, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.023603098466992378, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.01696127839386463, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.012966522015631199, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.012304143980145454, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.01177222654223442, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.011138373054564, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.009057406336069107, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.008805310353636742, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.007635244634002447, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.005993344821035862, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.22.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.09877453744411469, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08956434577703476, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.07973670959472656, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.0725289136171341, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.045579712837934494, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03869170323014259, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.06360765546560287, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.05781104788184166, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04672587662935257, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.040120042860507965, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03957948833703995, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.032331496477127075, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.027652788907289505, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.02208142727613449, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.020574579015374184, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.016180474311113358, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.011671765707433224, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.010717642493546009, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.010614946484565735, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.009668298065662384, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.008505980484187603, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.00833604484796524, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.006594098638743162, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.0055103180930018425, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.22.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.2146129459142685, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.19117869436740875, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.1782289296388626, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.16099302470684052, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.09880749136209488, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.08744488656520844, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.12621107697486877, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.11253862082958221, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.10159089416265488, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.08476818352937698, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.08218155801296234, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06464135646820068, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.053708694875240326, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.04738851636648178, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04576430097222328, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.032327935099601746, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.024311577901244164, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.023111021146178246, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.02147120237350464, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.02039506658911705, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.01654907315969467, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.01550542563199997, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.013749765232205391, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.009554313495755196, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.22.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.21571846306324005, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.18611060082912445, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.17531917989253998, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.14945052564144135, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.1008707731962204, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.09003767371177673, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.11816170811653137, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.10780710726976395, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.1029682606458664, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07973866164684296, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.0729612410068512, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.06060759350657463, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05197511985898018, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04882344603538513, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04804620519280434, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.030374085530638695, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.02580353058874607, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.025104764848947525, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.021735278889536858, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.02121789939701557, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.016415836289525032, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.016898665577173233, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.015385137870907784, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.0122162364423275, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.22.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.15724799036979675, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.14801055192947388, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.14456278085708618, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.13154108822345734, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07440970838069916, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07095904648303986, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08344663679599762, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07703141123056412, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07512904703617096, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06670434027910233, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06390923261642456, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.0426163449883461, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.036944154649972916, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.035757336765527725, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.035480011254549026, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.021316412836313248, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.01857135072350502, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.018309658393263817, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.017029915004968643, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.01685103215277195, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.011337440460920334, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01150318793952465, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.010952526703476906, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.007916001603007317, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.22.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.23904350399971008, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.2253643125295639, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2206437885761261, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.20078004896640778, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11306136846542358, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10818864405155182, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12597690522670746, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11627401411533356, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11410269141197205, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.1015218049287796, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09717012196779251, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06427565962076187, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05563081428408623, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05418301373720169, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05384523794054985, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.0321015864610672, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.0277982447296381, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02745252102613449, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.025441676378250122, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.02522149682044983, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.016887299716472626, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01666787452995777, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.016404885798692703, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010895858518779278, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.22.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.21165591478347778, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.18738839030265808, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.17656877636909485, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.1568971425294876, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.0969102680683136, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.0867493748664856, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.11748203635215759, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.1068744957447052, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09991614520549774, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08211133629083633, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.07816799730062485, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05971162021160126, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05125107616186142, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.0467614084482193, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04563642665743828, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.0302292350679636, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.024670062586665154, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02412590943276882, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.021808689460158348, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.021095577627420425, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.016624247655272484, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.01631348580121994, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.015167895704507828, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.011537257581949234, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.23.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.11144105345010757, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.10212317854166031, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.09526549279689789, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.0864814817905426, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.05190658941864967, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04660457372665405, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.06652797758579254, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.06018252298235893, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.0529869981110096, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04579433798789978, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.044573768973350525, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.033835429698228836, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.028793832287192345, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.025053590536117554, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.02409978397190571, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.016952916979789734, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.013133362866938114, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.012500988319516182, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.011926903389394283, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.011330543085932732, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.00897444412112236, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.008796442300081253, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.007630597334355116, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.005924706347286701, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.23.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.09834861755371094, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08901286870241165, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.08058246970176697, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.07312973588705063, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04527607560157776, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03916103392839432, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.06106571853160858, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.0556943379342556, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04643521085381508, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.039720963686704636, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03891979157924652, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.03095146454870701, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.02656635455787182, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.02190260961651802, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.020674949511885643, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.015520544722676277, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.011497283354401588, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.010694186203181744, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.010401714593172073, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.00962082203477621, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.008115554228425026, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.00796891562640667, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.006500585936009884, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.005228078458458185, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.23.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.22936303913593292, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.20820878446102142, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.19763407111167908, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.17858092486858368, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.106652170419693, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.0970284715294838, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.1297520250082016, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.11741314083337784, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.10894081741571426, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.09268859773874283, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.0893661305308342, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06626816838979721, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.056016065180301666, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05111228674650192, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04990869015455246, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.033119283616542816, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.026141654700040817, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.025206558406352997, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.023295940831303596, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.02249949984252453, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.016994085162878036, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.016197707504034042, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.014908474870026112, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.01002773828804493, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.23.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.2108493149280548, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.18377168476581573, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.17388375103473663, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.14398780465126038, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.0971338301897049, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.08699032664299011, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.11393299698829651, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.10438046604394913, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09974299371242523, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07697572559118271, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07006049156188965, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05831246078014374, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.05016935244202614, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04685966670513153, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04606541618704796, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.029204130172729492, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.024611441418528557, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.023848094046115875, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.02069205790758133, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.02015066333115101, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.015600213780999184, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.015930181369185448, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.014527887105941772, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.011241447180509567, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.23.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.16388827562332153, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1541646271944046, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.15059228241443634, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.13702112436294556, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07755082100629807, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.0739508718252182, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08709289133548737, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08020004630088806, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07829032838344574, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06949421763420105, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.0665753185749054, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04442601650953293, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.038472916930913925, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.037278834730386734, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03699669614434242, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02231348305940628, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.019385285675525665, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.01911500096321106, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.0177652295678854, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.017593974247574806, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012034471146762371, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012025880627334118, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01165598165243864, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008325144648551941, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.23.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.24376733601093292, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.22961558401584625, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2248132824897766, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.20449243485927582, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11534594744443893, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.11028263717889786, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12855149805545807, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11855839192867279, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11639999598264694, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.10343199223279953, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09889744967222214, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06556588411331177, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05673935264348984, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05528217926621437, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05493824928998947, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03280350938439369, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.028353199362754822, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.027995241805911064, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.025915885344147682, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.025698309764266014, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.017327969893813133, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.016974076628684998, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01683942787349224, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011072498746216297, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.23.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.21763044595718384, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.1923941969871521, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.18110594153404236, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.16124969720840454, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09962042421102524, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08897606283426285, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.120120570063591, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11001227796077728, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10273993015289307, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.0843755230307579, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08043208718299866, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.061254099011421204, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05274949595332146, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.0480387881398201, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04686591029167175, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.030863473191857338, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.025283673778176308, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02469497174024582, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.022322557866573334, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.021569600328803062, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.01666448265314102, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.01665089838206768, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.015107650309801102, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.011665080673992634, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.24.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.11982377618551254, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.1102212518453598, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.10303369909524918, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.09354404360055923, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.05602928623557091, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.05050589516758919, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.07108020782470703, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.06468871235847473, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05703223869204521, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.049530621618032455, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.048145826905965805, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.036255158483982086, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.030983267351984978, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.02705051563680172, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.026059577241539955, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.018155114725232124, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.014207431115210056, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.013546726666390896, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.012927585281431675, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.012312479317188263, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.009571581147611141, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.009526669979095459, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.008186751045286655, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.006474317982792854, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.24.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.105857253074646, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.09681237488985062, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.08756309002637863, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.07963361591100693, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.049108684062957764, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.0425257682800293, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.0668167695403099, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.060869138687849045, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.050196968019008636, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.04343812167644501, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.04267967864871025, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.03392903879284859, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.029076386243104935, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.0237272996455431, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.022276148200035095, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.016949808225035667, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.012385053560137749, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.011477666907012463, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.011267125606536865, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.010363537818193436, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.008854063227772713, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.008587241172790527, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.007035992108285427, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.005521143786609173, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.24.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.23770567774772644, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.21765241026878357, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.2083035111427307, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.1884317398071289, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.11104048788547516, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.1023450717329979, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.13261345028877258, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.12016896903514862, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.11312233656644821, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.09735787659883499, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.09344839304685593, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06752979010343552, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.05735821649432182, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.053194575011730194, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.05217171087861061, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.03375744819641113, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.027144575491547585, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02631070837378502, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.02430320531129837, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.023617621511220932, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.017297660931944847, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.016562573611736298, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.015533704310655594, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.01016708742827177, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.24.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.1822643131017685, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.1650846302509308, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.15686260163784027, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.1310717761516571, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.0854109600186348, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.0780898779630661, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10173016786575317, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.09326981008052826, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.08658351749181747, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07047121971845627, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.0642717108130455, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05195440351963043, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04468100890517235, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04109637439250946, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04020283371210098, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.025976350530982018, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.021295061334967613, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.020598160102963448, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.018290936946868896, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.017697649076581, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.013676178641617298, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.013509802520275116, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.012464023195207119, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.00896008126437664, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.24.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.1644883006811142, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1546863466501236, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.15115080773830414, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.13738568127155304, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07785758376121521, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07422785460948944, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08722874522209167, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08050209283828735, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07861840724945068, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06969896703958511, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06669887900352478, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.044481806457042694, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03859532251954079, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03740832582116127, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.037124037742614746, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02226800099015236, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.01938902959227562, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.01912275142967701, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.017748164013028145, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.017569800838828087, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.011835074983537197, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01194112841039896, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01145770400762558, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008156314492225647, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.24.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.24699078500270844, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.23270951211452484, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2278493493795395, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.20718933641910553, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11691351979970932, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.1118030995130539, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.13060861825942993, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.12017872929573059, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11802347749471664, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.10484223067760468, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.1002972275018692, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06657490879297256, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.057536207139492035, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05607863888144493, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.055738404393196106, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.033394187688827515, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.028808271512389183, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02845500037074089, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.026330342516303062, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.026110786944627762, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.01778838224709034, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01731196418404579, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01731100305914879, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011376766487956047, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.24.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.2215447723865509, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.19618546962738037, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.1850007325410843, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.16475047171115875, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.1013823002576828, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09081171452999115, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12227802723646164, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11167708784341812, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10460495948791504, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08610530942678452, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.0819515660405159, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06198311969637871, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05348736792802811, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.04883664473891258, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04767434298992157, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.031283020973205566, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.025610540062189102, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02503371611237526, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.022622372955083847, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.02187088504433632, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.016980202868580818, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.016727423295378685, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.01545341033488512, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.0115788159891963, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.25.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.12556305527687073, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.1159956231713295, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.10893036425113678, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.09871136397123337, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.0588366873562336, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.05336252599954605, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.07406643033027649, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.06730064749717712, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.059843894094228745, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.052128471434116364, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.05057583749294281, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03771689906716347, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.03223186358809471, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.02836601994931698, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.027398614212870598, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.018879787996411324, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.014800465665757656, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.01413826085627079, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.013466855511069298, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.012852605432271957, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.00989446323364973, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.00976674072444439, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.00851232185959816, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.006496401038020849, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.25.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.10687704384326935, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.09802205115556717, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.08918632566928864, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.08103649318218231, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.049645498394966125, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.043320197612047195, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.0663713738322258, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.060791224241256714, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.05062929913401604, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.04395134374499321, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.04291936010122299, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.03364581987261772, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.02906634286046028, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.02394874580204487, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.022595783695578575, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.016815830022096634, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.012461531907320023, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.011590085923671722, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.01133390050381422, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.010473296977579594, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.008740531280636787, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.00853308942168951, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.007042492739856243, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.00542002497240901, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.25.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.239474356174469, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.21975542604923248, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.2106943130493164, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.1905088871717453, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.11187077313661575, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.10350967943668365, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.1340796798467636, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.12072005122900009, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.11383438855409622, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.0981183797121048, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.09425124526023865, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06833413988351822, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.0576237253844738, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.053602997213602066, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.052614402025938034, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.034064799547195435, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.02730974368751049, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02648700773715973, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.024464676156640053, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.023826181888580322, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.01750899851322174, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.016577132046222687, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.01571711339056492, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.010110314004123211, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.25.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.19715285301208496, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.17291495203971863, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.1650565266609192, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.136726513504982, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.09188529849052429, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.08323013037443161, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10596141219139099, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.09645688533782959, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09369111061096191, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07316645234823227, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.06495869904756546, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05389346927404404, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04622245207428932, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04414921998977661, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04363996908068657, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.027014337480068207, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.022814135998487473, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.02226768620312214, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.01903679035604, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.018694404512643814, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.014341474510729313, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.014087200164794922, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.013615787029266357, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.009435413405299187, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.25.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.16994985938072205, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.15970930457115173, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.15601615607738495, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.14184722304344177, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.0805244967341423, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07675040513277054, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.0902675986289978, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08324528485536575, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.08130747824907303, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.07200834900140762, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06889088451862335, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.046129852533340454, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03994375467300415, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03870568424463272, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03841806575655937, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.023116545751690865, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.020098570734262466, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.019830169156193733, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.018395040184259415, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.01821562834084034, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.01236339844763279, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012440857477486134, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011966140940785408, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008563359268009663, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.25.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.25007277727127075, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.23540708422660828, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2304924726486206, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.209419846534729, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11847733706235886, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.11326760798692703, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.13172279298305511, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.12175863236188889, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11957496404647827, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.10608039051294327, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.10131092369556427, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06711410731077194, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05823390185832977, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.05675370618700981, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.056398868560791016, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03351031243801117, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.029027877375483513, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02866833098232746, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.026475690305233, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.026250038295984268, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.017450524494051933, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.017264489084482193, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.016953660175204277, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011098220013082027, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.25.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.22341962158679962, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.19779229164123535, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.18608558177947998, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.16591614484786987, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.10212691873311996, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09126339107751846, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12343217432498932, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11322055757045746, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10547378659248352, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08701349794864655, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08286767452955246, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06288120150566101, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.054267920553684235, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.049180008471012115, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04791262000799179, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.03156855329871178, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.02575099654495716, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.025119254365563393, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.02278914488852024, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.021966073662042618, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.0168871209025383, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.01683790795505047, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.015216715633869171, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.0115489661693573, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.26.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.12053371965885162, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.11076760292053223, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.10318152606487274, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.093513622879982, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.056388288736343384, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.05059025064110756, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.07188374549150467, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.06559693813323975, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05744076520204544, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04981626942753792, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.048390377312898636, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03664573282003403, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.031455881893634796, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.027232738211750984, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.026162054389715195, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.018347304314374924, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.014282265678048134, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.013571036979556084, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.012987343594431877, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.012313530780375004, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.00958429928869009, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.009602971374988556, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.008095034398138523, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.00645834906026721, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.26.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.10353386402130127, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.0944262221455574, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.08451580256223679, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.07670380175113678, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04793551564216614, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.04102899506688118, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.06642840802669525, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.06052868440747261, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04905562102794647, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.042328137904405594, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.041505519300699234, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.0335739329457283, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.02885543182492256, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.02319573424756527, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.021646760404109955, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.016795940697193146, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.012149459682404995, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.011178635992109776, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.011039599776268005, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.010068141855299473, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.008812550455331802, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.008530905470252037, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.006894449237734079, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.005481294821947813, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.26.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.23001907765865326, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.20809012651443481, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.1968235969543457, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.17770859599113464, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.1067451760172844, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.09663660824298859, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.13227549195289612, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.11854508519172668, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.10938778519630432, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.09268905967473984, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.08947689831256866, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06777966022491455, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.05658711493015289, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05123372748494148, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.049881577491760254, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.033908549696207047, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.026182422414422035, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02514859102666378, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.023289721459150314, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.02240009233355522, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.017397766932845116, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.016305483877658844, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.014977026730775833, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.009975658729672432, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.26.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.18919837474822998, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.17131733894348145, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.16575421392917633, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.14084136486053467, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.08916810154914856, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.08256521075963974, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10014063864946365, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.09214363992214203, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09038402140140533, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07386420667171478, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.06727684289216995, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.051175300031900406, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.044151365756988525, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04279990494251251, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04248662292957306, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.025581825524568558, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.02205626666545868, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.021664096042513847, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.018968136981129646, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.018748996779322624, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.013436826877295971, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.013422160409390926, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.012991671450436115, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.008961319923400879, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.26.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.17453797161579132, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.16400179266929626, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.16019724309444427, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.14550961554050446, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.08278516680002213, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07890249043703079, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.09273967146873474, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08559832721948624, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.0836155116558075, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.0739915668964386, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.07065751403570175, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04734564572572708, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.041043464094400406, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03977876156568527, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.039477959275245667, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.023681458085775375, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.020611200481653214, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.020329318940639496, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.018839513882994652, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.018649715930223465, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012530026957392693, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012689477764070034, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01211528293788433, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008654211647808552, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.26.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.2512197196483612, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.23645330965518951, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.23150409758090973, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.21027915179729462, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.11918137967586517, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.11390683054924011, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.13286268711090088, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.12249982357025146, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.12028618156909943, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.10664454847574234, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.10184229910373688, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06776771694421768, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.0586634986102581, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.057152822613716125, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.0567997507750988, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.033883705735206604, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.029367459937930107, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.029003577306866646, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.026795823127031326, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.026569750159978867, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.01788564771413803, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01764919050037861, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01738482527434826, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011606301181018353, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.26.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.22692427039146423, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.20162063837051392, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.19002698361873627, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.1692453920841217, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.10395738482475281, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09318237006664276, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.12549404799938202, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11499937623739243, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10729505121707916, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08877211809158325, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.08449645340442657, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06395233422517776, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.0551777146756649, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05010344088077545, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04885312542319298, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.03233486786484718, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.026354603469371796, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.025724049657583237, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.023392533883452415, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.022584237158298492, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.017627809196710587, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.017340756952762604, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.015992915257811546, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.012071800418198109, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.27.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.12452154606580734, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.11453695595264435, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.10809440165758133, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.09805474430322647, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.05838008224964142, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.053114622831344604, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.07246813923120499, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.06587137281894684, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05941365286707878, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.05149584263563156, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.0498381033539772, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03697812929749489, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.031559258699417114, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.028150558471679688, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.027310607954859734, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.01852261647582054, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.014705331064760685, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.01410877425223589, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.013334398157894611, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.012790704146027565, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.009739626199007034, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.009637013077735901, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.008491975255310535, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.0064606983214616776, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.27.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.10733161121606827, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.0989459753036499, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.09235842525959015, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.0837026834487915, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.05014695227146149, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.04509536176919937, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.06361188739538193, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.05822722986340523, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.05106978118419647, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.04435816779732704, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.04289370775222778, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.03225512057542801, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.02784596011042595, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.024184204638004303, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.023253321647644043, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.016180627048015594, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.012574711814522743, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.011957300826907158, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.011417492292821407, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.010832133702933788, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.008471757173538208, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.008316163904964924, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.007201797794550657, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.005438519176095724, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.27.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.22618617117404938, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.20783376693725586, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.19920216500759125, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.18024316430091858, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.10609455406665802, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.09800407290458679, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.125311478972435, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.11450386047363281, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.10792074352502823, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.09308404475450516, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.08924183249473572, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06385636329650879, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.05472434312105179, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05085771530866623, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04990743100643158, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.031863462179899216, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.02589772827923298, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02509617619216442, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.023191772401332855, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.022561104968190193, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.01634899154305458, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.015685463324189186, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.014880356378853321, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.009533638134598732, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.27.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.20709311962127686, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.18371646106243134, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.1756037026643753, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.14739514887332916, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.09714093059301376, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.08835341036319733, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.11171499639749527, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.10243193060159683, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.09894679486751556, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07798339426517487, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07212778180837631, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.057404011487960815, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04943528771400452, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04704694449901581, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.046497926115989685, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.02893695794045925, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.02502412721514702, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.02444585971534252, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.021348239853978157, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.020963555201888084, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.015841927379369736, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.016442058607935905, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.01509474590420723, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.012123816646635532, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.27.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.1696636974811554, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.15925085544586182, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.1554262638092041, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.14101669192314148, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.0805472731590271, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07662193477153778, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.09046194702386856, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.0834740698337555, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.08137940615415573, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.07186915725469589, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06864246726036072, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04620400816202164, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.04008869826793671, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.038752127438783646, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.038441333919763565, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02313356287777424, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.02013331465423107, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.019838061183691025, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01839279942214489, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.018199166283011436, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012288039550185204, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.012490564025938511, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011849011294543743, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.00861911941319704, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.27.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.243917778134346, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.22943468391895294, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.22448599338531494, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.20380203425884247, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.1157817617058754, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.11056102067232132, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.12916427850723267, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11916787177324295, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.11689942330121994, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.1034947857260704, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09887633472681046, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.06598954647779465, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.05710281804203987, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.055572204291820526, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.05520958453416824, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.0330083966255188, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.028569214046001434, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.0281955786049366, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02604973316192627, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.025813542306423187, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.017437513917684555, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.017210720106959343, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.016923565417528152, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011341173201799393, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.27.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.22700971364974976, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.20089219510555267, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.18851624429225922, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.16792292892932892, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.10385089367628098, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.09249718487262726, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.1275073140859604, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.11588882654905319, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.10735159367322922, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.08841566741466522, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.0842674970626831, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.06474127620458603, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.055522285401821136, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.05008291080594063, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04873017966747284, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.032673366367816925, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.026347137987613678, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.025668693706393242, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.023331694304943085, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.022461935877799988, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.017774414271116257, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.017407815903425217, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.015970418229699135, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.012096922844648361, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.28.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.11572492867708206, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.10626456886529922, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.09801332652568817, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.08891233056783676, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.054125506430864334, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04799175262451172, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.07131495326757431, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.0644075870513916, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05519085004925728, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04779224097728729, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.04679208621382713, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03645377233624458, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.030863337218761444, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.02618774026632309, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.024978477507829666, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.01823790743947029, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.013802701607346535, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.013025574386119843, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.012560619041323662, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.011811941862106323, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.009593253023922443, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.009462372399866581, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.007903127931058407, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.0064032673835754395, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.28.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.09731120616197586, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08851464837789536, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.07734818011522293, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.07040656358003616, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04492749646306038, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.037374380975961685, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.06498654931783676, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.059081148356199265, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04605111479759216, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.039760228246450424, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03935680538415909, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.03285251930356026, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.02815963886678219, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.021789396181702614, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.020010769367218018, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.016488082706928253, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.011497395113110542, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.01039078924804926, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.010493731126189232, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.009369997307658195, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.008600966073572636, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.008353759534657001, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.006392995826900005, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.005371517036110163, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.28.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.23069420456886292, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.2078019231557846, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.19543907046318054, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.17634005844593048, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.10710982978343964, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.09611407667398453, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.1343490481376648, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.1203572005033493, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.10976240783929825, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.0927569717168808, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.0896335318684578, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.06881999224424362, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.05757840350270271, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05147475376725197, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.049937356263399124, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.034555140882730484, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.026490207761526108, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02535529062151909, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.023588579148054123, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.022594979032874107, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.017789918929338455, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.01691047102212906, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.015112156048417091, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.010707507841289043, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.28.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.21962280571460724, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.18841341137886047, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.1776753067970276, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.15232819318771362, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.10183048993349075, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.09018845111131668, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.1187899112701416, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.10850027948617935, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.10426682978868484, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.07932937145233154, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.07580279558897018, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.06088311970233917, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.052138980478048325, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.04907821863889694, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04832329601049423, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.03049134463071823, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.025779221206903458, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.024988699704408646, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.02140943519771099, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.020911216735839844, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.016389766708016396, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.01662471890449524, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.015413561835885048, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.011755919083952904, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.28.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.15797848999500275, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1480463296175003, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.14414793252944946, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.1307307481765747, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07497101277112961, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07113786041736603, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08464109897613525, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.0781075581908226, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07579217851161957, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.0668020099401474, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06375230103731155, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04328674077987671, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03749806433916092, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03608153015375137, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.035743407905101776, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02166181430220604, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.018741797655820847, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.01844456046819687, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01710067130625248, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.01689102128148079, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.011502068489789963, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.011650791391730309, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011037035845220089, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008009687066078186, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.28.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.22683678567409515, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.2130274772644043, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.2081146389245987, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.18886259198188782, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.10766664892435074, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.10252943634986877, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.1204918920993805, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.11119615286588669, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.10873465240001678, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.09609802812337875, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.09165717661380768, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.061596956104040146, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.053289253264665604, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.0517064668238163, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.0513235367834568, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.03083229809999466, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.0266769677400589, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02631176821887493, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.024330344051122665, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.02409081533551216, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.01633727364242077, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.01625257357954979, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.015798402950167656, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.010893873870372772, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.28.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.20614174008369446, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.1810261756181717, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.168910950422287, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.14987972378730774, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09427877515554428, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08322176337242126, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.1160864531993866, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10599948465824127, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09758887439966202, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.07945822179317474, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.07577288150787354, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05936186760663986, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.051070112735033035, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.045723121613264084, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04437762871384621, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.030095098540186882, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.024472499266266823, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.023798203095793724, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.021657794713974, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.020795727148652077, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.01661045290529728, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.01673768274486065, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.014861950650811195, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.012166278436779976, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.29.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.11367612332105637, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.10432964563369751, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.09678502380847931, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.08786752074956894, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.053042247891426086, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04731567203998566, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.07015542685985565, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.06234882026910782, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05415944382548332, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04688425362110138, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.04583379998803139, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.035805508494377136, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.029868802055716515, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.025623805820941925, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.024538956582546234, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.01795237883925438, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.013443049974739552, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.012744519859552383, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.01222216710448265, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.011546935886144638, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.009424896910786629, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.009090684354305267, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.007751536555588245, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.006098092067986727, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.29.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.0973631888628006, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08995675295591354, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.08168809115886688, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.07412026077508926, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04534174129366875, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.039626553654670715, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.06180315464735031, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.05597155541181564, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04617545008659363, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.040325410664081573, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.039512503892183304, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.03140662983059883, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.02674219384789467, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.021911388263106346, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.020605646073818207, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.01570458523929119, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.01145564578473568, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.010617943480610847, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.010464434511959553, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.009630531072616577, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.00819715578109026, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.007936128415167332, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.006453374866396189, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.005081410985440016, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.29.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.24874486029148102, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.2292284071445465, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.22003290057182312, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.1993403136730194, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.11686625331640244, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.10830976068973541, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.13931411504745483, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.1263684183359146, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.11889489740133286, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.10296541452407837, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.09905707836151123, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.07132161408662796, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.06042494252324104, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05609932541847229, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.055037982761859894, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.035800572484731674, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.028698571026325226, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.027871983125805855, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.025843419134616852, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.025148339569568634, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.01840903051197529, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.01761748641729355, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.016524985432624817, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.011030805297195911, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.29.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.18839126825332642, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.16278888285160065, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.15342433750629425, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.13441592454910278, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.08790618926286697, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.07792135328054428, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.10402284562587738, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.09350671619176865, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.08984015882015228, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.06998612731695175, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.06666959822177887, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.05339683219790459, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.04496820643544197, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.042426321655511856, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.04180406033992767, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.026778001338243484, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.022253278642892838, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.021577540785074234, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.018739059567451477, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.01832115463912487, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.014231531880795956, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.014290078543126583, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.013338725082576275, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.01006705779582262, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.29.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.15341390669345856, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.14361542463302612, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.13970118761062622, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.1266121119260788, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07276876270771027, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.06895560771226883, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08237628638744354, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07598202675580978, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07361187040805817, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06476320326328278, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.061901893466711044, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04211081936955452, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03650331497192383, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03506097197532654, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03471751883625984, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.021085144951939583, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.018305864185094833, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.018002627417445183, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.016710253432393074, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.016491355374455452, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.01124162133783102, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.011525582522153854, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01075770240277052, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.008063158951699734, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.29.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.20209452509880066, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.1895570456981659, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.184932142496109, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.1678933948278427, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.09669799357652664, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.09197380393743515, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.10844431817531586, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.10006734728813171, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.09762997180223465, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.08626969903707504, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.08249043673276901, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.056075092405080795, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.04892962425947189, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.047404706478118896, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.047041989862918854, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.028243333101272583, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.0261384230107069, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.025827614590525627, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.02423454262316227, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.024021128192543983, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.015970636159181595, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.018166454508900642, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.01549572590738535, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.014622834511101246, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.29.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.20571546256542206, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.1815090775489807, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.17045757174491882, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.1500616818666458, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.09432607889175415, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.08405863493680954, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.1150156781077385, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.10463142395019531, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.09726862609386444, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.07922500371932983, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.07495211809873581, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05831105634570122, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.05018393322825432, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.045558180660009384, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.04440474137663841, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.029313212260603905, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.02412106655538082, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02354668453335762, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.02121189422905445, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.02046799287199974, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.01579117961227894, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.016077060252428055, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.014262380078434944, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.01144026592373848, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.30.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.1095728725194931, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.100243479013443, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.09128393232822418, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.08260037004947662, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.051102083176374435, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04463138431310654, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.06921995431184769, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.06240910664200783, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05219741538167, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04499661177396774, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.04423733800649643, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.0353984534740448, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.02993728592991829, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.02478559873998165, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.023427501320838928, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.017751535400748253, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.013142452575266361, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.01227540336549282, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.011954843997955322, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.011102452874183655, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.00936879962682724, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.009230590425431728, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.0075102150440216064, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.006275800988078117, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.30.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.09547897428274155, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08686858415603638, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.07536287605762482, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.0684126615524292, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04400666058063507, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.03634760528802872, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.06484340876340866, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.058620940893888474, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.04517835006117821, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.03889043256640434, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.03865814954042435, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.032974984496831894, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.027953792363405228, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.021396392956376076, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.019549990072846413, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.016563745215535164, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.01135834027081728, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.01020627561956644, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.010375534184277058, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.009200098924338818, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.008627797476947308, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.008383037522435188, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.006264181341975927, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.005445764400064945, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.30.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.22474946081638336, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.20137616991996765, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.18781088292598724, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.16936330497264862, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.10393724590539932, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.09239887446165085, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.13313470780849457, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.11901592463254929, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.10685907304286957, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.08965485543012619, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.08724135905504227, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.0685306265950203, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.05686498433351517, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.05003635585308075, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.04826974868774414, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.03444301709532738, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.025873634964227676, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.02458743378520012, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.022994350641965866, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.021843496710062027, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.01781383715569973, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.016799015924334526, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.01471610739827156, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.010716233402490616, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.30.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.16379593312740326, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.14769664406776428, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.1425313651561737, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.12055926024913788, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.07679182291030884, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.07108142971992493, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.08731945604085922, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.07948681712150574, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.07775185257196426, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.06287302076816559, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.05801357328891754, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.04490472376346588, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.03835875540971756, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.03710135444998741, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.036819204688072205, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.0225103497505188, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.01961367391049862, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.019267059862613678, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.01692737452685833, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.016717446967959404, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.012195341289043427, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.012644082307815552, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.011801408603787422, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.009256135672330856, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.30.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.15331622958183289, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.14363162219524384, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.1398492157459259, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.12690052390098572, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07327380031347275, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.0694596990942955, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.08264171332120895, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07623498886823654, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07402262091636658, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06524883955717087, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06245270371437073, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04276455193758011, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.03727343678474426, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.03593238443136215, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.035613540560007095, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02160659246146679, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.019766774028539658, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.019496247172355652, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01829102821648121, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.018106089904904366, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012209240347146988, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.013711347244679928, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.011786255054175854, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.01097308099269867, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.30.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.161906898021698, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.15187588334083557, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.14817042648792267, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.1344291865825653, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07737308740615845, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.07358581572771072, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.0868956670165062, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.08015749603509903, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07815074175596237, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06899557262659073, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06600962579250336, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.04486967250704765, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.039062194526195526, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.037813249975442886, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03752044960856438, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.022616958245635033, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.020649300888180733, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.02039312571287155, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.01909191906452179, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.01891881786286831, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.012667782604694366, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.014124766923487186, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.012267973273992538, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.011174408718943596, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.30.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.17595866322517395, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.15610191226005554, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.1475788950920105, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.12861225008964539, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.08129768073558807, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.07320835441350937, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.099279023706913, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.0883098840713501, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.08329994976520538, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.06792747229337692, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.06408126652240753, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.05014503374695778, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.042950693517923355, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.03983841836452484, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.03902662917971611, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.025673769414424896, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.021892568096518517, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.02151186391711235, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.019457319751381874, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.01897076703608036, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.014794301241636276, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.015404149889945984, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.013819348998367786, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.012092744931578636, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.31.self_attn.q_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.1061895564198494, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.0968628078699112, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.0882173702120781, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.07984215021133423, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.04949919879436493, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.043128401041030884, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.066900834441185, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.060333918780088425, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.05060521513223648, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.04350423440337181, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.04272889345884323, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.03416188061237335, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.028950683772563934, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.02398574724793434, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.022681990638375282, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.017103267833590508, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.01269147265702486, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.011865245178341866, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.011522394604980946, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.010716697201132774, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.009003915823996067, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.008881181478500366, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.007230670191347599, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.006025675218552351, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.31.self_attn.k_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.09517841041088104, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.08642280846834183, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.07565269619226456, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.06861930340528488, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.04396772384643555, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.036606211215257645, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.06458213925361633, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.05770289897918701, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.045150693506002426, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.038752928376197815, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.038485389202833176, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.03274136036634445, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.027539081871509552, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.02137724496424198, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.019634725525975227, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.016404755413532257, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.01133318804204464, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.010291951708495617, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.010328191332519054, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.009264237247407436, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.00861305184662342, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.00828433409333229, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.006385622546076775, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.005472538061439991, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.31.self_attn.v_proj", + "numel": 4194304, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.18896484375, + "total_bits": 9181184.0, + "err": 0.2369954138994217, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.37646484375, + "total_bits": 9967616.0, + "err": 0.20976445078849792, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.62646484375, + "total_bits": 11016192.0, + "err": 0.19394990801811218, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.72021484375, + "total_bits": 11409408.0, + "err": 0.17515923082828522, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.22021484375, + "total_bits": 13506560.0, + "err": 0.10949000716209412, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.68896484375, + "total_bits": 15472640.0, + "err": 0.09581846743822098, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.0316162109375, + "total_bits": 12715520.0, + "err": 0.140801802277565, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.12646484375, + "total_bits": 13113344.0, + "err": 0.12658214569091797, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.18896484375, + "total_bits": 13375488.0, + "err": 0.11301255971193314, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.53271484375, + "total_bits": 14817280.0, + "err": 0.09350759536027908, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.656982421875, + "total_bits": 15338496.0, + "err": 0.09111417084932327, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.0316162109375, + "total_bits": 16909824.0, + "err": 0.07231248170137405, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.12646484375, + "total_bits": 17307648.0, + "err": 0.06061223894357681, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.22021484375, + "total_bits": 17700864.0, + "err": 0.052722275257110596, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.31396484375, + "total_bits": 18094080.0, + "err": 0.05070376768708229, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.0316162109375, + "total_bits": 21104128.0, + "err": 0.03639683127403259, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.22021484375, + "total_bits": 21895168.0, + "err": 0.027204643934965134, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.37646484375, + "total_bits": 22550528.0, + "err": 0.025729410350322723, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.53271484375, + "total_bits": 23205888.0, + "err": 0.023973781615495682, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.72021484375, + "total_bits": 23992320.0, + "err": 0.02264346554875374, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.0316162109375, + "total_bits": 25298432.0, + "err": 0.018712390214204788, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.12646484375, + "total_bits": 25696256.0, + "err": 0.01772727072238922, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.2191162109375, + "total_bits": 26084864.0, + "err": 0.015440641902387142, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.12646484375, + "total_bits": 34084864.0, + "err": 0.01116263773292303, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.31.self_attn.o_proj", + "numel": 16777216, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1878662109375, + "total_bits": 36706304.0, + "err": 0.1016402393579483, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 39852032.0, + "err": 0.09097951650619507, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 44046336.0, + "err": 0.08753809332847595, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7191162109375, + "total_bits": 45619200.0, + "err": 0.07567872107028961, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2191162109375, + "total_bits": 54007808.0, + "err": 0.0461217425763607, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6878662109375, + "total_bits": 61872128.0, + "err": 0.04225469008088112, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 50857472.0, + "err": 0.054787587374448776, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 52434944.0, + "err": 0.04818283021450043, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1878662109375, + "total_bits": 53483520.0, + "err": 0.04686923325061798, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5316162109375, + "total_bits": 59250688.0, + "err": 0.0379980094730854, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.65643310546875, + "total_bits": 61344768.0, + "err": 0.03609740734100342, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 67634688.0, + "err": 0.02759561501443386, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 69212160.0, + "err": 0.024853995069861412, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.2191162109375, + "total_bits": 70785024.0, + "err": 0.023935209959745407, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.3128662109375, + "total_bits": 72357888.0, + "err": 0.02371959760785103, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 84411904.0, + "err": 0.014634456485509872, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.2191162109375, + "total_bits": 87562240.0, + "err": 0.014866461046040058, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.3753662109375, + "total_bits": 90183680.0, + "err": 0.014681817963719368, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.5316162109375, + "total_bits": 92805120.0, + "err": 0.013609597459435463, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.7191162109375, + "total_bits": 95950848.0, + "err": 0.013487079180777073, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 101189120.0, + "err": 0.009156208485364914, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 102766592.0, + "err": 0.011941846460103989, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218841552734375, + "total_bits": 104334848.0, + "err": 0.00888765323907137, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 136321024.0, + "err": 0.010768837295472622, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.31.mlp.gate_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.14864933490753174, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.139508455991745, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.13613034784793854, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.12340245395898819, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.07056599110364914, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.06713946163654327, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.07937955111265182, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.07310084253549576, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.07123634964227676, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.06288063526153564, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.06006962060928345, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.040562666952610016, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.035055696964263916, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.0339076891541481, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.03362968564033508, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.02027907967567444, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.01753162033855915, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.017274271696805954, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.015979034826159477, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.01580420695245266, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.010695081204175949, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.010762694291770458, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.010303606279194355, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.007279439829289913, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.31.mlp.up_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1876046316964284, + "total_bits": 128456703.99999999, + "err": 0.10389047116041183, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3751046316964284, + "total_bits": 139466752.0, + "err": 0.09726250171661377, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6251046316964284, + "total_bits": 154146816.0, + "err": 0.09485975652933121, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7188546316964284, + "total_bits": 159651840.0, + "err": 0.08593438565731049, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2188546316964284, + "total_bits": 189011968.0, + "err": 0.04926689341664314, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.6876046316964284, + "total_bits": 216537088.0, + "err": 0.04681713879108429, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031276157924107, + "total_bits": 177997312.0, + "err": 0.05551265552639961, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1251046316964284, + "total_bits": 183506944.0, + "err": 0.05106944218277931, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1876046316964284, + "total_bits": 187176960.0, + "err": 0.049770206212997437, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5313546316964284, + "total_bits": 207362048.0, + "err": 0.043839775025844574, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6563023158482144, + "total_bits": 214699008.0, + "err": 0.041905198246240616, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031276157924107, + "total_bits": 236717567.99999997, + "err": 0.028441954404115677, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.125104631696429, + "total_bits": 242227200.0, + "err": 0.024597933515906334, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.218854631696429, + "total_bits": 247732224.0, + "err": 0.023779118433594704, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.312604631696429, + "total_bits": 253237248.0, + "err": 0.02358659729361534, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031276157924107, + "total_bits": 295437824.0, + "err": 0.014284290373325348, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.218854631696429, + "total_bits": 306452480.0, + "err": 0.012519893236458302, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.375104631696429, + "total_bits": 315627520.0, + "err": 0.012341500259935856, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.531354631696429, + "total_bits": 324802560.0, + "err": 0.011453012935817242, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.718854631696429, + "total_bits": 335812608.0, + "err": 0.01133302878588438, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031276157924107, + "total_bits": 354158080.0, + "err": 0.007762065157294273, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.125104631696429, + "total_bits": 359667712.0, + "err": 0.007997272536158562, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.218776157924107, + "total_bits": 365168128.0, + "err": 0.0074990964494645596, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.125104631696429, + "total_bits": 477108224.0, + "err": 0.005783628206700087, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + }, + { + "key": "model.layers.31.mlp.down_proj", + "numel": 58720256, + "options": [ + { + "desc": "0.05:3b/0.95:2b 32g s4", + "bpw": 2.1789376395089284, + "total_bits": 127947775.99999999, + "err": 0.10926377028226852, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:3b/0.75:2b 32g s4", + "bpw": 2.3753662109375, + "total_bits": 139482112.0, + "err": 0.09788458794355392, + "qparams": { + "group_size": 32, + "bits": [ + 3, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.25:4b/0.75:2b 32g s4", + "bpw": 2.6253662109375, + "total_bits": 154162176.0, + "err": 0.09272480010986328, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 2 + ], + "bits_prop": [ + 0.25, + 0.75 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.4:3b/0.5:2b 32g s4", + "bpw": 2.7235804966517856, + "total_bits": 159929344.0, + "err": 0.08060410618782043, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3, + 2 + ], + "bits_prop": [ + 0.1, + 0.4, + 0.5 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:4b/0.9:3b 32g s4", + "bpw": 3.2235804966517856, + "total_bits": 189289472.0, + "err": 0.050845175981521606, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.2:6b/0.8:3b 32g s4", + "bpw": 3.7146519252232144, + "total_bits": 218125312.0, + "err": 0.04602187126874924, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 3 + ], + "bits_prop": [ + 0.2, + 0.8 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 128g s4", + "bpw": 3.031341552734375, + "total_bits": 178001152.0, + "err": 0.06256718933582306, + "qparams": { + "group_size": 128, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:3b 32g s4", + "bpw": 3.1253662109375, + "total_bits": 183522304.0, + "err": 0.05570872128009796, + "qparams": { + "group_size": 32, + "bits": [ + 3 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:4b/0.95:3b 32g s4", + "bpw": 3.1789376395089284, + "total_bits": 186668032.0, + "err": 0.05201271176338196, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.05, + 0.95 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:4b/0.6:3b 32g s4", + "bpw": 3.5271519252232144, + "total_bits": 207115264.0, + "err": 0.042947325855493546, + "qparams": { + "group_size": 32, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.6:4b/0.4:3b 64g s4", + "bpw": 3.6608973911830356, + "total_bits": 214968832.0, + "err": 0.040726907551288605, + "qparams": { + "group_size": 64, + "bits": [ + 4, + 3 + ], + "bits_prop": [ + 0.6, + 0.4 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 128g s4", + "bpw": 4.031341552734375, + "total_bits": 236721408.0, + "err": 0.032257549464702606, + "qparams": { + "group_size": 128, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:4b 32g s4", + "bpw": 4.1253662109375, + "total_bits": 242242560.0, + "err": 0.027281779795885086, + "qparams": { + "group_size": 32, + "bits": [ + 4 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:5b/0.9:4b 32g s4", + "bpw": 4.223580496651786, + "total_bits": 248009728.0, + "err": 0.025067172944545746, + "qparams": { + "group_size": 32, + "bits": [ + 5, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:4b 32g s4", + "bpw": 4.321794782366071, + "total_bits": 253776896.0, + "err": 0.024508720263838768, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 4 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:5b 128g s4", + "bpw": 5.031341552734375, + "total_bits": 295441664.0, + "err": 0.016622059047222137, + "qparams": { + "group_size": 128, + "bits": [ + 5 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:6b/0.9:5b 32g s4", + "bpw": 5.223580496651786, + "total_bits": 306729984.0, + "err": 0.014052143320441246, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.05:8b/0.05:6b/0.9:5b 32g s4", + "bpw": 5.339651925223214, + "total_bits": 313545728.0, + "err": 0.01378138829022646, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.05, + 0.05, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.4:6b/0.6:5b 32g s4", + "bpw": 5.527151925223214, + "total_bits": 324555776.0, + "err": 0.012646831572055817, + "qparams": { + "group_size": 32, + "bits": [ + 6, + 5 + ], + "bits_prop": [ + 0.4, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.3:6b/0.6:5b 32g s4", + "bpw": 5.723580496651786, + "total_bits": 336090112.0, + "err": 0.01232102606445551, + "qparams": { + "group_size": 32, + "bits": [ + 8, + 6, + 5 + ], + "bits_prop": [ + 0.1, + 0.3, + 0.6 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 128g s4", + "bpw": 6.031341552734375, + "total_bits": 354161920.0, + "err": 0.009740615263581276, + "qparams": { + "group_size": 128, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:6b 32g s4", + "bpw": 6.1253662109375, + "total_bits": 359683072.0, + "err": 0.010195295326411724, + "qparams": { + "group_size": 32, + "bits": [ + 6 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + }, + { + "desc": "0.1:8b/0.9:6b 128g s4", + "bpw": 6.227770124162946, + "total_bits": 365696256.0, + "err": 0.009029214270412922, + "qparams": { + "group_size": 128, + "bits": [ + 8, + 6 + ], + "bits_prop": [ + 0.1, + 0.9 + ], + "scale_bits": 4 + } + }, + { + "desc": "1.0:8b 32g s4", + "bpw": 8.1253662109375, + "total_bits": 477123584.0, + "err": 0.008230828680098057, + "qparams": { + "group_size": 32, + "bits": [ + 8 + ], + "bits_prop": [ + 1.0 + ], + "scale_bits": 4 + } + } + ] + } + ], + "last_module_idx": 66, + "base_perplexity": 6.128923067853044 +} \ No newline at end of file diff --git a/model-00001-of-00008.safetensors b/model-00001-of-00008.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..86076624c99db1803a123dc5b38e55cc937f8663 --- /dev/null +++ b/model-00001-of-00008.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8a326e1e4738e5542c4c7fcbd4a98042b656f710a58444aec9b65c03106a173f +size 1889587040 diff --git a/model-00002-of-00008.safetensors b/model-00002-of-00008.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..3a4575b4fcbe390493f7c35a6a03c7488318e9a5 --- /dev/null +++ b/model-00002-of-00008.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dd3780c719220b990e284d46d725de1e897a41d2a2c9cb91412e67eb37febd81 +size 1946243936 diff --git a/model-00003-of-00008.safetensors b/model-00003-of-00008.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..26c0552acab030f8924b14599144af19127a96a0 --- /dev/null +++ b/model-00003-of-00008.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f193bbbdbf45f96e70119c361fb65d577bd96fbc9929f6f8b8ef381f06dae78d +size 1979781432 diff --git a/model-00004-of-00008.safetensors b/model-00004-of-00008.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..2fa197250fa0dd5be92661146f603fe873c37fec --- /dev/null +++ b/model-00004-of-00008.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:57aa1516be6c70c2dc0bdc9358aa3fc21a0a9e1f495b4a489be0892a77fcc169 +size 1946243984 diff --git a/model-00005-of-00008.safetensors b/model-00005-of-00008.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..ea410bcc51cc02ad83eccd126f6af59eceddaeeb --- /dev/null +++ b/model-00005-of-00008.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e6d3de7f15f72b7efaea9fa7b4c3a8250c2b3977b44dbf6a541c9e833d87dafe +size 1979781448 diff --git a/model-00006-of-00008.safetensors b/model-00006-of-00008.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..67ee057c33802be2980c6e6716d75e1a51ff213c --- /dev/null +++ b/model-00006-of-00008.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f7802127af417a113424be223deaeb2dea0bd74518a0ae81a9bf8b7e90592f4f +size 1946243984 diff --git a/model-00007-of-00008.safetensors b/model-00007-of-00008.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..a0265feba24243083285b9be1887e82620764590 --- /dev/null +++ b/model-00007-of-00008.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2aa156167951408feeb432b4a459cab2a7b370b6d9bae8552e9438333ea7b73c +size 1979781448 diff --git a/model-00008-of-00008.safetensors b/model-00008-of-00008.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..ff3b27d743060732b3e3098b0428c6777d2e7ab3 --- /dev/null +++ b/model-00008-of-00008.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bc926df799ef165ad0dd8b4b648f6860ed06366583950a9922ada27633d4c477 +size 815834680 diff --git a/model.safetensors.index.json b/model.safetensors.index.json new file mode 100644 index 0000000000000000000000000000000000000000..fbc869b880f0c7287847c72de41d71522f62b685 --- /dev/null +++ b/model.safetensors.index.json @@ -0,0 +1,298 @@ +{ + "metadata": { + "total_size": 14483464192 + }, + "weight_map": { + "lm_head.weight": "model-00008-of-00008.safetensors", + "model.embed_tokens.weight": "model-00001-of-00008.safetensors", + "model.layers.0.input_layernorm.weight": "model-00001-of-00008.safetensors", + "model.layers.0.mlp.down_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.0.mlp.up_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00008.safetensors", + "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.1.input_layernorm.weight": "model-00001-of-00008.safetensors", + "model.layers.1.mlp.down_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.1.mlp.up_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00008.safetensors", + "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.10.input_layernorm.weight": "model-00003-of-00008.safetensors", + "model.layers.10.mlp.down_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.10.mlp.gate_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.10.mlp.up_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.10.post_attention_layernorm.weight": "model-00003-of-00008.safetensors", + "model.layers.10.self_attn.k_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.10.self_attn.o_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.10.self_attn.q_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.10.self_attn.v_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.11.input_layernorm.weight": "model-00003-of-00008.safetensors", + "model.layers.11.mlp.down_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.11.mlp.gate_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.11.mlp.up_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.11.post_attention_layernorm.weight": "model-00003-of-00008.safetensors", + "model.layers.11.self_attn.k_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.11.self_attn.o_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.11.self_attn.q_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.11.self_attn.v_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.12.input_layernorm.weight": "model-00004-of-00008.safetensors", + "model.layers.12.mlp.down_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.12.mlp.gate_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.12.mlp.up_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.12.post_attention_layernorm.weight": "model-00004-of-00008.safetensors", + "model.layers.12.self_attn.k_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.12.self_attn.o_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.12.self_attn.q_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.12.self_attn.v_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.13.input_layernorm.weight": "model-00004-of-00008.safetensors", + "model.layers.13.mlp.down_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.13.mlp.gate_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.13.mlp.up_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.13.post_attention_layernorm.weight": "model-00004-of-00008.safetensors", + "model.layers.13.self_attn.k_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.13.self_attn.o_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.13.self_attn.q_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.13.self_attn.v_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.14.input_layernorm.weight": "model-00004-of-00008.safetensors", + "model.layers.14.mlp.down_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.14.mlp.gate_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.14.mlp.up_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.14.post_attention_layernorm.weight": "model-00004-of-00008.safetensors", + "model.layers.14.self_attn.k_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.14.self_attn.o_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.14.self_attn.q_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.14.self_attn.v_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.15.input_layernorm.weight": "model-00004-of-00008.safetensors", + "model.layers.15.mlp.down_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.15.mlp.gate_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.15.mlp.up_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.15.post_attention_layernorm.weight": "model-00004-of-00008.safetensors", + "model.layers.15.self_attn.k_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.15.self_attn.o_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.15.self_attn.q_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.15.self_attn.v_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.16.input_layernorm.weight": "model-00004-of-00008.safetensors", + "model.layers.16.mlp.down_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.16.mlp.gate_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.16.mlp.up_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.16.post_attention_layernorm.weight": "model-00004-of-00008.safetensors", + "model.layers.16.self_attn.k_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.16.self_attn.o_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.16.self_attn.q_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.16.self_attn.v_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.17.input_layernorm.weight": "model-00005-of-00008.safetensors", + "model.layers.17.mlp.down_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.17.mlp.gate_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.17.mlp.up_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.17.post_attention_layernorm.weight": "model-00005-of-00008.safetensors", + "model.layers.17.self_attn.k_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.17.self_attn.o_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.17.self_attn.q_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.17.self_attn.v_proj.weight": "model-00004-of-00008.safetensors", + "model.layers.18.input_layernorm.weight": "model-00005-of-00008.safetensors", + "model.layers.18.mlp.down_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.18.mlp.gate_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.18.mlp.up_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.18.post_attention_layernorm.weight": "model-00005-of-00008.safetensors", + "model.layers.18.self_attn.k_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.18.self_attn.o_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.18.self_attn.q_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.18.self_attn.v_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.19.input_layernorm.weight": "model-00005-of-00008.safetensors", + "model.layers.19.mlp.down_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.19.mlp.gate_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.19.mlp.up_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.19.post_attention_layernorm.weight": "model-00005-of-00008.safetensors", + "model.layers.19.self_attn.k_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.19.self_attn.o_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.19.self_attn.q_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.19.self_attn.v_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.2.input_layernorm.weight": "model-00001-of-00008.safetensors", + "model.layers.2.mlp.down_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.2.mlp.up_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00008.safetensors", + "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.20.input_layernorm.weight": "model-00005-of-00008.safetensors", + "model.layers.20.mlp.down_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.20.mlp.gate_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.20.mlp.up_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.20.post_attention_layernorm.weight": "model-00005-of-00008.safetensors", + "model.layers.20.self_attn.k_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.20.self_attn.o_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.20.self_attn.q_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.20.self_attn.v_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.21.input_layernorm.weight": "model-00006-of-00008.safetensors", + "model.layers.21.mlp.down_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.21.mlp.gate_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.21.mlp.up_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.21.post_attention_layernorm.weight": "model-00006-of-00008.safetensors", + "model.layers.21.self_attn.k_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.21.self_attn.o_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.21.self_attn.q_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.21.self_attn.v_proj.weight": "model-00005-of-00008.safetensors", + "model.layers.22.input_layernorm.weight": "model-00006-of-00008.safetensors", + "model.layers.22.mlp.down_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.22.mlp.gate_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.22.mlp.up_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.22.post_attention_layernorm.weight": "model-00006-of-00008.safetensors", + "model.layers.22.self_attn.k_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.22.self_attn.o_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.22.self_attn.q_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.22.self_attn.v_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.23.input_layernorm.weight": "model-00006-of-00008.safetensors", + "model.layers.23.mlp.down_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.23.mlp.gate_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.23.mlp.up_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.23.post_attention_layernorm.weight": "model-00006-of-00008.safetensors", + "model.layers.23.self_attn.k_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.23.self_attn.o_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.23.self_attn.q_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.23.self_attn.v_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.24.input_layernorm.weight": "model-00006-of-00008.safetensors", + "model.layers.24.mlp.down_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.24.mlp.gate_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.24.mlp.up_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.24.post_attention_layernorm.weight": "model-00006-of-00008.safetensors", + "model.layers.24.self_attn.k_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.24.self_attn.o_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.24.self_attn.q_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.24.self_attn.v_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.25.input_layernorm.weight": "model-00006-of-00008.safetensors", + "model.layers.25.mlp.down_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.25.mlp.gate_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.25.mlp.up_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.25.post_attention_layernorm.weight": "model-00006-of-00008.safetensors", + "model.layers.25.self_attn.k_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.25.self_attn.o_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.25.self_attn.q_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.25.self_attn.v_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.26.input_layernorm.weight": "model-00007-of-00008.safetensors", + "model.layers.26.mlp.down_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.26.mlp.gate_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.26.mlp.up_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.26.post_attention_layernorm.weight": "model-00007-of-00008.safetensors", + "model.layers.26.self_attn.k_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.26.self_attn.o_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.26.self_attn.q_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.26.self_attn.v_proj.weight": "model-00006-of-00008.safetensors", + "model.layers.27.input_layernorm.weight": "model-00007-of-00008.safetensors", + "model.layers.27.mlp.down_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.27.mlp.gate_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.27.mlp.up_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.27.post_attention_layernorm.weight": "model-00007-of-00008.safetensors", + "model.layers.27.self_attn.k_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.27.self_attn.o_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.27.self_attn.q_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.27.self_attn.v_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.28.input_layernorm.weight": "model-00007-of-00008.safetensors", + "model.layers.28.mlp.down_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.28.mlp.gate_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.28.mlp.up_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.28.post_attention_layernorm.weight": "model-00007-of-00008.safetensors", + "model.layers.28.self_attn.k_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.28.self_attn.o_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.28.self_attn.q_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.28.self_attn.v_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.29.input_layernorm.weight": "model-00007-of-00008.safetensors", + "model.layers.29.mlp.down_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.29.mlp.gate_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.29.mlp.up_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.29.post_attention_layernorm.weight": "model-00007-of-00008.safetensors", + "model.layers.29.self_attn.k_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.29.self_attn.o_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.29.self_attn.q_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.29.self_attn.v_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.3.input_layernorm.weight": "model-00002-of-00008.safetensors", + "model.layers.3.mlp.down_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.3.mlp.up_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.3.post_attention_layernorm.weight": "model-00002-of-00008.safetensors", + "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00008.safetensors", + "model.layers.30.input_layernorm.weight": "model-00008-of-00008.safetensors", + "model.layers.30.mlp.down_proj.weight": "model-00008-of-00008.safetensors", + "model.layers.30.mlp.gate_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.30.mlp.up_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.30.post_attention_layernorm.weight": "model-00008-of-00008.safetensors", + "model.layers.30.self_attn.k_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.30.self_attn.o_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.30.self_attn.q_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.30.self_attn.v_proj.weight": "model-00007-of-00008.safetensors", + "model.layers.31.input_layernorm.weight": "model-00008-of-00008.safetensors", + "model.layers.31.mlp.down_proj.weight": "model-00008-of-00008.safetensors", + "model.layers.31.mlp.gate_proj.weight": "model-00008-of-00008.safetensors", + "model.layers.31.mlp.up_proj.weight": "model-00008-of-00008.safetensors", + "model.layers.31.post_attention_layernorm.weight": "model-00008-of-00008.safetensors", + "model.layers.31.self_attn.k_proj.weight": "model-00008-of-00008.safetensors", + "model.layers.31.self_attn.o_proj.weight": "model-00008-of-00008.safetensors", + "model.layers.31.self_attn.q_proj.weight": "model-00008-of-00008.safetensors", + "model.layers.31.self_attn.v_proj.weight": "model-00008-of-00008.safetensors", + "model.layers.4.input_layernorm.weight": "model-00002-of-00008.safetensors", + "model.layers.4.mlp.down_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.4.mlp.gate_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.4.mlp.up_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.4.post_attention_layernorm.weight": "model-00002-of-00008.safetensors", + "model.layers.4.self_attn.k_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.4.self_attn.o_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.4.self_attn.q_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.4.self_attn.v_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.5.input_layernorm.weight": "model-00002-of-00008.safetensors", + "model.layers.5.mlp.down_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.5.mlp.gate_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.5.mlp.up_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.5.post_attention_layernorm.weight": "model-00002-of-00008.safetensors", + "model.layers.5.self_attn.k_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.5.self_attn.o_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.5.self_attn.q_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.5.self_attn.v_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.6.input_layernorm.weight": "model-00002-of-00008.safetensors", + "model.layers.6.mlp.down_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.6.mlp.gate_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.6.mlp.up_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.6.post_attention_layernorm.weight": "model-00002-of-00008.safetensors", + "model.layers.6.self_attn.k_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.6.self_attn.o_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.6.self_attn.q_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.6.self_attn.v_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.7.input_layernorm.weight": "model-00002-of-00008.safetensors", + "model.layers.7.mlp.down_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.7.mlp.gate_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.7.mlp.up_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.7.post_attention_layernorm.weight": "model-00002-of-00008.safetensors", + "model.layers.7.self_attn.k_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.7.self_attn.o_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.7.self_attn.q_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.7.self_attn.v_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.8.input_layernorm.weight": "model-00003-of-00008.safetensors", + "model.layers.8.mlp.down_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.8.mlp.gate_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.8.mlp.up_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.8.post_attention_layernorm.weight": "model-00003-of-00008.safetensors", + "model.layers.8.self_attn.k_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.8.self_attn.o_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.8.self_attn.q_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.8.self_attn.v_proj.weight": "model-00002-of-00008.safetensors", + "model.layers.9.input_layernorm.weight": "model-00003-of-00008.safetensors", + "model.layers.9.mlp.down_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.9.mlp.gate_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.9.mlp.up_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.9.post_attention_layernorm.weight": "model-00003-of-00008.safetensors", + "model.layers.9.self_attn.k_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.9.self_attn.o_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.9.self_attn.q_proj.weight": "model-00003-of-00008.safetensors", + "model.layers.9.self_attn.v_proj.weight": "model-00003-of-00008.safetensors", + "model.norm.weight": "model-00008-of-00008.safetensors" + } +} diff --git a/out_tensor/model.layers.0.mlp.down_proj.safetensors b/out_tensor/model.layers.0.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..e2445eb8326473d9873d455caa0d2cf80e7a3138 --- /dev/null +++ b/out_tensor/model.layers.0.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0d1983e86d69750adc6e8dfa80ec58a21264489c3d0c739ac2dbdf0c04bc5ea3 +size 20049048 diff --git a/out_tensor/model.layers.0.mlp.gate_proj.safetensors b/out_tensor/model.layers.0.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..a6c5eb842f07b1b719ceb97bab14d0a0750e3ca8 --- /dev/null +++ b/out_tensor/model.layers.0.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:60ccaa791f7ac9d35b367458692f4b7316aaa564808ec874625bc1cc3964d6d4 +size 22955288 diff --git a/out_tensor/model.layers.0.mlp.up_proj.safetensors b/out_tensor/model.layers.0.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..7988d0dc39734209e2f0084a1f3e537efaaa1dd5 --- /dev/null +++ b/out_tensor/model.layers.0.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d25098a46a2f7eac5b7732611386670086d0206ed530adfa06dd4c09c1cc688b +size 26854280 diff --git a/out_tensor/model.layers.0.self_attn.k_proj.safetensors b/out_tensor/model.layers.0.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..4c2070da477be6f42cf9489367ee25ba278b29d3 --- /dev/null +++ b/out_tensor/model.layers.0.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f962a6d27245894d2ed152add3517744f4bb444ea798a1e7f49528416d704915 +size 1164576 diff --git a/out_tensor/model.layers.0.self_attn.o_proj.safetensors b/out_tensor/model.layers.0.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..e5285d993fe85d6f3cf7ddd020c41869254c4327 --- /dev/null +++ b/out_tensor/model.layers.0.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:945b96148e665c2f211801ced07c9d5535980d94c2d73228a0963382858cfc30 +size 5522720 diff --git a/out_tensor/model.layers.0.self_attn.q_proj.safetensors b/out_tensor/model.layers.0.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..da99b643b2376b700558370c5fe77500f1ccc63c --- /dev/null +++ b/out_tensor/model.layers.0.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:36f5a44316264ad4ed0229ce60c4faee41f1fe66916f371264df26fba2b469ae +size 4605216 diff --git a/out_tensor/model.layers.0.self_attn.v_proj.safetensors b/out_tensor/model.layers.0.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..99ceac9d42a111257e5d8abee6ba7da2e18865f9 --- /dev/null +++ b/out_tensor/model.layers.0.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:60e7099141846823b5fb6fbd9d9031b0abe9dd56655062aed8e767bcb1d12e38 +size 1393952 diff --git a/out_tensor/model.layers.1.mlp.down_proj.safetensors b/out_tensor/model.layers.1.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..7cd8dd685a29e8e58ab12aeacf35b676543c2fd2 --- /dev/null +++ b/out_tensor/model.layers.1.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b6637af0fc44d9e3da9d3bfb4fd48cc6da35766b577580419eb78ee4a53f2cee +size 16051352 diff --git a/out_tensor/model.layers.1.mlp.gate_proj.safetensors b/out_tensor/model.layers.1.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..28eb642c5043cb5d69eb88dafe1bc4dc67e894df --- /dev/null +++ b/out_tensor/model.layers.1.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2db8d7864bf3af6ea3b346ee63e903f45d5a53f8e3de4a6218d84765801a83b3 +size 29606616 diff --git a/out_tensor/model.layers.1.mlp.up_proj.safetensors b/out_tensor/model.layers.1.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..ebfb96388a19698b85983ce64cdcffa26424a100 --- /dev/null +++ b/out_tensor/model.layers.1.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:42280be30b8f3378a3e706fa58f92ed787200fb6661df5332abc8dc6c7d4479e +size 29606600 diff --git a/out_tensor/model.layers.1.self_attn.k_proj.safetensors b/out_tensor/model.layers.1.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..9dd5176a16e88b375d14a321a6a864c32415e9f1 --- /dev/null +++ b/out_tensor/model.layers.1.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ddfc8a157c9b51f164ae2c37f9fb9df2d92f17d018e0eca861c1640418f874de +size 1164576 diff --git a/out_tensor/model.layers.1.self_attn.o_proj.safetensors b/out_tensor/model.layers.1.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..2ec2ef6dd95d7b3e17bdcc78fb8ec59dec3f91f9 --- /dev/null +++ b/out_tensor/model.layers.1.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e5ad17428f192abc2a1d259b9d4827bf2833e4a1e9ef1d7b9f5939a800aa47c5 +size 7423264 diff --git a/out_tensor/model.layers.1.self_attn.q_proj.safetensors b/out_tensor/model.layers.1.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..50f22b16bc594591a7c04a28b4bc96697a321c25 --- /dev/null +++ b/out_tensor/model.layers.1.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d3f66cef70804505bd5d10823d4bbe18ae82491e685b846027b3b6504eb253c1 +size 4605216 diff --git a/out_tensor/model.layers.1.self_attn.v_proj.safetensors b/out_tensor/model.layers.1.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..d6782e25895110f6c33f774db2756b4db63948ac --- /dev/null +++ b/out_tensor/model.layers.1.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9bc220e72208769a2b5f9d3c37377fe0d147841ac49077f5235848b2944998e6 +size 1869088 diff --git a/out_tensor/model.layers.10.mlp.down_proj.safetensors b/out_tensor/model.layers.10.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..c5cce5e80852974b9a62eabb4655dbab42532e94 --- /dev/null +++ b/out_tensor/model.layers.10.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fa3a00b84a0aa667bb0cb98f67936e817e2efa758cb8c1fbfd77183d81409b0e +size 30338208 diff --git a/out_tensor/model.layers.10.mlp.gate_proj.safetensors b/out_tensor/model.layers.10.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..4b10f02c5117495cdfab5ea38f0ac6a56c6ede82 --- /dev/null +++ b/out_tensor/model.layers.10.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c237b1d1e38398f2f654ec574d303fb730a53e7397649f21d10c86dafc17b70c +size 29606616 diff --git a/out_tensor/model.layers.10.mlp.up_proj.safetensors b/out_tensor/model.layers.10.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..324da32265447a7eadcd6c3cb66e783b1fdf3a2c --- /dev/null +++ b/out_tensor/model.layers.10.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:84bb56054223615066a5e4ebe741bd6f22493af6d717a9202326bcadb14031c3 +size 30295312 diff --git a/out_tensor/model.layers.10.self_attn.k_proj.safetensors b/out_tensor/model.layers.10.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..724b2ccf417b9a656c5165ecd702e9d205fdd325 --- /dev/null +++ b/out_tensor/model.layers.10.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:03f525c29d9de8aaaa2926f1db9806a8ea170b195c70e0ec6844c374f01143c4 +size 1443104 diff --git a/out_tensor/model.layers.10.self_attn.o_proj.safetensors b/out_tensor/model.layers.10.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..fec1eeb7db4d9f1e8b2a55a6267d5f029de1460c --- /dev/null +++ b/out_tensor/model.layers.10.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:48d0a028e977c10ec32ff2902f60ebf0213165285633cc53c693c7e745985ca0 +size 8668456 diff --git a/out_tensor/model.layers.10.self_attn.q_proj.safetensors b/out_tensor/model.layers.10.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..dfbe75b7feed867f7108e38dd46e92cafc136876 --- /dev/null +++ b/out_tensor/model.layers.10.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8a7309bb2184ff9d54cd0c67b21e62a6e1dcfec0377ca8216e2ee8360343a648 +size 6374112 diff --git a/out_tensor/model.layers.10.self_attn.v_proj.safetensors b/out_tensor/model.layers.10.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..bbddfaff4f17d6f23708e1358a647a8c89786e02 --- /dev/null +++ b/out_tensor/model.layers.10.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bfb41ceec5dfdc2a3c1c31e412c6142a210e6dcdf76064b2d79f96e065e4f6fc +size 2130656 diff --git a/out_tensor/model.layers.11.mlp.down_proj.safetensors b/out_tensor/model.layers.11.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..fe79862cb2e6a559c755eb2fb7e9d0fe1cf43912 --- /dev/null +++ b/out_tensor/model.layers.11.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9d543a52f4e4b40ff9f6227b236b2a004b2f52acb8c33fe1d30df7e8494ddcff +size 31059104 diff --git a/out_tensor/model.layers.11.mlp.gate_proj.safetensors b/out_tensor/model.layers.11.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..c00f4ffd400dc7ef6605aa471fd996103c8b16ac --- /dev/null +++ b/out_tensor/model.layers.11.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:87e32ee466510d6ab403f8cc2d0af7df387f27caf4d2f7fb6a71a42dd934b7c4 +size 29606616 diff --git a/out_tensor/model.layers.11.mlp.up_proj.safetensors b/out_tensor/model.layers.11.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..cbfc0ca8f8d645fd20f4253507208f383e06cf23 --- /dev/null +++ b/out_tensor/model.layers.11.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:137348a44ac8a18806d7ad65e357fbd37daabe14b2214b8fb23d9e970a31bd2a +size 30295312 diff --git a/out_tensor/model.layers.11.self_attn.k_proj.safetensors b/out_tensor/model.layers.11.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..a687f749c9934ba299b07dbbde4a14fd1a1042e2 --- /dev/null +++ b/out_tensor/model.layers.11.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1d081fdc51ee72f90e67317bbc247295653993b52f6ef061ffa74e4d3e942adb +size 1606368 diff --git a/out_tensor/model.layers.11.self_attn.o_proj.safetensors b/out_tensor/model.layers.11.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..c94df344bcfb566ad5fe95c7c3848d655d0f98c6 --- /dev/null +++ b/out_tensor/model.layers.11.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:61399c8ee13597e9253816c116c94195689428bd2e2a54c0ad193bcefd9b8986 +size 8668456 diff --git a/out_tensor/model.layers.11.self_attn.q_proj.safetensors b/out_tensor/model.layers.11.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..4f11de9457fcf3baa4070364897acb6ea985508f --- /dev/null +++ b/out_tensor/model.layers.11.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5c9e4aedd433d1d66b5ad4b1b7c411681a1a61a0614aa5ebfde8f95e9eb235e5 +size 6571304 diff --git a/out_tensor/model.layers.11.self_attn.v_proj.safetensors b/out_tensor/model.layers.11.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..802786e531e4439254e6cc584eed6ff5cc73569c --- /dev/null +++ b/out_tensor/model.layers.11.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a1f18defa8985c65ec0a700c7eddfd4b47237b5ca8840e203350c6378245f0ab +size 2130656 diff --git a/out_tensor/model.layers.12.mlp.down_proj.safetensors b/out_tensor/model.layers.12.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..997a5972f70975b23703b0313dee7f759a1d2066 --- /dev/null +++ b/out_tensor/model.layers.12.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9d248304654a13e1e994d09651c86765e276627a131e281ba7acd2ac7d1ff304 +size 31059104 diff --git a/out_tensor/model.layers.12.mlp.gate_proj.safetensors b/out_tensor/model.layers.12.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..e09c9e588cb986f96e301350bac2f6db7f05de62 --- /dev/null +++ b/out_tensor/model.layers.12.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ae30a9e702f9a95cb704d3ac4e8edc9712c658380d3e170fca79f4e86a685037 +size 29606616 diff --git a/out_tensor/model.layers.12.mlp.up_proj.safetensors b/out_tensor/model.layers.12.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..a06ca498a132c615d78d9be12fde811fd254bcaa --- /dev/null +++ b/out_tensor/model.layers.12.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e043ef2ccf059c5397a139d22bb2ef4ec883a6a4661816d4a20c859cec7681a7 +size 30295312 diff --git a/out_tensor/model.layers.12.self_attn.k_proj.safetensors b/out_tensor/model.layers.12.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..3bdeb42f287f2231bbb0310933f2837bac6810bd --- /dev/null +++ b/out_tensor/model.layers.12.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:83b118eb4581030eb6c0c9236afdaf4b1fcee5cce77cdfd855a19f44473fff42 +size 1606368 diff --git a/out_tensor/model.layers.12.self_attn.o_proj.safetensors b/out_tensor/model.layers.12.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..2e4677df5f0af9cfdda928e04fd25cfec07ad725 --- /dev/null +++ b/out_tensor/model.layers.12.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:687c9d75c30dab3b3b0f848cb8ba932e11cf9c6ba0821e1766dd7df087879020 +size 8668456 diff --git a/out_tensor/model.layers.12.self_attn.q_proj.safetensors b/out_tensor/model.layers.12.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..b28514fa9e97369324ca1115076c2db33a631cfa --- /dev/null +++ b/out_tensor/model.layers.12.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f01ceda5c88cb1fd102420da6f725400714b495a8e9ad5ab7815c422f64f9c90 +size 6571304 diff --git a/out_tensor/model.layers.12.self_attn.v_proj.safetensors b/out_tensor/model.layers.12.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..966f3ca8150cc4497adb19700e31016eaa682103 --- /dev/null +++ b/out_tensor/model.layers.12.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:50ebcc4b32bbea5db0ba7cb3ead916ad88e5cafdd767c130ac5519c7fccc6629 +size 2130656 diff --git a/out_tensor/model.layers.13.mlp.down_proj.safetensors b/out_tensor/model.layers.13.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..56677569f19cd5f4e1152324ba44dfa2f6321bb6 --- /dev/null +++ b/out_tensor/model.layers.13.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a9a55d73f3e9dac0631194a35090a23ad3d00bd156e7f0bb043b7036e4d22498 +size 31780000 diff --git a/out_tensor/model.layers.13.mlp.gate_proj.safetensors b/out_tensor/model.layers.13.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..c0e218b185fe78509c62c1e2ef2fa2721f4c58f2 --- /dev/null +++ b/out_tensor/model.layers.13.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b8d4a3db6426a96df90fa5fb5237135fb089c2357c3e8734c40099a51d43df24 +size 29606616 diff --git a/out_tensor/model.layers.13.mlp.up_proj.safetensors b/out_tensor/model.layers.13.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..50b7da51badf3e9fdacbab22cd8ff42a6906f7b0 --- /dev/null +++ b/out_tensor/model.layers.13.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7d44c428faaeb8995cd8528a6cba0aa2943af1904deb58fb2ed5b4d8b359a712 +size 30983440 diff --git a/out_tensor/model.layers.13.self_attn.k_proj.safetensors b/out_tensor/model.layers.13.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..731c8875ac1a708517b656f691f49e373e6e495b --- /dev/null +++ b/out_tensor/model.layers.13.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0ad3873c4756b83830ebc0202d596f99fe9f972b0633e7745636e94f4fe4fdeb +size 1606368 diff --git a/out_tensor/model.layers.13.self_attn.o_proj.safetensors b/out_tensor/model.layers.13.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..da23cffcb29b57e307fc8b2e4411a27ee9e0b150 --- /dev/null +++ b/out_tensor/model.layers.13.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4e53c8e0aee98efb2dd008bb23e09c2471e4c9b02dd34fe42b44b242f14a1a61 +size 8865064 diff --git a/out_tensor/model.layers.13.self_attn.q_proj.safetensors b/out_tensor/model.layers.13.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..3242ab95614b687f409a6ace497553b01328e511 --- /dev/null +++ b/out_tensor/model.layers.13.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9d99b5b61d97eb836fdefbb5d67fb7da292302efdbe745ae9fac3954aab3ffe2 +size 6571304 diff --git a/out_tensor/model.layers.13.self_attn.v_proj.safetensors b/out_tensor/model.layers.13.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..504a338382b8cb3e9fb989d466ce1356a6e4e73a --- /dev/null +++ b/out_tensor/model.layers.13.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a1bba7557ece2ec72765703862238b8b6a585b013be3585b0731fcef730a4949 +size 2180384 diff --git a/out_tensor/model.layers.14.mlp.down_proj.safetensors b/out_tensor/model.layers.14.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..bbe5003bc1829899ef1a9f2d216428fbbe017570 --- /dev/null +++ b/out_tensor/model.layers.14.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3ab739601a698492ab2a576527a1b963a92143910b9662496d0e7ec3ee46f133 +size 31059104 diff --git a/out_tensor/model.layers.14.mlp.gate_proj.safetensors b/out_tensor/model.layers.14.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..c449f67c12b21d71cdb92d56d7a5b5dfc2e2ccf2 --- /dev/null +++ b/out_tensor/model.layers.14.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9b26b02fdc4650a4dc25525393a840721434c7acc963e7fad425d0155ab6d7b1 +size 29606616 diff --git a/out_tensor/model.layers.14.mlp.up_proj.safetensors b/out_tensor/model.layers.14.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..7682df81557edc1ebb1481df23db2cf43479ff46 --- /dev/null +++ b/out_tensor/model.layers.14.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8886c374c89f7dd4587ab91f27a9eaf5bcdb362deae187feeb8317b004ef5897 +size 30983440 diff --git a/out_tensor/model.layers.14.self_attn.k_proj.safetensors b/out_tensor/model.layers.14.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..d7bd3cc63e877c98ecf56b9c7409529a3e318fb8 --- /dev/null +++ b/out_tensor/model.layers.14.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d3adffaf8350398455dcfc23e29f233bc2fe30f843b3f166413b4747fda2d5de +size 1606368 diff --git a/out_tensor/model.layers.14.self_attn.o_proj.safetensors b/out_tensor/model.layers.14.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..e9ca98f00959f798186fbe8a15aef7884bf7d85f --- /dev/null +++ b/out_tensor/model.layers.14.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c8a50b822428643e0ed4c337c493dbb408f585947993633b4ee95ddac54640a8 +size 10568424 diff --git a/out_tensor/model.layers.14.self_attn.q_proj.safetensors b/out_tensor/model.layers.14.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..75461747084f2c34ceb52bfed026f5c4b2ae933a --- /dev/null +++ b/out_tensor/model.layers.14.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:34d347aa94b996c2ebdc877c28ead60270cd3d2cbb5f48253f198eebe2329932 +size 6702376 diff --git a/out_tensor/model.layers.14.self_attn.v_proj.safetensors b/out_tensor/model.layers.14.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..681c5e85a1300666563e3b9dfc777760e5eda527 --- /dev/null +++ b/out_tensor/model.layers.14.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:317491e08e4508008776a798cb64e27dc5ee320e305a0a8ee4d762b3202a29c1 +size 2180384 diff --git a/out_tensor/model.layers.15.mlp.down_proj.safetensors b/out_tensor/model.layers.15.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..fce270e283aaa26fbdd2a1d2dceb0ef588a2fa9e --- /dev/null +++ b/out_tensor/model.layers.15.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ecc3a56184ff4a1d8f583a70c970fb7a77fcff10ddd0aa00feba7e0f9037b15c +size 30338208 diff --git a/out_tensor/model.layers.15.mlp.gate_proj.safetensors b/out_tensor/model.layers.15.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..1caf38e3190c3070d4e6f9037832b1a4687b69af --- /dev/null +++ b/out_tensor/model.layers.15.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c160a44fd2920fb1fb439280af9cd3fbc6c7aa83f25d93d0405094d68d42ae70 +size 29606616 diff --git a/out_tensor/model.layers.15.mlp.up_proj.safetensors b/out_tensor/model.layers.15.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..8485e1a9db0c2b9e7513e011f8535f15b0567002 --- /dev/null +++ b/out_tensor/model.layers.15.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f39d5c4bc4bc3ae43027717249f68ff6364011ab5b84040ba8634c0cee0ceb30 +size 30983440 diff --git a/out_tensor/model.layers.15.self_attn.k_proj.safetensors b/out_tensor/model.layers.15.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..32ab7a270a89e82a4ce18749a5f23df17c22349f --- /dev/null +++ b/out_tensor/model.layers.15.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:58907d4cee279bd4502c842d73541260602f00377d56cf9bb5a54c9ac31e4957 +size 1656096 diff --git a/out_tensor/model.layers.15.self_attn.o_proj.safetensors b/out_tensor/model.layers.15.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..f4502489400153ed9193eb032e98ebe9cae53e84 --- /dev/null +++ b/out_tensor/model.layers.15.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:510e0e235e85f6999c3728d7783322fd55ce6e1b597cfefe034d7eedc1452263 +size 8668456 diff --git a/out_tensor/model.layers.15.self_attn.q_proj.safetensors b/out_tensor/model.layers.15.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..6c38ee7f0f3e1b05b22ba92ea05e5a401811e361 --- /dev/null +++ b/out_tensor/model.layers.15.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fe5f3c4cd07ac9a984e5fddbb7add7dd5a5001c8e1ec734bf0531239ab9ebbf0 +size 7423272 diff --git a/out_tensor/model.layers.15.self_attn.v_proj.safetensors b/out_tensor/model.layers.15.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..10e94a58f1fe8a9694eb77257445a519bda01379 --- /dev/null +++ b/out_tensor/model.layers.15.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0cdfd3545a03b4dfd30df5a2b75154a587a61039cf7248bff4083abefe8431ea +size 2229536 diff --git a/out_tensor/model.layers.16.mlp.down_proj.safetensors b/out_tensor/model.layers.16.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..65ac2f4a0a74de463b4a684a645da672f0b73407 --- /dev/null +++ b/out_tensor/model.layers.16.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9a67b0cb128353c9700940d48a581ecee1becaf94b6cf09f906130de82294b92 +size 30338208 diff --git a/out_tensor/model.layers.16.mlp.gate_proj.safetensors b/out_tensor/model.layers.16.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..6d874214a28cc249fb1e03ae286f3d8a22e75f1a --- /dev/null +++ b/out_tensor/model.layers.16.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1bda8693cf230d958b8844ab5d097fe2060d1bdf9eff74ce21d9d0fc82086cac +size 29606616 diff --git a/out_tensor/model.layers.16.mlp.up_proj.safetensors b/out_tensor/model.layers.16.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..b78c494ae71c420c579729ac71aea2e57655fab9 --- /dev/null +++ b/out_tensor/model.layers.16.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f657383337be8141784b39f52f50271c3f7fdaf42a1439f00823112f7da9f6d8 +size 30295312 diff --git a/out_tensor/model.layers.16.self_attn.k_proj.safetensors b/out_tensor/model.layers.16.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..b552d624a055632d61eb8a3aa0d1da5e0983e05a --- /dev/null +++ b/out_tensor/model.layers.16.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:231233a87df9c093349c51496af9c9d9edd11ca1caa6e6a01b47bf9bf93cbfc1 +size 1656096 diff --git a/out_tensor/model.layers.16.self_attn.o_proj.safetensors b/out_tensor/model.layers.16.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..d83df8912aa880c02f4ef4c540917555a71ae039 --- /dev/null +++ b/out_tensor/model.layers.16.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d70286839595651ca9f892ad6cc0a34ba99d32ff51631042ff7c0ebb443f7db4 +size 8865064 diff --git a/out_tensor/model.layers.16.self_attn.q_proj.safetensors b/out_tensor/model.layers.16.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..1b7a9828475042c065a50ede27a0f69e8f35726b --- /dev/null +++ b/out_tensor/model.layers.16.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:102346426b379f13c4ed55df0f249b4f065d519be35a4ada9475e24fb6ea3fc9 +size 6767912 diff --git a/out_tensor/model.layers.16.self_attn.v_proj.safetensors b/out_tensor/model.layers.16.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..644b0f74cd9f0a7346293ef3c021a5287ea5e8ad --- /dev/null +++ b/out_tensor/model.layers.16.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fd2b452a07e38c97a5ef6ab7fa0d5f28104ffc361d689418912f67832761ebaf +size 2229536 diff --git a/out_tensor/model.layers.17.mlp.down_proj.safetensors b/out_tensor/model.layers.17.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..81b74c38b96127842e5d8b323ff67e1639d899fc --- /dev/null +++ b/out_tensor/model.layers.17.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:37368d22ccc1667fbe8b32f00b4dd9cd709b785e2fe867bde4f519a073f2e20b +size 31059104 diff --git a/out_tensor/model.layers.17.mlp.gate_proj.safetensors b/out_tensor/model.layers.17.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..4a8c3d1ce6a3382833ae579c145834b299b6f6f1 --- /dev/null +++ b/out_tensor/model.layers.17.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:13d846996b88b764f9bf6486c421b647b4e79b40b07e977811048e5c77eebd03 +size 29606616 diff --git a/out_tensor/model.layers.17.mlp.up_proj.safetensors b/out_tensor/model.layers.17.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..07b0d30e44a3211a818e26bd1f640fea82b10838 --- /dev/null +++ b/out_tensor/model.layers.17.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:466c114b8d92ac83ee74d81ab86d0e2457300d712a628a47c47e3b74a74c8d43 +size 30983440 diff --git a/out_tensor/model.layers.17.self_attn.k_proj.safetensors b/out_tensor/model.layers.17.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..400a7c2d4acb0d9380fad60cff277ce68df3ca52 --- /dev/null +++ b/out_tensor/model.layers.17.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:be74b9b67c532868a764c60c50448993fa273f4f6d380f4ccf36f6241e574a32 +size 1606368 diff --git a/out_tensor/model.layers.17.self_attn.o_proj.safetensors b/out_tensor/model.layers.17.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..fbb688bba4dbeae1566d14bd60d62b2eb9286350 --- /dev/null +++ b/out_tensor/model.layers.17.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2f301dfbae11990a4491b8517b72485183a0f4271f3abe718ce9ba6f69ddc208 +size 8668456 diff --git a/out_tensor/model.layers.17.self_attn.q_proj.safetensors b/out_tensor/model.layers.17.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..ca4fc03910ae0fdd58b92559a21d80b72f6e7804 --- /dev/null +++ b/out_tensor/model.layers.17.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6c780a66765e80d6b0f42f49b710031dbd41ba602d4f088d6e56a99fa11d2d81 +size 6571304 diff --git a/out_tensor/model.layers.17.self_attn.v_proj.safetensors b/out_tensor/model.layers.17.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..ad9cae2601177d3c479b0804fee830cc69222afc --- /dev/null +++ b/out_tensor/model.layers.17.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d1d4df1f50f110de88d312ba0daa4f3c68a069bc8b85e2a79abc104044a86d77 +size 2180384 diff --git a/out_tensor/model.layers.18.mlp.down_proj.safetensors b/out_tensor/model.layers.18.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..595f063d23df65b77e0dfbf45ad7ccf254ba6023 --- /dev/null +++ b/out_tensor/model.layers.18.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:13ff96611185904fea79a988d2c112d2f235c4ddea239e0750718fcf2a9b44a6 +size 30338208 diff --git a/out_tensor/model.layers.18.mlp.gate_proj.safetensors b/out_tensor/model.layers.18.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..ad1415372aabbeae8f90118c4797553cd35c19a8 --- /dev/null +++ b/out_tensor/model.layers.18.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:56720c5efc528d65c10b3f86b46b9285674956ae15d7527e3a2d1e1b796a801b +size 29606616 diff --git a/out_tensor/model.layers.18.mlp.up_proj.safetensors b/out_tensor/model.layers.18.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..96f6ab5711e4b9f689d7daef770ac93d9a3cce69 --- /dev/null +++ b/out_tensor/model.layers.18.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:48526bf72a216a1a0f460b75925bda10242cf7f4ba743ddf8ece4e882aa3edf4 +size 30295312 diff --git a/out_tensor/model.layers.18.self_attn.k_proj.safetensors b/out_tensor/model.layers.18.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..ab34b819f99f087c90a07ad2e935b40d7812422e --- /dev/null +++ b/out_tensor/model.layers.18.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:be86cb1d2377c39600e36d5dc331fc16fa2e7e4123cb04d50be758a5856c7f95 +size 1656096 diff --git a/out_tensor/model.layers.18.self_attn.o_proj.safetensors b/out_tensor/model.layers.18.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..2beeb6340ae7907d7cc2ab98b126246c4f20187d --- /dev/null +++ b/out_tensor/model.layers.18.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:321288e0db1ff56d7c5e1ff44d1aec4b82524cd3bb417ce05677d9c98024db1e +size 8668456 diff --git a/out_tensor/model.layers.18.self_attn.q_proj.safetensors b/out_tensor/model.layers.18.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..88a43bc91f64717a814b4e0ab5910b6dfc87ac4d --- /dev/null +++ b/out_tensor/model.layers.18.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b58f5ce3ff97f39576cd8ab5faff0bfdf84664c26b0310cee71adcc65792a868 +size 6702376 diff --git a/out_tensor/model.layers.18.self_attn.v_proj.safetensors b/out_tensor/model.layers.18.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..340c084846a0808e318f476bbde4fe8715ac1a89 --- /dev/null +++ b/out_tensor/model.layers.18.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3b421e8a93c54801089089b79bc700c470e90df05ad37c7aa2542c14beeae9c2 +size 2180384 diff --git a/out_tensor/model.layers.19.mlp.down_proj.safetensors b/out_tensor/model.layers.19.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..fee947d100f08e9e007c60692f40171962bb1fcd --- /dev/null +++ b/out_tensor/model.layers.19.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0004152f2458bff6ad2f8d526dc8d4c6e08b9af03cbc7406e5a36bc2bea74a6d +size 30338208 diff --git a/out_tensor/model.layers.19.mlp.gate_proj.safetensors b/out_tensor/model.layers.19.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..456abae44030011270ac88738c63786f36380b96 --- /dev/null +++ b/out_tensor/model.layers.19.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:32208859150505a661611ffabb383ef8eb52e9b61cbf0712028371912dc5b84f +size 29606616 diff --git a/out_tensor/model.layers.19.mlp.up_proj.safetensors b/out_tensor/model.layers.19.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..4d263b529d986b27976513075ffcb27dd5ac4aa6 --- /dev/null +++ b/out_tensor/model.layers.19.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:76a1cb45f56d1dce6eec050556bd0e31aef9be710c92357241c169649181c037 +size 30295312 diff --git a/out_tensor/model.layers.19.self_attn.k_proj.safetensors b/out_tensor/model.layers.19.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..27264f907d7e6d974d7c5007690fbab78bf69424 --- /dev/null +++ b/out_tensor/model.layers.19.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:48c9ba9aa17be2019b3be7350a8ce2bce424a7e73841fa7aa08fd25d3496320f +size 1656096 diff --git a/out_tensor/model.layers.19.self_attn.o_proj.safetensors b/out_tensor/model.layers.19.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..8a5106e37cdfdecadb2861a06182fd6cc954bf41 --- /dev/null +++ b/out_tensor/model.layers.19.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ebe5e7cd34f6532dc921c742044a54d62a407ef0936c60af5ccc70ad38a7ec33 +size 8668456 diff --git a/out_tensor/model.layers.19.self_attn.q_proj.safetensors b/out_tensor/model.layers.19.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..3cf3465bd45d41aaa6aa3c16bd019bb3d1a5e9a9 --- /dev/null +++ b/out_tensor/model.layers.19.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:aedc2dcaed6c60c30a3152af03483b90d4747713475d975822674f0b2c94b683 +size 6767912 diff --git a/out_tensor/model.layers.19.self_attn.v_proj.safetensors b/out_tensor/model.layers.19.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..2e509b4cb726732ec675cce62f1b6c7b3b81dd83 --- /dev/null +++ b/out_tensor/model.layers.19.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3d8d693b7427e5916f439a548b341731aca02d7fa3592158c3c3d46797cedc96 +size 2229536 diff --git a/out_tensor/model.layers.2.mlp.down_proj.safetensors b/out_tensor/model.layers.2.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..65dc42ecf2a5abc74fb37f70cc97da668f480c68 --- /dev/null +++ b/out_tensor/model.layers.2.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0fdf2ce280cf77d6ba2765d7f83e538f1307ca56e6106c4e0897ef69198235fb +size 29648056 diff --git a/out_tensor/model.layers.2.mlp.gate_proj.safetensors b/out_tensor/model.layers.2.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..fd98c04325fc59efd54018bf971e8a3c61c51a33 --- /dev/null +++ b/out_tensor/model.layers.2.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7ee1886398744769f935856e9bf15e809cffa6abadad9efb27f6ad50f94c867c +size 29606616 diff --git a/out_tensor/model.layers.2.mlp.up_proj.safetensors b/out_tensor/model.layers.2.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..ddf6cb3c4cf5385c4c2360a9ec0f73d7b3c0e9b6 --- /dev/null +++ b/out_tensor/model.layers.2.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:197532904fa120ef3ed5516a2cc771520a48f9868baedb7272468a2519fadada +size 30295312 diff --git a/out_tensor/model.layers.2.self_attn.k_proj.safetensors b/out_tensor/model.layers.2.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..67d2bf9a1b064912f31a9dd1dbb69c38e993d376 --- /dev/null +++ b/out_tensor/model.layers.2.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:15459ea712103d4e9a13a17055fe494505d5906bbeacb09e1ff60ed360e4ba9e +size 1262880 diff --git a/out_tensor/model.layers.2.self_attn.o_proj.safetensors b/out_tensor/model.layers.2.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..a9ec067b0bb435a4ce5db2d56967e48b0d9b0b64 --- /dev/null +++ b/out_tensor/model.layers.2.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e0dec8e3aa46d27799fca34c0d55171bb6b02949644565ffdc0f76f388326f40 +size 8471256 diff --git a/out_tensor/model.layers.2.self_attn.q_proj.safetensors b/out_tensor/model.layers.2.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..7b70794a76579dd08432eb338a17fd78f6b2e0a0 --- /dev/null +++ b/out_tensor/model.layers.2.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:94e747f9240d1d873cea73e1e51ebb364731cf4f38e856b2400282a6d9c249e2 +size 4998432 diff --git a/out_tensor/model.layers.2.self_attn.v_proj.safetensors b/out_tensor/model.layers.2.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..ca2277c393588312c65dcdcc72117d0992304a51 --- /dev/null +++ b/out_tensor/model.layers.2.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b435b94e0eb00dadf58b99a74fdbbadc02128e720abd153419e1f691f037a08f +size 2130648 diff --git a/out_tensor/model.layers.20.mlp.down_proj.safetensors b/out_tensor/model.layers.20.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..d0a15864d3c2cc129c726bcf7ede3236615ee47f --- /dev/null +++ b/out_tensor/model.layers.20.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:aeaf60ddd06959e11973e0988cf6e20241ef09d0e89036bb8a7ad933069d41ef +size 30338208 diff --git a/out_tensor/model.layers.20.mlp.gate_proj.safetensors b/out_tensor/model.layers.20.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..65a3f6b63fe7d38138e4e6371771995b566983b4 --- /dev/null +++ b/out_tensor/model.layers.20.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:de38c262fedc355bdc52582e0d9086010c5c2e9c85847690b6753e4e77e55757 +size 29606616 diff --git a/out_tensor/model.layers.20.mlp.up_proj.safetensors b/out_tensor/model.layers.20.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..a48f58cdc76b4f508249538aee9d790a74ece0b3 --- /dev/null +++ b/out_tensor/model.layers.20.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f385300653581f95e3f0f3b9d32cb683c2078939b8f8ec28660e9473f0528d46 +size 30295312 diff --git a/out_tensor/model.layers.20.self_attn.k_proj.safetensors b/out_tensor/model.layers.20.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..4823efbb734e3af837500c64795bf40327b2712a --- /dev/null +++ b/out_tensor/model.layers.20.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:93b1ea6e503003e9eddb8fc26d715b7c47ce8c99c0cb67ba51d6fb687c9bddb9 +size 1656096 diff --git a/out_tensor/model.layers.20.self_attn.o_proj.safetensors b/out_tensor/model.layers.20.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..ef9516c9f1aa2b342528689c7f7a9b63f8586ae8 --- /dev/null +++ b/out_tensor/model.layers.20.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eabab48054b109ef4707c9f62b30d30d5c5786a2e25386c0c285a6bf8c6630f6 +size 8865064 diff --git a/out_tensor/model.layers.20.self_attn.q_proj.safetensors b/out_tensor/model.layers.20.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..435cf25bbca63223e8402e3f4ea72742e3487a48 --- /dev/null +++ b/out_tensor/model.layers.20.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c736eb87eec80236bd2865bfd17c42d1b8069c4777c1fbbdf682544fcf94d3ac +size 6767912 diff --git a/out_tensor/model.layers.20.self_attn.v_proj.safetensors b/out_tensor/model.layers.20.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..65f1c29ef722343211352eb16727cfa26f88efdb --- /dev/null +++ b/out_tensor/model.layers.20.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b0c72ed0e541ec5c904993af335394ec3d6baea5f093f6247bc0cb3637a0cf68 +size 2229536 diff --git a/out_tensor/model.layers.21.mlp.down_proj.safetensors b/out_tensor/model.layers.21.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..b04d59cd27c32511ab598428fb1f6267d3e43d8e --- /dev/null +++ b/out_tensor/model.layers.21.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6d899b5deb5c0f517e389fa017c4542d802d909d2a097b45173e1455f4d09166 +size 30338208 diff --git a/out_tensor/model.layers.21.mlp.gate_proj.safetensors b/out_tensor/model.layers.21.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..f0dd86bc7ae3c8dfcd2b959bd36704cf5ab8f818 --- /dev/null +++ b/out_tensor/model.layers.21.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cfa3df7d7c9303586081b2dac1e711631ecb33b36ee76b0a6e9f24d05a6fde8d +size 29606616 diff --git a/out_tensor/model.layers.21.mlp.up_proj.safetensors b/out_tensor/model.layers.21.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..425aab01e795e5dd48823e4fb152dca501bb03ee --- /dev/null +++ b/out_tensor/model.layers.21.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6a2decb10da26edb0910525eb3a0bd7370d0a5222b74e68b3f61a9dddb0c4dd7 +size 30983440 diff --git a/out_tensor/model.layers.21.self_attn.k_proj.safetensors b/out_tensor/model.layers.21.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..01acc57137c1a3fe92e3963f886acb300828ace0 --- /dev/null +++ b/out_tensor/model.layers.21.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:70e07d2faf6eac06c77fe8b926bdbc32e2defa45d3b4754aa393a98620540dce +size 1656096 diff --git a/out_tensor/model.layers.21.self_attn.o_proj.safetensors b/out_tensor/model.layers.21.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..76da42762269205182b5932c5e912f6650074b60 --- /dev/null +++ b/out_tensor/model.layers.21.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:97e0d1c22506ee6121b87b6be22e58549610cc6e4c970852dd15e8f1a91f836c +size 8865064 diff --git a/out_tensor/model.layers.21.self_attn.q_proj.safetensors b/out_tensor/model.layers.21.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..8acfe21415ebfb906402814a6026a2f658bf3314 --- /dev/null +++ b/out_tensor/model.layers.21.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e513d8dadce3014555417cc951468c4f7acf69ce7d67448dee0e4efb8f8b3b9f +size 6702376 diff --git a/out_tensor/model.layers.21.self_attn.v_proj.safetensors b/out_tensor/model.layers.21.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..4646e455e67fcccdc10f345791f7fd5384da59ee --- /dev/null +++ b/out_tensor/model.layers.21.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fd5ac896c199c07246bccd854c694a12876978878c5679beffe5308eb197ec4f +size 2180384 diff --git a/out_tensor/model.layers.22.mlp.down_proj.safetensors b/out_tensor/model.layers.22.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..a3ae58ca6e87ffe99fd466dbe0cfd639dfb2f7a5 --- /dev/null +++ b/out_tensor/model.layers.22.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a2c2d22608fada0e7e7885929d4f391adc9c45e9fdb92201873758171d96e978 +size 30338208 diff --git a/out_tensor/model.layers.22.mlp.gate_proj.safetensors b/out_tensor/model.layers.22.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..b72efe2cc6a298a330e30a5634201f733f5daabf --- /dev/null +++ b/out_tensor/model.layers.22.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:93dd23c808b5ee745fe59eed84900e3915335b28a5e1a4b257d62ac6e6fa5bf6 +size 29606616 diff --git a/out_tensor/model.layers.22.mlp.up_proj.safetensors b/out_tensor/model.layers.22.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..86b3d9de0b0f00ded899aca1a5b3b4ef68a4d5e0 --- /dev/null +++ b/out_tensor/model.layers.22.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b2a363130d72402fa118d252f0f765767769088182ed6898bcf9a3a9bda7527c +size 31671568 diff --git a/out_tensor/model.layers.22.self_attn.k_proj.safetensors b/out_tensor/model.layers.22.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..f281bc88179a89277f1408889e38055669bda2f5 --- /dev/null +++ b/out_tensor/model.layers.22.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:678be0209d18463af530e423ce18478668ef1322bee977be93cd78064ca4f6a3 +size 1688864 diff --git a/out_tensor/model.layers.22.self_attn.o_proj.safetensors b/out_tensor/model.layers.22.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..6856e4ef379d80e9e2745bb5fac3723ae5ad071a --- /dev/null +++ b/out_tensor/model.layers.22.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8d393ff5e88dc08b6e488d0aefaec26b41293d1b61b7603e7fac6bf734599919 +size 8668456 diff --git a/out_tensor/model.layers.22.self_attn.q_proj.safetensors b/out_tensor/model.layers.22.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..96b6d732d72dd29b11d880a37b51073da4d6c17f --- /dev/null +++ b/out_tensor/model.layers.22.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d35c99b3a37b7910db7d444516a3ee01af7caf6a74023217e2a2f5d6ec5aa845 +size 6702376 diff --git a/out_tensor/model.layers.22.self_attn.v_proj.safetensors b/out_tensor/model.layers.22.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..b9459b8d03215e65c947984c38b88f21cabbd0cf --- /dev/null +++ b/out_tensor/model.layers.22.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:042e4cc1ef2176723b22197e4c77f2f69fa74b770c05ffeea674e89f2763009f +size 2180384 diff --git a/out_tensor/model.layers.23.mlp.down_proj.safetensors b/out_tensor/model.layers.23.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..de893a159aca5921f0a9e749528af12af9511550 --- /dev/null +++ b/out_tensor/model.layers.23.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9657ceb917ec57ba742734faf781b1ee386171c53e33822a71dd7b9d9bd2de14 +size 30338208 diff --git a/out_tensor/model.layers.23.mlp.gate_proj.safetensors b/out_tensor/model.layers.23.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..f6ebb9a55c3cfd3b254a2da2176d8b24fec473c7 --- /dev/null +++ b/out_tensor/model.layers.23.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:793b97b41b6896a7425ef2048921d003ed282d3ed87781d90da4cb0bc74c344d +size 29606616 diff --git a/out_tensor/model.layers.23.mlp.up_proj.safetensors b/out_tensor/model.layers.23.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..3f4357b6af4ed50fa64c5a2af6fe383704dfc56f --- /dev/null +++ b/out_tensor/model.layers.23.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ea1e99832e39323617bd29401f50b75bd9c999516ccc0b017e5c955f67e58562 +size 36946640 diff --git a/out_tensor/model.layers.23.self_attn.k_proj.safetensors b/out_tensor/model.layers.23.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..9340a9bdcbe0aa4f9d5ee86344b0f6398e177ed4 --- /dev/null +++ b/out_tensor/model.layers.23.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0cbf85186d9bffc3c48facc3696cc2dabcd098cf0d5b9a845c1f8387bca82451 +size 1688864 diff --git a/out_tensor/model.layers.23.self_attn.o_proj.safetensors b/out_tensor/model.layers.23.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..8f24717d51c2d670586dad737a9441191b7c6048 --- /dev/null +++ b/out_tensor/model.layers.23.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c74cf20b63f9735d1b48a9c651068ef0e0d4fc7bd195254579baa78f6d68af5d +size 8668456 diff --git a/out_tensor/model.layers.23.self_attn.q_proj.safetensors b/out_tensor/model.layers.23.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..e48f9c2d4af3a8920e506d5a8929ad0ff52986f3 --- /dev/null +++ b/out_tensor/model.layers.23.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a233a84e2c6eb9a659254f586563695076402fd00c71af061c26dc58eb9c8bad +size 6702376 diff --git a/out_tensor/model.layers.23.self_attn.v_proj.safetensors b/out_tensor/model.layers.23.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..b013222967b6941d607011a812108c9c566ce27c --- /dev/null +++ b/out_tensor/model.layers.23.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:27beca4de04e01a27ab395107f7b16b6020ded5cf7d55da81f0194078e2a3f8a +size 2229536 diff --git a/out_tensor/model.layers.24.mlp.down_proj.safetensors b/out_tensor/model.layers.24.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..cfbf84414d11e0255c0ab3d79ae3424cb1558dea --- /dev/null +++ b/out_tensor/model.layers.24.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ce17129c67fdc66507d8d1b1d90e07fcf45e0d95df24eec49e15055a08c97039 +size 30338208 diff --git a/out_tensor/model.layers.24.mlp.gate_proj.safetensors b/out_tensor/model.layers.24.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..673554deb2415f95eada4b057dbb271be43214a4 --- /dev/null +++ b/out_tensor/model.layers.24.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:65875557b83b524a301774660c036c258971aeef3119e9ca8ce9f5eeaf3ef93f +size 29606616 diff --git a/out_tensor/model.layers.24.mlp.up_proj.safetensors b/out_tensor/model.layers.24.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..d12d8e2a2614cb0f4e332d7496e8c26d51e3661b --- /dev/null +++ b/out_tensor/model.layers.24.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:94540a715277d8a4fd62204e24e2eb9aea16ec2d9ec75e06a2235efeddef16c9 +size 36946640 diff --git a/out_tensor/model.layers.24.self_attn.k_proj.safetensors b/out_tensor/model.layers.24.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..17525b378f035bd1cb40d54db504183d053340f0 --- /dev/null +++ b/out_tensor/model.layers.24.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:44b3a6d061c13dd264687372ebe75052cfd52bc5b635c6070a4c8c34836579b0 +size 1688864 diff --git a/out_tensor/model.layers.24.self_attn.o_proj.safetensors b/out_tensor/model.layers.24.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..f2fb7c03942810fae2d380be03a1b16f597eadcc --- /dev/null +++ b/out_tensor/model.layers.24.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dc56366cf0bcf3ec628fc931564c8c831b442fddf88ec91dbfccb0b8cdaa1ec9 +size 8471264 diff --git a/out_tensor/model.layers.24.self_attn.q_proj.safetensors b/out_tensor/model.layers.24.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..c8edb5902c1d8eb596f6f0845e4a21a350df220b --- /dev/null +++ b/out_tensor/model.layers.24.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6f3a09071d9d11a4a54f60c45255c448a38a2e429d2a24ba2c7f77d8636516f6 +size 7423272 diff --git a/out_tensor/model.layers.24.self_attn.v_proj.safetensors b/out_tensor/model.layers.24.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..7622a14213fd81277021ef7d926d70d7e75b9062 --- /dev/null +++ b/out_tensor/model.layers.24.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c382d2130ff7727e5729accdacf4ab5ee4401e32a502886b5e2559bb0a66d105 +size 2229536 diff --git a/out_tensor/model.layers.25.mlp.down_proj.safetensors b/out_tensor/model.layers.25.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..2aea6c4f42ade6e02e34ddc8b2a346700148628c --- /dev/null +++ b/out_tensor/model.layers.25.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8f4612abd73df586ca5da73b927e75f7a8e37f2a33b93bdd5f791ed6c4385352 +size 30338208 diff --git a/out_tensor/model.layers.25.mlp.gate_proj.safetensors b/out_tensor/model.layers.25.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..66554ed36607abd94b62f3fb766ac8fbe2782b9c --- /dev/null +++ b/out_tensor/model.layers.25.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:39873d67b8e323405ef48ee588ec4b744cb0e2195576ca882553b61fe98bd84b +size 29606616 diff --git a/out_tensor/model.layers.25.mlp.up_proj.safetensors b/out_tensor/model.layers.25.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..330ae21faa7346ddaaa8865681fc9de8b63e77af --- /dev/null +++ b/out_tensor/model.layers.25.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:16e8f617cb1455a6d1bd6a604d9e93db886adc46c1925bfee50a63952f13e392 +size 36946640 diff --git a/out_tensor/model.layers.25.self_attn.k_proj.safetensors b/out_tensor/model.layers.25.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..bb2bc12e9437d1a7e6e09be56c100ec0aea2e1da --- /dev/null +++ b/out_tensor/model.layers.25.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6f9ba413471167f8dbe8f4fe754fcae6de5c02a2131a148ccc4cf3ab80971faf +size 1688864 diff --git a/out_tensor/model.layers.25.self_attn.o_proj.safetensors b/out_tensor/model.layers.25.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..0b77dc4c3d615bf89089ee0c9b025d4068dad99b --- /dev/null +++ b/out_tensor/model.layers.25.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1f4ea38343cb078649964b0c4739536dcbd8fa2839aceb6d716dc1874980e4e8 +size 8471264 diff --git a/out_tensor/model.layers.25.self_attn.q_proj.safetensors b/out_tensor/model.layers.25.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..ed21ccdec85f51d82d1016ce93d0c84f6c9febfd --- /dev/null +++ b/out_tensor/model.layers.25.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d95bfc282f169e154f525bf7287cd051990db06c27fdccbca68ec1ac8b09fe13 +size 7423272 diff --git a/out_tensor/model.layers.25.self_attn.v_proj.safetensors b/out_tensor/model.layers.25.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..b067ea81d505d9af515e477d306ce5ad73a42169 --- /dev/null +++ b/out_tensor/model.layers.25.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:753a732291e03ac446f29203cbf42b9fa264f83b0e26105076c401f7d48a1c64 +size 2229536 diff --git a/out_tensor/model.layers.26.mlp.down_proj.safetensors b/out_tensor/model.layers.26.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..86ba541f4608165466c569d7700b27d5fc022757 --- /dev/null +++ b/out_tensor/model.layers.26.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:61a7361213a7fa496d47e0f3872fe5fd811dfe0f1125e0678044ddf857fa32c2 +size 31059104 diff --git a/out_tensor/model.layers.26.mlp.gate_proj.safetensors b/out_tensor/model.layers.26.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..8a6ac5265ca5c65ab06eb55723b4ba63fac9375f --- /dev/null +++ b/out_tensor/model.layers.26.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6f32fcbdec5d17c22305fa7dd08f00cf88c951502bee9800a74a3fad025ccf6c +size 29606616 diff --git a/out_tensor/model.layers.26.mlp.up_proj.safetensors b/out_tensor/model.layers.26.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..456cbbd477f96a78afb6da719b288a4748128ae8 --- /dev/null +++ b/out_tensor/model.layers.26.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0de5da4d715f9f974c97eaf492172c9265a8ced2b46bec5343d35b224be4c9ba +size 36946640 diff --git a/out_tensor/model.layers.26.self_attn.k_proj.safetensors b/out_tensor/model.layers.26.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..e75f9609dd300b65bbb0ef1d83174407207d7679 --- /dev/null +++ b/out_tensor/model.layers.26.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7cea309d857a1462135684dbd4f3323e703055a7aca0e553cf2a634779111e72 +size 1688864 diff --git a/out_tensor/model.layers.26.self_attn.o_proj.safetensors b/out_tensor/model.layers.26.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..8e15c0ac66130feb665d4ec8314c34f86c8e1663 --- /dev/null +++ b/out_tensor/model.layers.26.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e3b01188731eca89c96f75a2bd2e931a34cf3d2c41a48a9255edd78b1521a82d +size 8471264 diff --git a/out_tensor/model.layers.26.self_attn.q_proj.safetensors b/out_tensor/model.layers.26.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..0df5ccc7f0fbff27948005f056968ffc805ab91e --- /dev/null +++ b/out_tensor/model.layers.26.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5c2b1683af15fb4f5839bcea76475de299cffaf039edd823cbaf7b13e55ffb64 +size 7423272 diff --git a/out_tensor/model.layers.26.self_attn.v_proj.safetensors b/out_tensor/model.layers.26.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..8efd344abcb1f1dede67ec1862a4717a482cd563 --- /dev/null +++ b/out_tensor/model.layers.26.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eb707cbf5d1389bb4e0e65086baf0877ea720ac009cbe7585ada280412b69ad8 +size 2229536 diff --git a/out_tensor/model.layers.27.mlp.down_proj.safetensors b/out_tensor/model.layers.27.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..41826330bac4e41566eca75292c32bda79900507 --- /dev/null +++ b/out_tensor/model.layers.27.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cb7511c1cd99a763c3666ae268e86f9f4bd9ef13dc172c333d06ad1249366a19 +size 31059104 diff --git a/out_tensor/model.layers.27.mlp.gate_proj.safetensors b/out_tensor/model.layers.27.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..efac366eabade26390070d47ff2e9ee938805e6e --- /dev/null +++ b/out_tensor/model.layers.27.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:939d2bbd67adfa8de2ffbff1a378e24665b711c543d00715a465401b46614268 +size 29606616 diff --git a/out_tensor/model.layers.27.mlp.up_proj.safetensors b/out_tensor/model.layers.27.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..0b55ddba560e91589b1073b8def2ab61fb0e5603 --- /dev/null +++ b/out_tensor/model.layers.27.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c9342f7ab79d0bbcf39d9122a2b860f5100fc955bf88691613876b1029ff9f47 +size 36946640 diff --git a/out_tensor/model.layers.27.self_attn.k_proj.safetensors b/out_tensor/model.layers.27.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..ebe85979b460faf956fba24748a2d3d1346b2b4a --- /dev/null +++ b/out_tensor/model.layers.27.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2656dc10adbf40923887d8ab762f73cef43de59a41aa096ee1d3d72d1be459a9 +size 1688864 diff --git a/out_tensor/model.layers.27.self_attn.o_proj.safetensors b/out_tensor/model.layers.27.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..d849ef9cb01fd26c015b1127f46000079761285a --- /dev/null +++ b/out_tensor/model.layers.27.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2a838d86d800ddef3d49df8ce9e9dcf1bb93f584e5c6179cd7da3d37bd48607e +size 8668456 diff --git a/out_tensor/model.layers.27.self_attn.q_proj.safetensors b/out_tensor/model.layers.27.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..61c586ccd2f429c0586a9ed9353fd5c4a699f058 --- /dev/null +++ b/out_tensor/model.layers.27.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:52e7abf009e6442f066355a901fe5ca97f35abeec7c1263ebc16d1ba3c8c1433 +size 7423272 diff --git a/out_tensor/model.layers.27.self_attn.v_proj.safetensors b/out_tensor/model.layers.27.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..73ffa1741e10a3e865c0a914e1ffea03e209cb84 --- /dev/null +++ b/out_tensor/model.layers.27.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:20c6322ff1f8f602bb9810c46f8878c6651ef5a00c7d2bb72cb06a5bd70ab612 +size 2229536 diff --git a/out_tensor/model.layers.28.mlp.down_proj.safetensors b/out_tensor/model.layers.28.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..4f82000eb34cf949436f7a84d573d6d5ec6af029 --- /dev/null +++ b/out_tensor/model.layers.28.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9fdc2b14c35e53fdbc7b207ec911bbf9819bc01594361bf344764fd8cbff87d8 +size 30338208 diff --git a/out_tensor/model.layers.28.mlp.gate_proj.safetensors b/out_tensor/model.layers.28.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..b01914f21b8a1b4d7105880561906fbef268b9b6 --- /dev/null +++ b/out_tensor/model.layers.28.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7cb2fa4094e6997aa100f30b0e283c096fb013974ed088e588df6d72f8eab484 +size 29606616 diff --git a/out_tensor/model.layers.28.mlp.up_proj.safetensors b/out_tensor/model.layers.28.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..8e5f50075250f196652ce2201951af3d06601b59 --- /dev/null +++ b/out_tensor/model.layers.28.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:72e95f41cfdf25c5d5a4d47e1ca436461ce08f47370689c4c68283d0f6cacedf +size 30295312 diff --git a/out_tensor/model.layers.28.self_attn.k_proj.safetensors b/out_tensor/model.layers.28.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..512231cc88764c6214f2192feb826d36f9f7ff58 --- /dev/null +++ b/out_tensor/model.layers.28.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c496fae1c03b68d71f69d864d596b2b743c76faa19fdb4a75cd8f1931ee34f5c +size 1688864 diff --git a/out_tensor/model.layers.28.self_attn.o_proj.safetensors b/out_tensor/model.layers.28.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..4d93552ab86d9792c3805c5290b41e9f29c18d27 --- /dev/null +++ b/out_tensor/model.layers.28.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:89257e8f43cafa8defcd53c009d5667385830e48c5b4d761bb375cbc71fe5f55 +size 8668456 diff --git a/out_tensor/model.layers.28.self_attn.q_proj.safetensors b/out_tensor/model.layers.28.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..655355a0603f5ee097472337ca3361c124e4d3cf --- /dev/null +++ b/out_tensor/model.layers.28.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bc70349acca60f9f24c4892334f9b8ec3da007eb4f1a5c4ee163dce2b6fccd93 +size 6767912 diff --git a/out_tensor/model.layers.28.self_attn.v_proj.safetensors b/out_tensor/model.layers.28.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..93107085e82087dadc22a1f90012108967bd1220 --- /dev/null +++ b/out_tensor/model.layers.28.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:de85eb75da3c59cc141252f512f3c2df09f645e28db0170b081270e1c1afd944 +size 2229536 diff --git a/out_tensor/model.layers.29.mlp.down_proj.safetensors b/out_tensor/model.layers.29.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..5071393790f402895c79ce55405e812358611169 --- /dev/null +++ b/out_tensor/model.layers.29.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0645f112657765c4e8409a50c9664387f4355e7527c3e9ea29c1a16518b2d493 +size 30338208 diff --git a/out_tensor/model.layers.29.mlp.gate_proj.safetensors b/out_tensor/model.layers.29.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..b42621e70d50850c0b69569b1371f31fc7f2d0ba --- /dev/null +++ b/out_tensor/model.layers.29.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2719700fb77c67d6675e6a26d2e81da59b27746a4671a360aaac68a710b108dc +size 29606616 diff --git a/out_tensor/model.layers.29.mlp.up_proj.safetensors b/out_tensor/model.layers.29.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..b39c8f49f792538614bde7751f7888974290a606 --- /dev/null +++ b/out_tensor/model.layers.29.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:910ebed2d57e1dff6feb45a1d9c30df8dc0421f5409b558494f7285dd52c4c10 +size 30295312 diff --git a/out_tensor/model.layers.29.self_attn.k_proj.safetensors b/out_tensor/model.layers.29.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..17e7ddbe8dc11a142434a9909a1674313031ce10 --- /dev/null +++ b/out_tensor/model.layers.29.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0df8de40def5b40830d160c05d944aaa6b593ee4d873d78a2f96b4a2251c1e9a +size 1688864 diff --git a/out_tensor/model.layers.29.self_attn.o_proj.safetensors b/out_tensor/model.layers.29.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..61061aa00356785514703301702226a3ed37c550 --- /dev/null +++ b/out_tensor/model.layers.29.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ae954566ab1b9a8cbecf8ca1fcea4e56c78ad31a8b202b22dfc5be47a9b53657 +size 8471264 diff --git a/out_tensor/model.layers.29.self_attn.q_proj.safetensors b/out_tensor/model.layers.29.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..0b92e58d8afb2d2071336a1f9a1c24e4e861b78e --- /dev/null +++ b/out_tensor/model.layers.29.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:202d7ea4119364f2fc8ab97225766c9da895bd4a129d573d49f8e565656a7def +size 6702376 diff --git a/out_tensor/model.layers.29.self_attn.v_proj.safetensors b/out_tensor/model.layers.29.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..36b6aa628fe69be27b36aa964c0e7054310c08a3 --- /dev/null +++ b/out_tensor/model.layers.29.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e5d8ebd2207f7155ed78e3a5c41a440918bdd1c89eaba2eaa40b113514720b7d +size 2654944 diff --git a/out_tensor/model.layers.3.mlp.down_proj.safetensors b/out_tensor/model.layers.3.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..b19ae56d5df6fa6910491d38a3f4ae5257a7a82a --- /dev/null +++ b/out_tensor/model.layers.3.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:348057040e9f4096c4ff6a4dfed7aaaced13ae1af89b70e09d71c649e91a5995 +size 30338200 diff --git a/out_tensor/model.layers.3.mlp.gate_proj.safetensors b/out_tensor/model.layers.3.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..453d0f4ab73895471967a91459be43922f37e91a --- /dev/null +++ b/out_tensor/model.layers.3.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eecc45c34f785ff19ef44f01731ff7bc0d827f680a78c6b69772831ea5353c30 +size 30295320 diff --git a/out_tensor/model.layers.3.mlp.up_proj.safetensors b/out_tensor/model.layers.3.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..98be13539b73b409e7a1943ae6a46d9f6123bf28 --- /dev/null +++ b/out_tensor/model.layers.3.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:51b150e4dacea0c06eb02fb27cd9de5f42be232f9ca84b0fc0bf26f044ec0e4e +size 36946632 diff --git a/out_tensor/model.layers.3.self_attn.k_proj.safetensors b/out_tensor/model.layers.3.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..187f281ab1a039a42e9c8ddb950198a0c8023f20 --- /dev/null +++ b/out_tensor/model.layers.3.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0515622328ea0712ce879867bfe34389120c2e9749459a875af99cf685083280 +size 1164576 diff --git a/out_tensor/model.layers.3.self_attn.o_proj.safetensors b/out_tensor/model.layers.3.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..d38f1834795a5fa43f66c3f25ccf546bab99b9e3 --- /dev/null +++ b/out_tensor/model.layers.3.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e40739d378d8bf914ae8daf761c526aa252104e3dc0d95c818eda69a519fa456 +size 8471256 diff --git a/out_tensor/model.layers.3.self_attn.q_proj.safetensors b/out_tensor/model.layers.3.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..b25bb2af22f1087757e75b3db9fcf5bb0cf10fec --- /dev/null +++ b/out_tensor/model.layers.3.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d78c8f5b91a7b8a40b41d21a89cb55da9d3b7fa37c367325f7b1d896dda40fde +size 4605216 diff --git a/out_tensor/model.layers.3.self_attn.v_proj.safetensors b/out_tensor/model.layers.3.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..20b8b6036bb37b39ce8eef3e0ae80e28d8849383 --- /dev/null +++ b/out_tensor/model.layers.3.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:32bff13a1815668a41f8dd1922f6fed86f9e7f6b7d91c792a51744b63cded099 +size 2130648 diff --git a/out_tensor/model.layers.30.mlp.down_proj.safetensors b/out_tensor/model.layers.30.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..9b2cfc07336d167535ea85f06f1e6ed7e1c736d3 --- /dev/null +++ b/out_tensor/model.layers.30.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c115270418a61cd18d788286482e81439b4851efa54f50a74f5749e0dc6ca7bc +size 29648064 diff --git a/out_tensor/model.layers.30.mlp.gate_proj.safetensors b/out_tensor/model.layers.30.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..eb1109aeb9e9f3d0d0ee7cb2b72393fda5dbe838 --- /dev/null +++ b/out_tensor/model.layers.30.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9be003e448396e3d292c384f547c491712a7bd550532317c65ffc4094b00d91b +size 29606616 diff --git a/out_tensor/model.layers.30.mlp.up_proj.safetensors b/out_tensor/model.layers.30.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..7c4eb1316cfb7f0ef489da5ba1f0f5b250cbaf67 --- /dev/null +++ b/out_tensor/model.layers.30.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:40d2acb8a68f69e7b3fd5a2a67e242d8caff20bd8c871b6137a5b12b2f21c0b5 +size 29606608 diff --git a/out_tensor/model.layers.30.self_attn.k_proj.safetensors b/out_tensor/model.layers.30.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..101bc87416db802bd6ad0c333ed811d0e7bc03d2 --- /dev/null +++ b/out_tensor/model.layers.30.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:246c720619292a53a1849863bfec80dbe5ca94ec5173637fb65ac2139245f643 +size 1688864 diff --git a/out_tensor/model.layers.30.self_attn.o_proj.safetensors b/out_tensor/model.layers.30.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..3c950fd3790ca35059e0a952c1f563bcfe8c5f02 --- /dev/null +++ b/out_tensor/model.layers.30.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7b7547a5d2878ca592b62c42302b4d7cb794cacfa260b4c3f4c42a0dab9ba06f +size 8471264 diff --git a/out_tensor/model.layers.30.self_attn.q_proj.safetensors b/out_tensor/model.layers.30.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..c53d9e0cb093089d1ff6efd594794f22b3a81940 --- /dev/null +++ b/out_tensor/model.layers.30.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:46d80a6d4ae25efea895cf94aba79ef38617cbc8bba5b56af909b0112ee5effc +size 6702376 diff --git a/out_tensor/model.layers.30.self_attn.v_proj.safetensors b/out_tensor/model.layers.30.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..ac72495e9ba3674f16fcec32e56378503dfbe3a4 --- /dev/null +++ b/out_tensor/model.layers.30.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:542e8ee2fd460e1f8a4a9cfdbfed17b6b017712d2ddee3d5c0f06fd072072f19 +size 2229536 diff --git a/out_tensor/model.layers.31.mlp.down_proj.safetensors b/out_tensor/model.layers.31.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..f9ee3c04488f1132e0b3536a7c635c94877703df --- /dev/null +++ b/out_tensor/model.layers.31.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0c82f03b4976ca27674231d96b9c4f7a7799046596e4f20590ac815b66d05d35 +size 23391392 diff --git a/out_tensor/model.layers.31.mlp.gate_proj.safetensors b/out_tensor/model.layers.31.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..d55d3b1adb25d762fcdda7af63ae0f79926c4df8 --- /dev/null +++ b/out_tensor/model.layers.31.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fa1b97ede8eebbee961d5df9737c5dad3e4b4e45b23991ed498dc390bb87d881 +size 29606616 diff --git a/out_tensor/model.layers.31.mlp.up_proj.safetensors b/out_tensor/model.layers.31.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..594ed59c621dba47e6b2d1bbdeb4ed6a92a820ec --- /dev/null +++ b/out_tensor/model.layers.31.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:df5d3b05b2d48ed4560d19a9cb83666941a9bb4534a01f882522d70d53107b04 +size 22955280 diff --git a/out_tensor/model.layers.31.self_attn.k_proj.safetensors b/out_tensor/model.layers.31.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..15cf0342f82d8c16290aa58ceda77de9a365ad73 --- /dev/null +++ b/out_tensor/model.layers.31.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:55bf4113921068aa92026571597e641461b07065a2393e34cbd24a843ed721d0 +size 1688864 diff --git a/out_tensor/model.layers.31.self_attn.o_proj.safetensors b/out_tensor/model.layers.31.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..dcfc893d4a2c1a4ea4c7371bd0eb7d78f57c2068 --- /dev/null +++ b/out_tensor/model.layers.31.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:902e45f66cac287ab3d45ef8348463e462cc4fdf1bd85a0961f41b1a93cd4d10 +size 6571304 diff --git a/out_tensor/model.layers.31.self_attn.q_proj.safetensors b/out_tensor/model.layers.31.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..9406da2ccfcf31c6c41805559f8c14066724d64c --- /dev/null +++ b/out_tensor/model.layers.31.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:83ae4fbdd973c200a7f4dd0792f9bee6ea6de572e5da2a75ce7382cbc1c57918 +size 6702376 diff --git a/out_tensor/model.layers.31.self_attn.v_proj.safetensors b/out_tensor/model.layers.31.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..5d3da0f73788fd40d1b296b62c25cc60d0c73713 --- /dev/null +++ b/out_tensor/model.layers.31.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f49279ec65ebb30f0d2db61a225ee07001093ec6e6218587bb2315793cc466de +size 2229536 diff --git a/out_tensor/model.layers.4.mlp.down_proj.safetensors b/out_tensor/model.layers.4.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..7d5c84262e086a5ec273782cff4f574ff984e379 --- /dev/null +++ b/out_tensor/model.layers.4.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ac85e198e04a0aa17d28ec035b8370c07502fda6dc5d4926fba49526305f6b0a +size 30338200 diff --git a/out_tensor/model.layers.4.mlp.gate_proj.safetensors b/out_tensor/model.layers.4.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..288e77990eace04ef488a07ededdce2cef9b7551 --- /dev/null +++ b/out_tensor/model.layers.4.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ba891ef5a60ddaa3a73756d561e7bc38d9e13d4287a1b952e1a0cdba4c6be8f0 +size 29606616 diff --git a/out_tensor/model.layers.4.mlp.up_proj.safetensors b/out_tensor/model.layers.4.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..50d25ea3883b4cad38c3ea083f24360c29a289e0 --- /dev/null +++ b/out_tensor/model.layers.4.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7d0eb4fd72506d8ffd87854cd52e82fd047b498bcccced43cd02c6e3ca5da6c4 +size 36946632 diff --git a/out_tensor/model.layers.4.self_attn.k_proj.safetensors b/out_tensor/model.layers.4.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..e6e7ff1a642cb8eb13c9b5a584c1ca38a0f047d7 --- /dev/null +++ b/out_tensor/model.layers.4.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a419c9ec5b2622522b23c2cf0e8ef28d4f16407b399d6e46200373dc3dab8f1d +size 1262880 diff --git a/out_tensor/model.layers.4.self_attn.o_proj.safetensors b/out_tensor/model.layers.4.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..cb28ec87d9d372f69b175384dabfe0ebed5a5d37 --- /dev/null +++ b/out_tensor/model.layers.4.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ba9edcfd6f2acf453957672646c4bc9987e04663ab5f3ceac3bb5aba6d179a83 +size 8471256 diff --git a/out_tensor/model.layers.4.self_attn.q_proj.safetensors b/out_tensor/model.layers.4.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..c071670433d7e1673a914258b327c15110e8d6c6 --- /dev/null +++ b/out_tensor/model.layers.4.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:20e0b239c19b5770b7aaf829d4fe8de1bb478837c0de33fa36d942846e62c3c9 +size 4998432 diff --git a/out_tensor/model.layers.4.self_attn.v_proj.safetensors b/out_tensor/model.layers.4.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..55199eb591f727460aa44a704201cafb9a28f093 --- /dev/null +++ b/out_tensor/model.layers.4.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fc2b72421a24d23333ac5966e261dc81825de9115c19acfdeb26436febaff2c7 +size 2130648 diff --git a/out_tensor/model.layers.5.mlp.down_proj.safetensors b/out_tensor/model.layers.5.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..60589fc1a71b14a847a92b99f0159252aed8ea9d --- /dev/null +++ b/out_tensor/model.layers.5.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ec14ac8ea83fe1389330c2f2104d6573ce8555bdb315d992b728fe75cf2f38a4 +size 30338200 diff --git a/out_tensor/model.layers.5.mlp.gate_proj.safetensors b/out_tensor/model.layers.5.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..807d5e1690cc4709d65219bfe54906dad7a57331 --- /dev/null +++ b/out_tensor/model.layers.5.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:96e362daab05ac0b2127f35c95e33e502e0d5c2bf58e3d3584ebb7d7adc9161a +size 29606616 diff --git a/out_tensor/model.layers.5.mlp.up_proj.safetensors b/out_tensor/model.layers.5.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..40225ae6de21880ac95e6cc0fbfc569f56964e18 --- /dev/null +++ b/out_tensor/model.layers.5.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:26521e3a71eb83bc50b97a80ba18c84de87697a46504685f88983523b380d9a3 +size 36946632 diff --git a/out_tensor/model.layers.5.self_attn.k_proj.safetensors b/out_tensor/model.layers.5.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..0049c7d98f00b512190fea913abb6ae5b8c518d2 --- /dev/null +++ b/out_tensor/model.layers.5.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b80f30d385dc8fa6ccf03071ff57d5603f3b2fa9b139c9ba42906520c29e297d +size 1393952 diff --git a/out_tensor/model.layers.5.self_attn.o_proj.safetensors b/out_tensor/model.layers.5.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..5bcb7777396ad7130f443847e5f4cbf01fd7b661 --- /dev/null +++ b/out_tensor/model.layers.5.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:893d07038794c6e0e035c83b502200d26e006cfe34c869c3381e38eff0f8428f +size 8471256 diff --git a/out_tensor/model.layers.5.self_attn.q_proj.safetensors b/out_tensor/model.layers.5.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..ae659b11c096fd56a7e2cbf892319d761524829f --- /dev/null +++ b/out_tensor/model.layers.5.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1db7063fcc9e9345354066dc15a5e68fb7b221b674c9c01a373e114a22db5090 +size 5719328 diff --git a/out_tensor/model.layers.5.self_attn.v_proj.safetensors b/out_tensor/model.layers.5.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..05a6b01b15cf0306f2ec00754897943f1790ce09 --- /dev/null +++ b/out_tensor/model.layers.5.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c784ba1b56502eef2380590af048266575321170a4f52f6b401d043de79aaa31 +size 2180384 diff --git a/out_tensor/model.layers.6.mlp.down_proj.safetensors b/out_tensor/model.layers.6.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..58c0368d77e995f909d0768c7efc60ffbc10b5ef --- /dev/null +++ b/out_tensor/model.layers.6.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:488b959f6068ee05ece86c0139e9ac8dc0995c160e2ffa34466698140a6844eb +size 30338200 diff --git a/out_tensor/model.layers.6.mlp.gate_proj.safetensors b/out_tensor/model.layers.6.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..59e645fa2eae0764fcbcf1260e9c985894511975 --- /dev/null +++ b/out_tensor/model.layers.6.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c1e4d6b98982cf81eeccb07d91dc88599abd38a2e9b09ba6f06af0c3a385db8f +size 29606616 diff --git a/out_tensor/model.layers.6.mlp.up_proj.safetensors b/out_tensor/model.layers.6.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..1556c39714fa93c90fc39a11df2f1d10ccd8edda --- /dev/null +++ b/out_tensor/model.layers.6.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:09b0266fa800850ecb195e4d363f07c01c8e268a2331641e5fef0cf6defc8cda +size 30983440 diff --git a/out_tensor/model.layers.6.self_attn.k_proj.safetensors b/out_tensor/model.layers.6.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..1668da5f73480d3002977bc901c760051f7711db --- /dev/null +++ b/out_tensor/model.layers.6.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1bd98b3e1fdd0f28e1643fd5b7db878627d5fdd5f2d0131d6873a4b7a657d856 +size 1262880 diff --git a/out_tensor/model.layers.6.self_attn.o_proj.safetensors b/out_tensor/model.layers.6.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..5eaf6026a464672d7c8aff64c28d967fc670efd9 --- /dev/null +++ b/out_tensor/model.layers.6.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8984579490c13788e3c1b899c68e4979354a8b3d3e0022f2cce591cf4b3aade8 +size 8471256 diff --git a/out_tensor/model.layers.6.self_attn.q_proj.safetensors b/out_tensor/model.layers.6.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..25dc9e70721ffec1fd6851cefe50534659893e72 --- /dev/null +++ b/out_tensor/model.layers.6.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:717849ad80a94d748d9cd77e0416ac54d821a2b202a7ac905f9b1a59ca400c52 +size 5522720 diff --git a/out_tensor/model.layers.6.self_attn.v_proj.safetensors b/out_tensor/model.layers.6.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..6c91a9109e973ad16d17c1e6dc2c473f8c5339d2 --- /dev/null +++ b/out_tensor/model.layers.6.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:96013518fc4d428c70cdac8a16dff167dcd7873e5d9f14b5ff9391f764904bbe +size 2130648 diff --git a/out_tensor/model.layers.7.mlp.down_proj.safetensors b/out_tensor/model.layers.7.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..62a804066e1acb6f552bcd1fe18c51f640b11209 --- /dev/null +++ b/out_tensor/model.layers.7.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1202904d6177b3f8f5b3b2a5fe451c7b89402e470ab530148d0bf0acaad3194e +size 30338200 diff --git a/out_tensor/model.layers.7.mlp.gate_proj.safetensors b/out_tensor/model.layers.7.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..5f3e370e0b81724e116a59a79152fa8a33656355 --- /dev/null +++ b/out_tensor/model.layers.7.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fabcc1a6b318696cd3b2849e3ff5dd658c247e6552a6a8971464b96f113e0414 +size 29606616 diff --git a/out_tensor/model.layers.7.mlp.up_proj.safetensors b/out_tensor/model.layers.7.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..c63b2aef6bf9f2c9c732ae05a740f2b177bf5886 --- /dev/null +++ b/out_tensor/model.layers.7.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7ac265164d5f30c6ec3b24202720fb4be813d4a8b2ba34266496682d6d93ec4b +size 30983440 diff --git a/out_tensor/model.layers.7.self_attn.k_proj.safetensors b/out_tensor/model.layers.7.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..a15e8ee8a09206c6fcea587a6891a80784304228 --- /dev/null +++ b/out_tensor/model.layers.7.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c2adb184e2a89141167d43053b782c238aaa717238b1e11ecf3d3e7107bb1b8a +size 1393952 diff --git a/out_tensor/model.layers.7.self_attn.o_proj.safetensors b/out_tensor/model.layers.7.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..05a35bbfa279057f24b7aa32e18ddd726b242a80 --- /dev/null +++ b/out_tensor/model.layers.7.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:04286cf1907b895645a43749c50a2ef9aa69e79662e72e087cefd37fdcef7d0a +size 8668448 diff --git a/out_tensor/model.layers.7.self_attn.q_proj.safetensors b/out_tensor/model.layers.7.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..847eec473f4ec6d6bfa8ff0b8d37513eb68f33d1 --- /dev/null +++ b/out_tensor/model.layers.7.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:53281807d3a4e7196d95f54e1f044f3ba935bfff8089dd7bcc6436d132be3b56 +size 5719328 diff --git a/out_tensor/model.layers.7.self_attn.v_proj.safetensors b/out_tensor/model.layers.7.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..559e93182cfdc3a097b2685f292d1f337b1ebfe7 --- /dev/null +++ b/out_tensor/model.layers.7.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2dad56861aad9fe8ef70970df5afb351dbab1846024031f4d3bdd78bc25e6e4d +size 2180384 diff --git a/out_tensor/model.layers.8.mlp.down_proj.safetensors b/out_tensor/model.layers.8.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..23fda20b4edee0756ddc7d0eabb9bddb4edd9e9c --- /dev/null +++ b/out_tensor/model.layers.8.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e2c428c4d68ebb12bc101713def3b82972b93b68e2697e0c1998f30e947510f5 +size 30338200 diff --git a/out_tensor/model.layers.8.mlp.gate_proj.safetensors b/out_tensor/model.layers.8.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..43f2980f7bdbb45969ed643733ff8844bc53cf8e --- /dev/null +++ b/out_tensor/model.layers.8.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:17d3e00e37b88c69a39a84ad875415271592255ca6d24ee3e9af2776e57f5694 +size 29606616 diff --git a/out_tensor/model.layers.8.mlp.up_proj.safetensors b/out_tensor/model.layers.8.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..9410a3f8315379bf6cffb14276e373ff86c36587 --- /dev/null +++ b/out_tensor/model.layers.8.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bcc3b4206ac3025077c807c114946dbca5a3f0e56a729ff9fdda3f221f2d3fa7 +size 30295312 diff --git a/out_tensor/model.layers.8.self_attn.k_proj.safetensors b/out_tensor/model.layers.8.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..9e8573d321228d21845b0c1b5e4169514e21115e --- /dev/null +++ b/out_tensor/model.layers.8.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c61344b74d5f7183f4ec58b014b229c3f42869956a70f630598f42c6d192a77f +size 1393952 diff --git a/out_tensor/model.layers.8.self_attn.o_proj.safetensors b/out_tensor/model.layers.8.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..3ed068caefbafcec7b64b5b64d7fb53d920f5af2 --- /dev/null +++ b/out_tensor/model.layers.8.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1dc35e869dcb2c2b4b0bf834037c569a6cc6686808b4578ab6f280f85711fc0d +size 8668448 diff --git a/out_tensor/model.layers.8.self_attn.q_proj.safetensors b/out_tensor/model.layers.8.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..2601f805613273148e652f11c4c9c8648d27a2d6 --- /dev/null +++ b/out_tensor/model.layers.8.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:78f9edcadcff65e1ce1f878eee3cc3d5e57fb5968f692373379e8ceefdf69eeb +size 5719328 diff --git a/out_tensor/model.layers.8.self_attn.v_proj.safetensors b/out_tensor/model.layers.8.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..c44d8d153d11edd1d00501c9d0054a30cfdb0a39 --- /dev/null +++ b/out_tensor/model.layers.8.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:92688c4f182450437763cbb758aa73fee08d2a9d768ebf92164bcb9ea491473f +size 2130648 diff --git a/out_tensor/model.layers.9.mlp.down_proj.safetensors b/out_tensor/model.layers.9.mlp.down_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..685aee2fb1e4be365548751e2ec4a86bdb706cb7 --- /dev/null +++ b/out_tensor/model.layers.9.mlp.down_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3ec91b98446ad6b063aeba9dab6ba99404a0e597a025e1d1ca29285583f970d3 +size 30338200 diff --git a/out_tensor/model.layers.9.mlp.gate_proj.safetensors b/out_tensor/model.layers.9.mlp.gate_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..c7f0f025115f4318dce81ba4b04053aa760e728f --- /dev/null +++ b/out_tensor/model.layers.9.mlp.gate_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ff75811838d2712867c6b90c43388432fdcbcbd8352b1babf0160d50021daa91 +size 29606616 diff --git a/out_tensor/model.layers.9.mlp.up_proj.safetensors b/out_tensor/model.layers.9.mlp.up_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..f1ff0cb672a4eaa07f077f91d8166d5943272556 --- /dev/null +++ b/out_tensor/model.layers.9.mlp.up_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7ba89bc7ff81f4cc7991e490c8ff80642b557f538e654ff9a408bc1fe0cdc12c +size 30295312 diff --git a/out_tensor/model.layers.9.self_attn.k_proj.safetensors b/out_tensor/model.layers.9.self_attn.k_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..198b64026a844974a84ac9b4dee63940fc9b3bc8 --- /dev/null +++ b/out_tensor/model.layers.9.self_attn.k_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:93d9921e29364f334f2f9a9ff51d3d4b2df19b4eb166d356c6af4c5866298b50 +size 1606360 diff --git a/out_tensor/model.layers.9.self_attn.o_proj.safetensors b/out_tensor/model.layers.9.self_attn.o_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..54c33e25838545fb604ec0b637be69094caa9a1a --- /dev/null +++ b/out_tensor/model.layers.9.self_attn.o_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b8e6bb318e6bc060cbb1c2e77adb9d6454b0aeb4d9e78bc027a30440d797e156 +size 8668448 diff --git a/out_tensor/model.layers.9.self_attn.q_proj.safetensors b/out_tensor/model.layers.9.self_attn.q_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..9d92c82187974d2415444b76605e9c6f30336550 --- /dev/null +++ b/out_tensor/model.layers.9.self_attn.q_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7ec03704d0d4bce9755ab40ba33a05660e3b371c9f9ae05a71c46afbedaf1c92 +size 6374104 diff --git a/out_tensor/model.layers.9.self_attn.v_proj.safetensors b/out_tensor/model.layers.9.self_attn.v_proj.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..8241a01f26f3e47d89b817e851999d56b7b8efc7 --- /dev/null +++ b/out_tensor/model.layers.9.self_attn.v_proj.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d929542150ce489d5f337adeda05de3d568adf546750832bca15793b2057d9b9 +size 2130648 diff --git a/pytorch_model-00001-of-00008.bin b/pytorch_model-00001-of-00008.bin new file mode 100644 index 0000000000000000000000000000000000000000..6e791e1e2e1f3cc9e4a3a09bae5ca9b2b147a90f --- /dev/null +++ b/pytorch_model-00001-of-00008.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5e6e08a052e25fc9766d3c8dc03f9badf7ed78f5ac8bf667bee18d223464a65d +size 1889594419 diff --git a/pytorch_model-00002-of-00008.bin b/pytorch_model-00002-of-00008.bin new file mode 100644 index 0000000000000000000000000000000000000000..5a4098ff80fad9dc8c732308c82868a322b6184f --- /dev/null +++ b/pytorch_model-00002-of-00008.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a211772beab275d1696c295a77cc0fd0b9e5e585d004bdb23541bdf1985a4157 +size 1946253333 diff --git a/pytorch_model-00003-of-00008.bin b/pytorch_model-00003-of-00008.bin new file mode 100644 index 0000000000000000000000000000000000000000..73713d3a4326f882ada86214807e3d0839b638c6 --- /dev/null +++ b/pytorch_model-00003-of-00008.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:715d009ddd4e40d069fc88904eaa6f029fbf87bd7b2b985367618eb2f99c17d0 +size 1979789691 diff --git a/pytorch_model-00004-of-00008.bin b/pytorch_model-00004-of-00008.bin new file mode 100644 index 0000000000000000000000000000000000000000..359e5cd7d7cfedaf7a2a515ebab1e0345cdcb0cb --- /dev/null +++ b/pytorch_model-00004-of-00008.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f6ea860edada7b276bd174f095a03f86fe0c4fdd6d09831d9db2f77fdb3333ec +size 1946253397 diff --git a/pytorch_model-00005-of-00008.bin b/pytorch_model-00005-of-00008.bin new file mode 100644 index 0000000000000000000000000000000000000000..bb8b499f1a2765e5f18300bc19da26e1baf1abc2 --- /dev/null +++ b/pytorch_model-00005-of-00008.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:660240279eed274f7da19d6689621b2a607b05bd136b50d58be93e1401c7856a +size 1979789691 diff --git a/pytorch_model-00006-of-00008.bin b/pytorch_model-00006-of-00008.bin new file mode 100644 index 0000000000000000000000000000000000000000..3e6299d10a6db04ed1d6c3f4ba63b25f79d61ec2 --- /dev/null +++ b/pytorch_model-00006-of-00008.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9ee421c50c3494420a11263a4f1d4b8d70bd22dcf21e4f09414efd60cce0b70b +size 1946253397 diff --git a/pytorch_model-00007-of-00008.bin b/pytorch_model-00007-of-00008.bin new file mode 100644 index 0000000000000000000000000000000000000000..ab4cfe89a7f5b638772575a46df2d4ad56c52195 --- /dev/null +++ b/pytorch_model-00007-of-00008.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7145f22d245fdea964607b314efd4a4e7821cc991b7dd728c5154f1121af6c37 +size 1979789691 diff --git a/pytorch_model-00008-of-00008.bin b/pytorch_model-00008-of-00008.bin new file mode 100644 index 0000000000000000000000000000000000000000..cd8c9ca5c12f00ca761b57fa4d18801dce7f2ee4 --- /dev/null +++ b/pytorch_model-00008-of-00008.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:58ee4fdfec2bf9b297f2943f043461699aed20b2fb1a5632ac168e8866851eb9 +size 815838027 diff --git a/pytorch_model.bin.index.json b/pytorch_model.bin.index.json new file mode 100644 index 0000000000000000000000000000000000000000..3a5060738fb6bf5c4dda463c55bcbad880710785 --- /dev/null +++ b/pytorch_model.bin.index.json @@ -0,0 +1,298 @@ +{ + "metadata": { + "total_size": 14483464192 + }, + "weight_map": { + "lm_head.weight": "pytorch_model-00008-of-00008.bin", + "model.embed_tokens.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.0.input_layernorm.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.0.mlp.down_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.0.mlp.gate_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.0.mlp.up_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.0.post_attention_layernorm.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.0.self_attn.k_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.0.self_attn.o_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.0.self_attn.q_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.0.self_attn.v_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.1.input_layernorm.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.1.mlp.down_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.1.mlp.gate_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.1.mlp.up_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.1.post_attention_layernorm.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.1.self_attn.k_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.1.self_attn.o_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.1.self_attn.q_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.1.self_attn.v_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.10.input_layernorm.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.10.mlp.down_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.10.mlp.gate_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.10.mlp.up_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.10.post_attention_layernorm.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.10.self_attn.k_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.10.self_attn.o_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.10.self_attn.q_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.10.self_attn.v_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.11.input_layernorm.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.11.mlp.down_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.11.mlp.gate_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.11.mlp.up_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.11.post_attention_layernorm.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.11.self_attn.k_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.11.self_attn.o_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.11.self_attn.q_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.11.self_attn.v_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.12.input_layernorm.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.12.mlp.down_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.12.mlp.gate_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.12.mlp.up_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.12.post_attention_layernorm.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.12.self_attn.k_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.12.self_attn.o_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.12.self_attn.q_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.12.self_attn.v_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.13.input_layernorm.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.13.mlp.down_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.13.mlp.gate_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.13.mlp.up_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.13.post_attention_layernorm.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.13.self_attn.k_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.13.self_attn.o_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.13.self_attn.q_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.13.self_attn.v_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.14.input_layernorm.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.14.mlp.down_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.14.mlp.gate_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.14.mlp.up_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.14.post_attention_layernorm.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.14.self_attn.k_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.14.self_attn.o_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.14.self_attn.q_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.14.self_attn.v_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.15.input_layernorm.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.15.mlp.down_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.15.mlp.gate_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.15.mlp.up_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.15.post_attention_layernorm.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.15.self_attn.k_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.15.self_attn.o_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.15.self_attn.q_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.15.self_attn.v_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.16.input_layernorm.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.16.mlp.down_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.16.mlp.gate_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.16.mlp.up_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.16.post_attention_layernorm.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.16.self_attn.k_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.16.self_attn.o_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.16.self_attn.q_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.16.self_attn.v_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.17.input_layernorm.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.17.mlp.down_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.17.mlp.gate_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.17.mlp.up_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.17.post_attention_layernorm.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.17.self_attn.k_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.17.self_attn.o_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.17.self_attn.q_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.17.self_attn.v_proj.weight": "pytorch_model-00004-of-00008.bin", + "model.layers.18.input_layernorm.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.18.mlp.down_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.18.mlp.gate_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.18.mlp.up_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.18.post_attention_layernorm.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.18.self_attn.k_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.18.self_attn.o_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.18.self_attn.q_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.18.self_attn.v_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.19.input_layernorm.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.19.mlp.down_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.19.mlp.gate_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.19.mlp.up_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.19.post_attention_layernorm.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.19.self_attn.k_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.19.self_attn.o_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.19.self_attn.q_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.19.self_attn.v_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.2.input_layernorm.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.2.mlp.down_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.2.mlp.gate_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.2.mlp.up_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.2.post_attention_layernorm.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.2.self_attn.k_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.2.self_attn.o_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.2.self_attn.q_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.2.self_attn.v_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.20.input_layernorm.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.20.mlp.down_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.20.mlp.gate_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.20.mlp.up_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.20.post_attention_layernorm.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.20.self_attn.k_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.20.self_attn.o_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.20.self_attn.q_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.20.self_attn.v_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.21.input_layernorm.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.21.mlp.down_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.21.mlp.gate_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.21.mlp.up_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.21.post_attention_layernorm.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.21.self_attn.k_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.21.self_attn.o_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.21.self_attn.q_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.21.self_attn.v_proj.weight": "pytorch_model-00005-of-00008.bin", + "model.layers.22.input_layernorm.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.22.mlp.down_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.22.mlp.gate_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.22.mlp.up_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.22.post_attention_layernorm.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.22.self_attn.k_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.22.self_attn.o_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.22.self_attn.q_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.22.self_attn.v_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.23.input_layernorm.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.23.mlp.down_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.23.mlp.gate_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.23.mlp.up_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.23.post_attention_layernorm.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.23.self_attn.k_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.23.self_attn.o_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.23.self_attn.q_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.23.self_attn.v_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.24.input_layernorm.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.24.mlp.down_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.24.mlp.gate_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.24.mlp.up_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.24.post_attention_layernorm.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.24.self_attn.k_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.24.self_attn.o_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.24.self_attn.q_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.24.self_attn.v_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.25.input_layernorm.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.25.mlp.down_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.25.mlp.gate_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.25.mlp.up_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.25.post_attention_layernorm.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.25.self_attn.k_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.25.self_attn.o_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.25.self_attn.q_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.25.self_attn.v_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.26.input_layernorm.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.26.mlp.down_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.26.mlp.gate_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.26.mlp.up_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.26.post_attention_layernorm.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.26.self_attn.k_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.26.self_attn.o_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.26.self_attn.q_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.26.self_attn.v_proj.weight": "pytorch_model-00006-of-00008.bin", + "model.layers.27.input_layernorm.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.27.mlp.down_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.27.mlp.gate_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.27.mlp.up_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.27.post_attention_layernorm.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.27.self_attn.k_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.27.self_attn.o_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.27.self_attn.q_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.27.self_attn.v_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.28.input_layernorm.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.28.mlp.down_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.28.mlp.gate_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.28.mlp.up_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.28.post_attention_layernorm.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.28.self_attn.k_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.28.self_attn.o_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.28.self_attn.q_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.28.self_attn.v_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.29.input_layernorm.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.29.mlp.down_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.29.mlp.gate_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.29.mlp.up_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.29.post_attention_layernorm.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.29.self_attn.k_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.29.self_attn.o_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.29.self_attn.q_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.29.self_attn.v_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.3.input_layernorm.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.3.mlp.down_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.3.mlp.gate_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.3.mlp.up_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.3.post_attention_layernorm.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.3.self_attn.k_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.3.self_attn.o_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.3.self_attn.q_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.3.self_attn.v_proj.weight": "pytorch_model-00001-of-00008.bin", + "model.layers.30.input_layernorm.weight": "pytorch_model-00008-of-00008.bin", + "model.layers.30.mlp.down_proj.weight": "pytorch_model-00008-of-00008.bin", + "model.layers.30.mlp.gate_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.30.mlp.up_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.30.post_attention_layernorm.weight": "pytorch_model-00008-of-00008.bin", + "model.layers.30.self_attn.k_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.30.self_attn.o_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.30.self_attn.q_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.30.self_attn.v_proj.weight": "pytorch_model-00007-of-00008.bin", + "model.layers.31.input_layernorm.weight": "pytorch_model-00008-of-00008.bin", + "model.layers.31.mlp.down_proj.weight": "pytorch_model-00008-of-00008.bin", + "model.layers.31.mlp.gate_proj.weight": "pytorch_model-00008-of-00008.bin", + "model.layers.31.mlp.up_proj.weight": "pytorch_model-00008-of-00008.bin", + "model.layers.31.post_attention_layernorm.weight": "pytorch_model-00008-of-00008.bin", + "model.layers.31.self_attn.k_proj.weight": "pytorch_model-00008-of-00008.bin", + "model.layers.31.self_attn.o_proj.weight": "pytorch_model-00008-of-00008.bin", + "model.layers.31.self_attn.q_proj.weight": "pytorch_model-00008-of-00008.bin", + "model.layers.31.self_attn.v_proj.weight": "pytorch_model-00008-of-00008.bin", + "model.layers.4.input_layernorm.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.4.mlp.down_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.4.mlp.gate_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.4.mlp.up_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.4.post_attention_layernorm.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.4.self_attn.k_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.4.self_attn.o_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.4.self_attn.q_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.4.self_attn.v_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.5.input_layernorm.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.5.mlp.down_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.5.mlp.gate_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.5.mlp.up_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.5.post_attention_layernorm.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.5.self_attn.k_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.5.self_attn.o_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.5.self_attn.q_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.5.self_attn.v_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.6.input_layernorm.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.6.mlp.down_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.6.mlp.gate_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.6.mlp.up_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.6.post_attention_layernorm.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.6.self_attn.k_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.6.self_attn.o_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.6.self_attn.q_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.6.self_attn.v_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.7.input_layernorm.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.7.mlp.down_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.7.mlp.gate_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.7.mlp.up_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.7.post_attention_layernorm.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.7.self_attn.k_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.7.self_attn.o_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.7.self_attn.q_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.7.self_attn.v_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.8.input_layernorm.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.8.mlp.down_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.8.mlp.gate_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.8.mlp.up_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.8.post_attention_layernorm.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.8.self_attn.k_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.8.self_attn.o_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.8.self_attn.q_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.8.self_attn.v_proj.weight": "pytorch_model-00002-of-00008.bin", + "model.layers.9.input_layernorm.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.9.mlp.down_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.9.mlp.gate_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.9.mlp.up_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.9.post_attention_layernorm.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.9.self_attn.k_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.9.self_attn.o_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.9.self_attn.q_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.layers.9.self_attn.v_proj.weight": "pytorch_model-00003-of-00008.bin", + "model.norm.weight": "pytorch_model-00008-of-00008.bin" + } +} diff --git a/special_tokens_map.json b/special_tokens_map.json new file mode 100644 index 0000000000000000000000000000000000000000..8cd5f1eb30d4e97d74cbf915c36db116aea5eca7 --- /dev/null +++ b/special_tokens_map.json @@ -0,0 +1,11 @@ +{ + "additional_special_tokens": [ + "", + "", + "" + ], + "bos_token": "", + "eos_token": "", + "pad_token": "", + "unk_token": "" +} diff --git a/thumbnail.png b/thumbnail.png new file mode 100644 index 0000000000000000000000000000000000000000..164cdf3cc45c6c502f9f639d3d820b1baff0d0e3 Binary files /dev/null and b/thumbnail.png differ diff --git a/tokenizer.json b/tokenizer.json new file mode 100644 index 0000000000000000000000000000000000000000..43e6daf936dc0f953cb867ec864adab78f92d9ce --- /dev/null +++ b/tokenizer.json @@ -0,0 +1,91122 @@ +{ + "version": "1.0", + "truncation": null, + "padding": null, + "added_tokens": [ + { + "id": 0, + "content": "", + "single_word": false, + "lstrip": false, + "rstrip": false, + "normalized": false, + "special": true + }, + { + "id": 1, + "content": "", + "single_word": false, + "lstrip": false, + "rstrip": false, + "normalized": false, + "special": true + }, + { + "id": 2, + "content": "", + "single_word": false, + "lstrip": false, + "rstrip": false, + "normalized": false, + "special": true + } + ], + "normalizer": { + "type": "Sequence", + "normalizers": [ + { + "type": "Prepend", + "prepend": "▁" + }, + { + "type": "Replace", + "pattern": { + "String": " " + }, + "content": "▁" + } + ] + }, + "pre_tokenizer": null, + "post_processor": { + "type": "TemplateProcessing", + "single": [ + { + "SpecialToken": { + "id": "", + "type_id": 0 + } + }, + { + "Sequence": { + "id": "A", + "type_id": 0 + } + } + ], + "pair": [ + { + "SpecialToken": { + "id": "", + "type_id": 0 + } + }, + { + "Sequence": { + "id": "A", + "type_id": 0 + } + }, + { + "SpecialToken": { + "id": "", + "type_id": 1 + } + }, + { + "Sequence": { + "id": "B", + "type_id": 1 + } + } + ], + "special_tokens": { + "": { + "id": "", + "ids": [ + 1 + ], + "tokens": [ + "" + ] + } + } + }, + "decoder": { + "type": "Sequence", + "decoders": [ + { + "type": "Replace", + "pattern": { + "String": "▁" + }, + "content": " " + }, + { + "type": "ByteFallback" + }, + { + "type": "Fuse" + }, + { + "type": "Strip", + "content": " ", + "start": 1, + "stop": 0 + } + ] + }, + "model": { + "type": "BPE", + "dropout": null, + "unk_token": "", + "continuing_subword_prefix": null, + "end_of_word_suffix": null, + "fuse_unk": true, + "byte_fallback": true, + "vocab": { + "": 0, + "": 1, + "": 2, + "<0x00>": 3, + "<0x01>": 4, + "<0x02>": 5, + "<0x03>": 6, + "<0x04>": 7, + "<0x05>": 8, + "<0x06>": 9, + "<0x07>": 10, + "<0x08>": 11, + "<0x09>": 12, + "<0x0A>": 13, + "<0x0B>": 14, + "<0x0C>": 15, + "<0x0D>": 16, + "<0x0E>": 17, + "<0x0F>": 18, + "<0x10>": 19, + "<0x11>": 20, + "<0x12>": 21, + "<0x13>": 22, + "<0x14>": 23, + "<0x15>": 24, + "<0x16>": 25, + "<0x17>": 26, + "<0x18>": 27, + "<0x19>": 28, + "<0x1A>": 29, + "<0x1B>": 30, + "<0x1C>": 31, + "<0x1D>": 32, + "<0x1E>": 33, + "<0x1F>": 34, + "<0x20>": 35, + "<0x21>": 36, + "<0x22>": 37, + "<0x23>": 38, + "<0x24>": 39, + "<0x25>": 40, + "<0x26>": 41, + "<0x27>": 42, + "<0x28>": 43, + "<0x29>": 44, + "<0x2A>": 45, + "<0x2B>": 46, + "<0x2C>": 47, + "<0x2D>": 48, + "<0x2E>": 49, + "<0x2F>": 50, + "<0x30>": 51, + "<0x31>": 52, + "<0x32>": 53, + "<0x33>": 54, + "<0x34>": 55, + "<0x35>": 56, + "<0x36>": 57, + "<0x37>": 58, + "<0x38>": 59, + "<0x39>": 60, + "<0x3A>": 61, + "<0x3B>": 62, + "<0x3C>": 63, + "<0x3D>": 64, + "<0x3E>": 65, + "<0x3F>": 66, + "<0x40>": 67, + "<0x41>": 68, + "<0x42>": 69, + "<0x43>": 70, + "<0x44>": 71, + "<0x45>": 72, + "<0x46>": 73, + "<0x47>": 74, + "<0x48>": 75, + "<0x49>": 76, + "<0x4A>": 77, + "<0x4B>": 78, + "<0x4C>": 79, + "<0x4D>": 80, + "<0x4E>": 81, + "<0x4F>": 82, + "<0x50>": 83, + "<0x51>": 84, + "<0x52>": 85, + "<0x53>": 86, + "<0x54>": 87, + "<0x55>": 88, + "<0x56>": 89, + "<0x57>": 90, + "<0x58>": 91, + "<0x59>": 92, + "<0x5A>": 93, + "<0x5B>": 94, + "<0x5C>": 95, + "<0x5D>": 96, + "<0x5E>": 97, + "<0x5F>": 98, + "<0x60>": 99, + "<0x61>": 100, + "<0x62>": 101, + "<0x63>": 102, + "<0x64>": 103, + "<0x65>": 104, + "<0x66>": 105, + "<0x67>": 106, + "<0x68>": 107, + "<0x69>": 108, + "<0x6A>": 109, + "<0x6B>": 110, + "<0x6C>": 111, + "<0x6D>": 112, + "<0x6E>": 113, + "<0x6F>": 114, + "<0x70>": 115, + "<0x71>": 116, + "<0x72>": 117, + "<0x73>": 118, + "<0x74>": 119, + "<0x75>": 120, + "<0x76>": 121, + "<0x77>": 122, + "<0x78>": 123, + "<0x79>": 124, + "<0x7A>": 125, + "<0x7B>": 126, + "<0x7C>": 127, + "<0x7D>": 128, + "<0x7E>": 129, + "<0x7F>": 130, + "<0x80>": 131, + "<0x81>": 132, + "<0x82>": 133, + "<0x83>": 134, + "<0x84>": 135, + "<0x85>": 136, + "<0x86>": 137, + "<0x87>": 138, + "<0x88>": 139, + "<0x89>": 140, + "<0x8A>": 141, + "<0x8B>": 142, + "<0x8C>": 143, + "<0x8D>": 144, + "<0x8E>": 145, + "<0x8F>": 146, + "<0x90>": 147, + "<0x91>": 148, + "<0x92>": 149, + "<0x93>": 150, + "<0x94>": 151, + "<0x95>": 152, + "<0x96>": 153, + "<0x97>": 154, + "<0x98>": 155, + "<0x99>": 156, + "<0x9A>": 157, + "<0x9B>": 158, + "<0x9C>": 159, + "<0x9D>": 160, + "<0x9E>": 161, + "<0x9F>": 162, + "<0xA0>": 163, + "<0xA1>": 164, + "<0xA2>": 165, + "<0xA3>": 166, + "<0xA4>": 167, + "<0xA5>": 168, + "<0xA6>": 169, + "<0xA7>": 170, + "<0xA8>": 171, + "<0xA9>": 172, + "<0xAA>": 173, + "<0xAB>": 174, + "<0xAC>": 175, + "<0xAD>": 176, + "<0xAE>": 177, + "<0xAF>": 178, + "<0xB0>": 179, + "<0xB1>": 180, + "<0xB2>": 181, + "<0xB3>": 182, + "<0xB4>": 183, + "<0xB5>": 184, + "<0xB6>": 185, + "<0xB7>": 186, + "<0xB8>": 187, + "<0xB9>": 188, + "<0xBA>": 189, + "<0xBB>": 190, + "<0xBC>": 191, + "<0xBD>": 192, + "<0xBE>": 193, + "<0xBF>": 194, + "<0xC0>": 195, + "<0xC1>": 196, + "<0xC2>": 197, + "<0xC3>": 198, + "<0xC4>": 199, + "<0xC5>": 200, + "<0xC6>": 201, + "<0xC7>": 202, + "<0xC8>": 203, + "<0xC9>": 204, + "<0xCA>": 205, + "<0xCB>": 206, + "<0xCC>": 207, + "<0xCD>": 208, + "<0xCE>": 209, + "<0xCF>": 210, + "<0xD0>": 211, + "<0xD1>": 212, + "<0xD2>": 213, + "<0xD3>": 214, + "<0xD4>": 215, + "<0xD5>": 216, + "<0xD6>": 217, + "<0xD7>": 218, + "<0xD8>": 219, + "<0xD9>": 220, + "<0xDA>": 221, + "<0xDB>": 222, + "<0xDC>": 223, + "<0xDD>": 224, + "<0xDE>": 225, + "<0xDF>": 226, + "<0xE0>": 227, + "<0xE1>": 228, + "<0xE2>": 229, + "<0xE3>": 230, + "<0xE4>": 231, + "<0xE5>": 232, + "<0xE6>": 233, + "<0xE7>": 234, + "<0xE8>": 235, + "<0xE9>": 236, + "<0xEA>": 237, + "<0xEB>": 238, + "<0xEC>": 239, + "<0xED>": 240, + "<0xEE>": 241, + "<0xEF>": 242, + "<0xF0>": 243, + "<0xF1>": 244, + "<0xF2>": 245, + "<0xF3>": 246, + "<0xF4>": 247, + "<0xF5>": 248, + "<0xF6>": 249, + "<0xF7>": 250, + "<0xF8>": 251, + "<0xF9>": 252, + "<0xFA>": 253, + "<0xFB>": 254, + "<0xFC>": 255, + "<0xFD>": 256, + "<0xFE>": 257, + "<0xFF>": 258, + "▁▁": 259, + "▁▁▁▁": 260, + "▁t": 261, + "in": 262, + "er": 263, + "▁a": 264, + "he": 265, + "on": 266, + "re": 267, + "▁s": 268, + "en": 269, + "at": 270, + "or": 271, + "▁the": 272, + "▁▁▁▁▁▁▁▁": 273, + "es": 274, + "▁w": 275, + "an": 276, + "▁c": 277, + "is": 278, + "it": 279, + "ou": 280, + "▁d": 281, + "al": 282, + "ar": 283, + "▁p": 284, + "▁f": 285, + "ed": 286, + "▁b": 287, + "ing": 288, + "▁o": 289, + "▁m": 290, + "le": 291, + "nd": 292, + "as": 293, + "ic": 294, + "▁h": 295, + "ion": 296, + "▁in": 297, + "▁to": 298, + "et": 299, + "om": 300, + "el": 301, + "▁of": 302, + "st": 303, + "▁and": 304, + "▁l": 305, + "▁th": 306, + "▁n": 307, + "ent": 308, + "il": 309, + "ct": 310, + "ro": 311, + "▁re": 312, + "id": 313, + "am": 314, + "▁I": 315, + "ad": 316, + "▁e": 317, + "▁S": 318, + "▁g": 319, + "▁T": 320, + "im": 321, + "ot": 322, + "ac": 323, + "ur": 324, + "▁(": 325, + "ig": 326, + "▁=": 327, + "ol": 328, + "ut": 329, + "▁A": 330, + "se": 331, + "▁u": 332, + "ve": 333, + "▁C": 334, + "if": 335, + "ow": 336, + "▁y": 337, + "ch": 338, + "ay": 339, + "▁de": 340, + "▁st": 341, + "▁|": 342, + "ver": 343, + ");": 344, + "▁\"": 345, + "ly": 346, + "▁be": 347, + "**": 348, + "▁is": 349, + "od": 350, + "▁M": 351, + "ation": 352, + "ul": 353, + "▁for": 354, + "▁▁▁▁▁": 355, + "▁on": 356, + "ag": 357, + "ce": 358, + "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁": 359, + "ter": 360, + "ir": 361, + "th": 362, + "▁v": 363, + "qu": 364, + "▁B": 365, + "em": 366, + "▁P": 367, + "▁you": 368, + "▁that": 369, + "un": 370, + "▁{": 371, + "ith": 372, + "ri": 373, + "est": 374, + "ab": 375, + "--": 376, + "ap": 377, + "▁it": 378, + "▁con": 379, + "ate": 380, + "us": 381, + "▁H": 382, + "um": 383, + "▁D": 384, + "os": 385, + "pe": 386, + "▁-": 387, + "▁wh": 388, + "▁al": 389, + "▁as": 390, + "and": 391, + "ist": 392, + "▁L": 393, + "▁W": 394, + "▁with": 395, + "▁an": 396, + "ere": 397, + "▁*": 398, + "▁R": 399, + "▁he": 400, + "▁F": 401, + "oc": 402, + "▁was": 403, + "ers": 404, + "ke": 405, + "out": 406, + "ht": 407, + "▁r": 408, + "ess": 409, + "op": 410, + "res": 411, + "ie": 412, + "▁E": 413, + "▁\\": 414, + "▁The": 415, + "end": 416, + "ld": 417, + "▁N": 418, + "ort": 419, + "▁G": 420, + "//": 421, + "▁#": 422, + "our": 423, + "te": 424, + "ill": 425, + "ain": 426, + "▁se": 427, + "▁▁▁▁▁▁": 428, + "▁$": 429, + "▁pro": 430, + "ore": 431, + "▁com": 432, + "ame": 433, + "tr": 434, + "▁ne": 435, + "rom": 436, + "ub": 437, + "▁at": 438, + "▁ex": 439, + "ant": 440, + "ue": 441, + "▁or": 442, + "▁}": 443, + "art": 444, + "ction": 445, + "▁k": 446, + "pt": 447, + "nt": 448, + "iv": 449, + "de": 450, + "▁O": 451, + "pl": 452, + "urn": 453, + "ight": 454, + "all": 455, + "▁this": 456, + "ser": 457, + "ave": 458, + "▁not": 459, + "▁are": 460, + "▁j": 461, + "▁le": 462, + "iz": 463, + "▁'": 464, + "age": 465, + "ment": 466, + "▁tr": 467, + "ack": 468, + "ust": 469, + "()": 470, + "->": 471, + "ity": 472, + "ine": 473, + "ould": 474, + "▁J": 475, + "og": 476, + "▁from": 477, + "▁we": 478, + "ell": 479, + "▁sh": 480, + "▁en": 481, + "ure": 482, + "port": 483, + "▁ch": 484, + "ne": 485, + "▁by": 486, + "per": 487, + "ard": 488, + "ass": 489, + "ge": 490, + "ak": 491, + "are": 492, + "ok": 493, + "av": 494, + "ive": 495, + "ff": 496, + "ies": 497, + "ath": 498, + "turn": 499, + "▁U": 500, + "int": 501, + "----": 502, + "▁im": 503, + "ost": 504, + "ial": 505, + "▁have": 506, + "ind": 507, + "ip": 508, + "ans": 509, + "xt": 510, + "▁do": 511, + "cl": 512, + "▁if": 513, + "con": 514, + "ia": 515, + "▁his": 516, + "ult": 517, + "rou": 518, + "▁su": 519, + "ra": 520, + "▁un": 521, + "able": 522, + "▁<": 523, + "▁K": 524, + "ome": 525, + "▁qu": 526, + "get": 527, + "▁me": 528, + "ast": 529, + "ect": 530, + "▁##": 531, + "to": 532, + "▁cl": 533, + "▁ab": 534, + "ice": 535, + "ire": 536, + "ber": 537, + "one": 538, + "ich": 539, + "hen": 540, + "▁can": 541, + "▁Th": 542, + "▁la": 543, + "▁all": 544, + "ime": 545, + "ile": 546, + "ide": 547, + "\",": 548, + "▁pl": 549, + "▁V": 550, + "ru": 551, + "orm": 552, + "▁had": 553, + "ud": 554, + "ase": 555, + "ord": 556, + "),": 557, + "▁▁▁▁▁▁▁▁▁▁▁▁": 558, + "▁her": 559, + "▁In": 560, + "ace": 561, + "▁but": 562, + "ata": 563, + "::": 564, + "****": 565, + "ong": 566, + "▁&": 567, + "..": 568, + "▁▁▁▁▁▁▁▁▁▁▁▁▁": 569, + "ite": 570, + "ype": 571, + "act": 572, + "ode": 573, + "▁your": 574, + "▁out": 575, + "▁go": 576, + "lic": 577, + "ally": 578, + "▁so": 579, + "ork": 580, + "au": 581, + "▁up": 582, + "▁_": 583, + "ll": 584, + "==": 585, + "▁my": 586, + "pp": 587, + "cc": 588, + "▁//": 589, + "▁they": 590, + "gh": 591, + "▁us": 592, + "ib": 593, + "ions": 594, + "ach": 595, + "ens": 596, + "▁ar": 597, + "ob": 598, + "elf": 599, + "ook": 600, + "ated": 601, + "ang": 602, + "ign": 603, + "▁return": 604, + "▁res": 605, + "ck": 606, + "ous": 607, + "ст": 608, + ").": 609, + "▁п": 610, + ".\"": 611, + "на": 612, + "▁i": 613, + "ail": 614, + "ep": 615, + "▁ad": 616, + "ance": 617, + "(\"": 618, + "▁**": 619, + "ther": 620, + "ake": 621, + "▁will": 622, + "▁comp": 623, + "▁one": 624, + "▁get": 625, + "ov": 626, + "▁Y": 627, + "ary": 628, + "ock": 629, + "▁she": 630, + "che": 631, + "ft": 632, + "▁new": 633, + "▁des": 634, + "▁li": 635, + "ence": 636, + "▁sa": 637, + "ress": 638, + "▁el": 639, + "▁und": 640, + "eg": 641, + "fer": 642, + "ry": 643, + "ear": 644, + "ose": 645, + "very": 646, + "',": 647, + "▁+": 648, + "▁в": 649, + "▁He": 650, + "ublic": 651, + "▁their": 652, + "ize": 653, + "▁were": 654, + "ink": 655, + "own": 656, + "In": 657, + "{\\": 658, + "▁has": 659, + "▁per": 660, + "▁It": 661, + "▁St": 662, + "her": 663, + "ject": 664, + "ра": 665, + "ild": 666, + "so": 667, + "▁sp": 668, + "ни": 669, + "du": 670, + "row": 671, + "alue": 672, + "set": 673, + "form": 674, + "com": 675, + "▁man": 676, + "ont": 677, + "ull": 678, + "▁cont": 679, + "▁more": 680, + "ick": 681, + "▁would": 682, + "▁ev": 683, + "▁about": 684, + "ition": 685, + "▁z": 686, + "ound": 687, + "ree": 688, + "▁Ch": 689, + "▁which": 690, + "io": 691, + "();": 692, + "▁who": 693, + "err": 694, + "ory": 695, + "ount": 696, + "ations": 697, + "▁с": 698, + "ring": 699, + "": 876, + "▁em": 877, + "▁$\\": 878, + "▁year": 879, + "wn": 880, + "},": 881, + "▁del": 882, + "ale": 883, + "ty": 884, + "fig": 885, + "sp": 886, + "hed": 887, + "round": 888, + "ew": 889, + "▁di": 890, + "▁der": 891, + "ри": 892, + "red": 893, + "this": 894, + "let": 895, + "RE": 896, + "ax": 897, + "fr": 898, + "essage": 899, + "ough": 900, + "▁comm": 901, + "fo": 902, + "uch": 903, + "oy": 904, + "▁people": 905, + "ystem": 906, + "▁first": 907, + "▁function": 908, + "ange": 909, + "▁how": 910, + "▁et": 911, + "ah": 912, + "▁look": 913, + "то": 914, + "und": 915, + "▁under": 916, + "ка": 917, + "▁!": 918, + "ray": 919, + "ST": 920, + "ific": 921, + "ли": 922, + "read": 923, + "▁bet": 924, + "ious": 925, + "arg": 926, + "▁need": 927, + "math": 928, + "▁на": 929, + "ert": 930, + "▁op": 931, + "▁acc": 932, + "Pro": 933, + "▁est": 934, + "▁Un": 935, + "▁ent": 936, + "▁rec": 937, + "▁use": 938, + "ен": 939, + "▁par": 940, + "az": 941, + "▁д": 942, + "▁Wh": 943, + "self": 944, + "▁ke": 945, + "та": 946, + "▁want": 947, + "▁end": 948, + "▁don": 949, + "ek": 950, + "ren": 951, + "Name": 952, + "▁=>": 953, + "▁app": 954, + "▁que": 955, + "igh": 956, + "▁bu": 957, + "equ": 958, + "vel": 959, + "▁act": 960, + "cre": 961, + "AT": 962, + "▁var": 963, + "cess": 964, + "====": 965, + "Ex": 966, + "▁add": 967, + "▁mod": 968, + "ung": 969, + "▁where": 970, + "ning": 971, + "▁fl": 972, + "als": 973, + "tern": 974, + "}}": 975, + "▁Al": 976, + "▁pos": 977, + "ank": 978, + "▁ap": 979, + "eng": 980, + "▁“": 981, + "ble": 982, + "▁reg": 983, + "^{": 984, + "▁She": 985, + "▁*/": 986, + "ude": 987, + "add": 988, + "▁two": 989, + "▁col": 990, + "▁sm": 991, + "air": 992, + "▁may": 993, + "fore": 994, + "▁You": 995, + "rough": 996, + "▁che": 997, + "▁att": 998, + "oth": 999, + "ла": 1000, + "▁co": 1001, + "ates": 1002, + "▁rem": 1003, + "ood": 1004, + "Type": 1005, + "led": 1006, + "ful": 1007, + "▁self": 1008, + "of": 1009, + "▁Ar": 1010, + "que": 1011, + "▁every": 1012, + "ref": 1013, + "The": 1014, + "▁And": 1015, + "▁rel": 1016, + "OR": 1017, + "Id": 1018, + "▁even": 1019, + "EN": 1020, + "▁hand": 1021, + "ait": 1022, + "▁should": 1023, + "▁after": 1024, + "▁dif": 1025, + "ght": 1026, + "ife": 1027, + "ator": 1028, + "ash": 1029, + "ribut": 1030, + "umber": 1031, + "▁see": 1032, + "ms": 1033, + "▁call": 1034, + "yn": 1035, + "dd": 1036, + "▁es": 1037, + "▁make": 1038, + "other": 1039, + "▁—": 1040, + "\");": 1041, + "str": 1042, + "▁long": 1043, + "lement": 1044, + "▁wor": 1045, + "its": 1046, + "▁If": 1047, + "alse": 1048, + "ль": 1049, + "ward": 1050, + "▁по": 1051, + "val": 1052, + "ons": 1053, + "▁Z": 1054, + "▁now": 1055, + "data": 1056, + "amp": 1057, + "ense": 1058, + "▁through": 1059, + "▁down": 1060, + "att": 1061, + "▁static": 1062, + "ics": 1063, + "##": 1064, + "pos": 1065, + "▁void": 1066, + "aw": 1067, + "oun": 1068, + "▁way": 1069, + "ible": 1070, + "vent": 1071, + "ower": 1072, + "▁think": 1073, + "ts": 1074, + "*/": 1075, + "▁again": 1076, + "ating": 1077, + "те": 1078, + "ner": 1079, + "▁most": 1080, + "line": 1081, + "ym": 1082, + "▁sub": 1083, + "erson": 1084, + "▁requ": 1085, + "AL": 1086, + "AR": 1087, + "abel": 1088, + "ond": 1089, + "));": 1090, + "▁Se": 1091, + "▁But": 1092, + "alk": 1093, + "▁An": 1094, + "new": 1095, + "▁because": 1096, + "ger": 1097, + "ular": 1098, + "roup": 1099, + "ta": 1100, + "...": 1101, + "▁cons": 1102, + "▁right": 1103, + "▁fr": 1104, + "be": 1105, + "ily": 1106, + "ки": 1107, + "▁ph": 1108, + "ead": 1109, + "?\"": 1110, + "▁gu": 1111, + "▁else": 1112, + "▁som": 1113, + "rent": 1114, + "co": 1115, + "ement": 1116, + "▁str": 1117, + "ault": 1118, + "▁з": 1119, + "ло": 1120, + "sert": 1121, + "var": 1122, + "type": 1123, + "▁Com": 1124, + "ле": 1125, + "ins": 1126, + "me": 1127, + "way": 1128, + "ident": 1129, + "▁prov": 1130, + "▁м": 1131, + "▁true": 1132, + "▁Pro": 1133, + "fl": 1134, + "▁sl": 1135, + "▁As": 1136, + "}\\": 1137, + "ID": 1138, + "ues": 1139, + "▁inst": 1140, + "▁name": 1141, + "ox": 1142, + "▁)": 1143, + "li": 1144, + "ames": 1145, + "Res": 1146, + "▁sur": 1147, + "param": 1148, + "▁start": 1149, + "aj": 1150, + "SE": 1151, + "ask": 1152, + "IT": 1153, + "String": 1154, + "▁ass": 1155, + "▁play": 1156, + "ting": 1157, + "ton": 1158, + "▁before": 1159, + "▁pol": 1160, + "arch": 1161, + "▁well": 1162, + "Com": 1163, + "any": 1164, + "olog": 1165, + "▁err": 1166, + "▁these": 1167, + "ars": 1168, + "eb": 1169, + "▁br": 1170, + "▁incl": 1171, + "▁hel": 1172, + "ern": 1173, + "ody": 1174, + "во": 1175, + "▁ind": 1176, + "----------------": 1177, + "▁data": 1178, + "▁good": 1179, + "LE": 1180, + "],": 1181, + "▁av": 1182, + "▁ac": 1183, + "ider": 1184, + "не": 1185, + "▁Q": 1186, + "▁min": 1187, + "▁much": 1188, + "ci": 1189, + "els": 1190, + "▁cur": 1191, + "▁value": 1192, + "ery": 1193, + "uf": 1194, + "▁loc": 1195, + "reak": 1196, + "ative": 1197, + "imes": 1198, + "Cl": 1199, + "▁,": 1200, + "▁ser": 1201, + "▁die": 1202, + "▁trans": 1203, + "▁result": 1204, + "ext": 1205, + "▁aut": 1206, + "land": 1207, + "▁&&": 1208, + "Ch": 1209, + "ten": 1210, + "}$": 1211, + "▁type": 1212, + "cond": 1213, + "ices": 1214, + "▁very": 1215, + "▁own": 1216, + "▁fil": 1217, + "ities": 1218, + "▁produ": 1219, + "▁read": 1220, + "▁form": 1221, + "▁case": 1222, + "ather": 1223, + "ти": 1224, + "да": 1225, + "ер": 1226, + "Th": 1227, + "aut": 1228, + "▁spec": 1229, + "ij": 1230, + "bl": 1231, + "ility": 1232, + "▁é": 1233, + "▁er": 1234, + "▁does": 1235, + "▁here": 1236, + "the": 1237, + "ures": 1238, + "▁%": 1239, + "min": 1240, + "▁null": 1241, + "rap": 1242, + "\")": 1243, + "rr": 1244, + "List": 1245, + "right": 1246, + "▁User": 1247, + "UL": 1248, + "ational": 1249, + "▁being": 1250, + "AN": 1251, + "sk": 1252, + "▁car": 1253, + "ole": 1254, + "▁dist": 1255, + "plic": 1256, + "ollow": 1257, + "▁pres": 1258, + "▁such": 1259, + "ream": 1260, + "ince": 1261, + "gan": 1262, + "▁For": 1263, + "\":": 1264, + "son": 1265, + "rivate": 1266, + "▁years": 1267, + "▁serv": 1268, + "▁made": 1269, + "def": 1270, + ";\r": 1271, + "▁gl": 1272, + "▁bel": 1273, + "▁list": 1274, + "▁cor": 1275, + "▁det": 1276, + "ception": 1277, + "egin": 1278, + "▁б": 1279, + "▁char": 1280, + "trans": 1281, + "▁fam": 1282, + "▁!=": 1283, + "ouse": 1284, + "▁dec": 1285, + "ica": 1286, + "▁many": 1287, + "aking": 1288, + "▁à": 1289, + "▁sim": 1290, + "ages": 1291, + "uff": 1292, + "ased": 1293, + "man": 1294, + "▁Sh": 1295, + "iet": 1296, + "irect": 1297, + "▁Re": 1298, + "▁differ": 1299, + "▁find": 1300, + "ethod": 1301, + "▁\r": 1302, + "ines": 1303, + "▁inv": 1304, + "▁point": 1305, + "▁They": 1306, + "▁used": 1307, + "ctions": 1308, + "▁still": 1309, + "ió": 1310, + "ined": 1311, + "▁while": 1312, + "It": 1313, + "ember": 1314, + "▁say": 1315, + "▁help": 1316, + "▁cre": 1317, + "▁x": 1318, + "▁Tr": 1319, + "ument": 1320, + "▁sk": 1321, + "ought": 1322, + "ually": 1323, + "message": 1324, + "▁Con": 1325, + "▁mon": 1326, + "ared": 1327, + "work": 1328, + "):": 1329, + "ister": 1330, + "arn": 1331, + "ized": 1332, + "Data": 1333, + "orn": 1334, + "▁head": 1335, + "DE": 1336, + "▁Le": 1337, + "▁person": 1338, + "ments": 1339, + "ength": 1340, + "▁false": 1341, + "▁med": 1342, + "▁De": 1343, + "ache": 1344, + "ited": 1345, + "▁let": 1346, + "▁show": 1347, + "▁same": 1348, + "uss": 1349, + "▁gener": 1350, + "▁у": 1351, + "cur": 1352, + "▁real": 1353, + "ced": 1354, + "\">": 1355, + "struct": 1356, + "begin": 1357, + "cept": 1358, + "▁bo": 1359, + "ired": 1360, + "▁Fr": 1361, + "▁stud": 1362, + "dev": 1363, + "Ar": 1364, + "(\\": 1365, + "▁Cl": 1366, + "ween": 1367, + "▁too": 1368, + "▁test": 1369, + "▁day": 1370, + "oh": 1371, + "▁follow": 1372, + "ature": 1373, + "ze": 1374, + "ien": 1375, + "reg": 1376, + "ces": 1377, + "uring": 1378, + "amb": 1379, + "ina": 1380, + "cri": 1381, + "▁ed": 1382, + "SS": 1383, + "uck": 1384, + "▁/*": 1385, + "CT": 1386, + "▁There": 1387, + "▁take": 1388, + "par": 1389, + "ule": 1390, + "cal": 1391, + "for": 1392, + "****************": 1393, + "source": 1394, + "▁those": 1395, + "col": 1396, + "▁eff": 1397, + "mod": 1398, + "cont": 1399, + "}{": 1400, + "▁around": 1401, + "press": 1402, + "by": 1403, + "▁going": 1404, + "ponse": 1405, + "▁С": 1406, + "▁line": 1407, + "date": 1408, + "code": 1409, + "['": 1410, + "▁life": 1411, + "ason": 1412, + "▁using": 1413, + "▁val": 1414, + "▁du": 1415, + "yp": 1416, + "▁▁▁▁▁▁▁▁▁▁▁▁▁▁": 1417, + "▁On": 1418, + "▁found": 1419, + "olut": 1420, + "']": 1421, + "arent": 1422, + "▁string": 1423, + "▁met": 1424, + "▁wr": 1425, + "ush": 1426, + "string": 1427, + "size": 1428, + "▁ver": 1429, + "▁each": 1430, + "value": 1431, + "▁last": 1432, + "▁got": 1433, + "ven": 1434, + "back": 1435, + "Set": 1436, + "ey": 1437, + "rol": 1438, + "▁cr": 1439, + "thing": 1440, + "ret": 1441, + "és": 1442, + "ism": 1443, + "▁between": 1444, + "Ob": 1445, + "ething": 1446, + "mp": 1447, + "▁lo": 1448, + "ats": 1449, + "▁New": 1450, + "ви": 1451, + "ado": 1452, + "dex": 1453, + "ди": 1454, + "▁pass": 1455, + "wh": 1456, + "▁den": 1457, + "Get": 1458, + "apt": 1459, + "▁ask": 1460, + "▁sup": 1461, + "Value": 1462, + "ны": 1463, + "▁try": 1464, + "lation": 1465, + "day": 1466, + "ness": 1467, + "ets": 1468, + "▁exper": 1469, + "Tr": 1470, + "▁Mar": 1471, + "serv": 1472, + "br": 1473, + "▁number": 1474, + "inal": 1475, + "cent": 1476, + "/*": 1477, + "not": 1478, + "ional": 1479, + "▁final": 1480, + "')": 1481, + "▁run": 1482, + "over": 1483, + "▁never": 1484, + "uc": 1485, + "▁high": 1486, + "yle": 1487, + "▁ins": 1488, + "▁best": 1489, + "ittle": 1490, + "ric": 1491, + "▁sign": 1492, + "▁dem": 1493, + "iness": 1494, + "gy": 1495, + "▁war": 1496, + "ished": 1497, + "▁giv": 1498, + "key": 1499, + "▁X": 1500, + "($": 1501, + "▁child": 1502, + "less": 1503, + "ways": 1504, + "incl": 1505, + "rop": 1506, + "raw": 1507, + "://": 1508, + "▁«": 1509, + "no": 1510, + "indow": 1511, + "fe": 1512, + "riend": 1513, + "▁les": 1514, + "▁los": 1515, + "file": 1516, + "formation": 1517, + "ccess": 1518, + "▁В": 1519, + "na": 1520, + "▁il": 1521, + "ision": 1522, + "ler": 1523, + "▁art": 1524, + "Cont": 1525, + "▁world": 1526, + "▁turn": 1527, + "▁really": 1528, + "▁Ex": 1529, + "ма": 1530, + "▁П": 1531, + "ters": 1532, + "arget": 1533, + "Err": 1534, + "▁happ": 1535, + "time": 1536, + "▁So": 1537, + "div": 1538, + "▁didn": 1539, + "ada": 1540, + "oot": 1541, + "})": 1542, + "▁sch": 1543, + "▁cle": 1544, + "▁something": 1545, + "().": 1546, + "▁cour": 1547, + "ever": 1548, + "ants": 1549, + "▁?": 1550, + "To": 1551, + "▁`": 1552, + "try": 1553, + "ux": 1554, + "ais": 1555, + "ross": 1556, + "hip": 1557, + "▁rep": 1558, + "label": 1559, + "▁both": 1560, + "*,": 1561, + "ott": 1562, + "ми": 1563, + "ane": 1564, + "▁open": 1565, + "ww": 1566, + "▁come": 1567, + "▁ext": 1568, + "rem": 1569, + "_{\\": 1570, + "▁old": 1571, + "ched": 1572, + "._": 1573, + "ME": 1574, + "ify": 1575, + "gg": 1576, + "Col": 1577, + "view": 1578, + "▁bus": 1579, + "▁must": 1580, + "▁different": 1581, + "log": 1582, + "ists": 1583, + "roll": 1584, + "ai": 1585, + "▁за": 1586, + "▁system": 1587, + "ivers": 1588, + "atus": 1589, + "ote": 1590, + "med": 1591, + "].": 1592, + "akes": 1593, + "RO": 1594, + "▁cent": 1595, + "gram": 1596, + "▁private": 1597, + "▁great": 1598, + "\";": 1599, + "opy": 1600, + "▁feel": 1601, + "▁How": 1602, + "////": 1603, + "IC": 1604, + "▁dr": 1605, + "ains": 1606, + "lock": 1607, + "En": 1608, + "▁Sch": 1609, + "▁mat": 1610, + "▁home": 1611, + "perty": 1612, + "test": 1613, + "loc": 1614, + "▁wom": 1615, + "sw": 1616, + "arly": 1617, + "▁En": 1618, + "▁ко": 1619, + "den": 1620, + "ста": 1621, + "▁а": 1622, + "eter": 1623, + "▁includ": 1624, + "ULL": 1625, + "▁mem": 1626, + "▁po": 1627, + "▁little": 1628, + "▁arg": 1629, + "▁},": 1630, + "include": 1631, + "eta": 1632, + "▁place": 1633, + "idth": 1634, + "ustom": 1635, + "▁||": 1636, + "▁tem": 1637, + "ried": 1638, + "▁fact": 1639, + "ience": 1640, + "▁Pl": 1641, + "opt": 1642, + "ele": 1643, + "go": 1644, + "AC": 1645, + "inter": 1646, + "========": 1647, + "(),": 1648, + "ots": 1649, + "ral": 1650, + "ique": 1651, + "aving": 1652, + "ml": 1653, + "▁thought": 1654, + "frac": 1655, + "▁care": 1656, + "());": 1657, + "▁put": 1658, + "▁might": 1659, + "▁Amer": 1660, + "▁(!": 1661, + "ample": 1662, + "alth": 1663, + "▁few": 1664, + "▁state": 1665, + "sub": 1666, + "▁Or": 1667, + "];": 1668, + "▁size": 1669, + "▁Sp": 1670, + "▁without": 1671, + "▁poss": 1672, + "eq": 1673, + "play": 1674, + "▁expect": 1675, + "▁second": 1676, + "▁String": 1677, + "uild": 1678, + "▁next": 1679, + "++": 1680, + "requ": 1681, + "▁All": 1682, + "▁men": 1683, + "▁When": 1684, + "iter": 1685, + "ament": 1686, + "net": 1687, + "▁К": 1688, + "ron": 1689, + "aint": 1690, + "▁Is": 1691, + "ве": 1692, + "pend": 1693, + "translation": 1694, + "▁го": 1695, + "че": 1696, + "▁van": 1697, + "▁another": 1698, + "▁ret": 1699, + "▁La": 1700, + "Mod": 1701, + "ION": 1702, + "list": 1703, + "▁post": 1704, + "da": 1705, + "ware": 1706, + "▁word": 1707, + "Error": 1708, + "▁seem": 1709, + "▁contin": 1710, + "atic": 1711, + "▁three": 1712, + "Object": 1713, + "▁partic": 1714, + "$.": 1715, + "▁mark": 1716, + "▁vis": 1717, + "rc": 1718, + "▁sw": 1719, + "ptions": 1720, + "▁break": 1721, + "▁things": 1722, + "ute": 1723, + "ui": 1724, + "▁That": 1725, + "urs": 1726, + "gl": 1727, + "ру": 1728, + "▁file": 1729, + "use": 1730, + "igned": 1731, + "part": 1732, + "Un": 1733, + "▁equ": 1734, + "(&": 1735, + "▁lead": 1736, + "rm": 1737, + "ained": 1738, + "▁Be": 1739, + "path": 1740, + "▁small": 1741, + "ager": 1742, + "▁always": 1743, + "▁El": 1744, + "▁order": 1745, + "▁ey": 1746, + "▁won": 1747, + "ape": 1748, + "▁left": 1749, + "ava": 1750, + "item": 1751, + "hor": 1752, + "▁away": 1753, + "bb": 1754, + "fun": 1755, + "▁Ind": 1756, + "mb": 1757, + "▁struct": 1758, + "▁process": 1759, + "▁support": 1760, + ");\r": 1761, + "ión": 1762, + "LO": 1763, + "▁oper": 1764, + "UT": 1765, + "▁·": 1766, + "PE": 1767, + "load": 1768, + "off": 1769, + "▁No": 1770, + "ives": 1771, + "ican": 1772, + "▁ve": 1773, + "action": 1774, + "';": 1775, + "▁vo": 1776, + "$,": 1777, + "▁Gr": 1778, + "pre": 1779, + "ny": 1780, + "aining": 1781, + "ior": 1782, + "init": 1783, + "lection": 1784, + "arm": 1785, + "umn": 1786, + "ags": 1787, + "ци": 1788, + "ско": 1789, + "version": 1790, + "▁To": 1791, + "▁ref": 1792, + "stand": 1793, + "▁At": 1794, + "ift": 1795, + "▁ein": 1796, + "face": 1797, + "bo": 1798, + "ified": 1799, + "ved": 1800, + "sum": 1801, + "une": 1802, + "ital": 1803, + "ump": 1804, + "comm": 1805, + "▁mov": 1806, + "elt": 1807, + "▁von": 1808, + "velop": 1809, + "ctor": 1810, + "head": 1811, + "cle": 1812, + "▁build": 1813, + "inc": 1814, + ".'": 1815, + "bs": 1816, + "info": 1817, + "chn": 1818, + "▁week": 1819, + "▁book": 1820, + "HE": 1821, + "bar": 1822, + "icense": 1823, + "▁What": 1824, + "▁quest": 1825, + "urch": 1826, + "ato": 1827, + "left": 1828, + "▁mar": 1829, + "▁top": 1830, + "FF": 1831, + "▁friend": 1832, + "▁beh": 1833, + "▁field": 1834, + "▁against": 1835, + "ract": 1836, + "ization": 1837, + "user": 1838, + "chen": 1839, + "▁keep": 1840, + "AD": 1841, + "itor": 1842, + "▁non": 1843, + "ird": 1844, + "ope": 1845, + "▁rest": 1846, + "▁dev": 1847, + "▁__": 1848, + "▁una": 1849, + "▁term": 1850, + "IS": 1851, + "▁pop": 1852, + "rist": 1853, + "▁since": 1854, + "ves": 1855, + "▁hard": 1856, + "pi": 1857, + "util": 1858, + "▁soc": 1859, + "ene": 1860, + "Exception": 1861, + "▁local": 1862, + "▁direct": 1863, + "▁sure": 1864, + "▁bro": 1865, + "▁da": 1866, + "▁": 2370, + "aim": 2371, + "▁service": 2372, + "▁within": 2373, + "angu": 2374, + "▁Д": 2375, + "uffer": 2376, + "AG": 2377, + "▁Do": 2378, + "▁incre": 2379, + "▁understand": 2380, + "}^": 2381, + "▁looked": 2382, + "gen": 2383, + "ailed": 2384, + "▁е": 2385, + "ayer": 2386, + "▁One": 2387, + "▁bas": 2388, + "▁job": 2389, + "mu": 2390, + "but": 2391, + "elta": 2392, + "▁Christ": 2393, + "uration": 2394, + "▁record": 2395, + "▁Univers": 2396, + "ivid": 2397, + "valid": 2398, + "▁Р": 2399, + "▁hold": 2400, + "▁table": 2401, + "ones": 2402, + "link": 2403, + "▁Ge": 2404, + "▁offer": 2405, + "ster": 2406, + "Form": 2407, + "={": 2408, + "▁не": 2409, + "stance": 2410, + "▁govern": 2411, + "▁techn": 2412, + "▁prim": 2413, + "*.": 2414, + "cho": 2415, + "max": 2416, + "▁fore": 2417, + "▁Can": 2418, + "▁polit": 2419, + "ories": 2420, + "▁times": 2421, + "▁dans": 2422, + "▁air": 2423, + "▁anything": 2424, + "▁sever": 2425, + "acy": 2426, + "}_": 2427, + "He": 2428, + "▁least": 2429, + "ips": 2430, + "ENT": 2431, + "do": 2432, + "▁от": 2433, + "▁cost": 2434, + ".”": 2435, + "▁children": 2436, + "ability": 2437, + "But": 2438, + "▁path": 2439, + "result": 2440, + "acter": 2441, + "▁element": 2442, + "ee": 2443, + "▁wait": 2444, + "▁money": 2445, + "Map": 2446, + "td": 2447, + "oin": 2448, + "iving": 2449, + "icht": 2450, + "icy": 2451, + "sch": 2452, + "ste": 2453, + "ду": 2454, + "ored": 2455, + "oud": 2456, + "ille": 2457, + "ised": 2458, + "plication": 2459, + "▁custom": 2460, + "▁having": 2461, + "ponent": 2462, + "▁By": 2463, + "ules": 2464, + "ued": 2465, + "atter": 2466, + "And": 2467, + "itive": 2468, + "Def": 2469, + "▁moment": 2470, + "aterial": 2471, + "Class": 2472, + "ograph": 2473, + "ike": 2474, + "▁large": 2475, + "▁####": 2476, + "▁either": 2477, + "duct": 2478, + "▁Then": 2479, + "▁Gu": 2480, + "olean": 2481, + "pert": 2482, + "▁Get": 2483, + "▁Ab": 2484, + "▁short": 2485, + "On": 2486, + "iment": 2487, + "▁project": 2488, + "cript": 2489, + "▁including": 2490, + "ния": 2491, + "▁making": 2492, + "▁someone": 2493, + "▁Fl": 2494, + "▁sat": 2495, + "▁company": 2496, + "ocus": 2497, + "pu": 2498, + "▁God": 2499, + "ification": 2500, + "No": 2501, + "▁sn": 2502, + "ano": 2503, + "ga": 2504, + "▁au": 2505, + "▁cou": 2506, + "ás": 2507, + "ended": 2508, + "ту": 2509, + "ober": 2510, + "▁nothing": 2511, + "▁net": 2512, + "▁pot": 2513, + "▁typ": 2514, + "▁item": 2515, + "rew": 2516, + "Att": 2517, + "▁young": 2518, + "}\r": 2519, + "nder": 2520, + "start": 2521, + "▁Sc": 2522, + "*)": 2523, + "▁enc": 2524, + "▁women": 2525, + "▁looking": 2526, + "▁ро": 2527, + "▁health": 2528, + "Path": 2529, + "▁After": 2530, + "▁mult": 2531, + "▁{\\": 2532, + "▁land": 2533, + "orld": 2534, + "▁Des": 2535, + "▁eng": 2536, + "input": 2537, + "▁Pol": 2538, + "\"\"": 2539, + "Code": 2540, + "▁supp": 2541, + "ainer": 2542, + "heck": 2543, + "▁mor": 2544, + "▁mill": 2545, + "▁aw": 2546, + "fs": 2547, + "▁doing": 2548, + "tings": 2549, + "ades": 2550, + "▁toget": 2551, + "▁certain": 2552, + "▁together": 2553, + "CE": 2554, + "ideo": 2555, + "▁American": 2556, + "ony": 2557, + "idd": 2558, + "II": 2559, + "ged": 2560, + "ables": 2561, + "▁ident": 2562, + "iod": 2563, + "▁parent": 2564, + "For": 2565, + "ambda": 2566, + "ando": 2567, + "=\\": 2568, + "aged": 2569, + "ending": 2570, + "Int": 2571, + "▁possible": 2572, + "▁со": 2573, + "ivity": 2574, + "num": 2575, + "rt": 2576, + "ajor": 2577, + "create": 2578, + "ride": 2579, + "▁knew": 2580, + "bit": 2581, + "itional": 2582, + "▁lik": 2583, + "▁Her": 2584, + "ension": 2585, + "\".": 2586, + "oto": 2587, + "▁exist": 2588, + "aken": 2589, + "▁actually": 2590, + "ca": 2591, + "▁Г": 2592, + "хо": 2593, + "inn": 2594, + "All": 2595, + "buf": 2596, + "▁Me": 2597, + "▁seen": 2598, + "ops": 2599, + "▁▁▁▁▁▁▁▁▁": 2600, + "Not": 2601, + "▁control": 2602, + "▁respon": 2603, + "};": 2604, + "ilt": 2605, + "isk": 2606, + "▁bad": 2607, + "▁often": 2608, + "▁past": 2609, + "aper": 2610, + "▁reason": 2611, + "eters": 2612, + "▁wanted": 2613, + "ura": 2614, + "table": 2615, + "ormal": 2616, + "width": 2617, + "га": 2618, + "ptr": 2619, + "▁dest": 2620, + "▁design": 2621, + "▁sound": 2622, + "▁plan": 2623, + "▁base": 2624, + "hand": 2625, + "gs": 2626, + "▁says": 2627, + "function": 2628, + "▁tri": 2629, + "mt": 2630, + "▁invest": 2631, + "▁available": 2632, + "ayout": 2633, + "▁och": 2634, + "▁las": 2635, + "illed": 2636, + "Val": 2637, + "▁ф": 2638, + "iety": 2639, + "mon": 2640, + "Hand": 2641, + "Fr": 2642, + "iam": 2643, + "pace": 2644, + "▁Ob": 2645, + "▁para": 2646, + "▁meet": 2647, + "▁sum": 2648, + "Message": 2649, + "ici": 2650, + "▁known": 2651, + "▁gen": 2652, + "amma": 2653, + "arr": 2654, + "▁tre": 2655, + "oke": 2656, + "uth": 2657, + "~\\": 2658, + "▁experience": 2659, + "icle": 2660, + "▁Il": 2661, + "▁sent": 2662, + "▁others": 2663, + "▁soft": 2664, + "IP": 2665, + "▁max": 2666, + "ball": 2667, + "▁market": 2668, + "▁pour": 2669, + "pression": 2670, + "eps": 2671, + "▁saw": 2672, + "▁across": 2673, + "▁Su": 2674, + "Over": 2675, + "ние": 2676, + "ulation": 2677, + "▁Reg": 2678, + "▁+=": 2679, + "body": 2680, + ")\\": 2681, + "▁print": 2682, + "▁при": 2683, + "db": 2684, + "ources": 2685, + "wards": 2686, + "▁black": 2687, + "со": 2688, + "ili": 2689, + "▁Ed": 2690, + "▁complet": 2691, + "▁single": 2692, + "▁IN": 2693, + "ached": 2694, + "bt": 2695, + "▁code": 2696, + "▁bool": 2697, + "▁area": 2698, + "▁require": 2699, + "▁problem": 2700, + "aced": 2701, + "Equ": 2702, + "▁config": 2703, + "vec": 2704, + "ney": 2705, + "cy": 2706, + "Al": 2707, + "▁account": 2708, + "ymbol": 2709, + "▁ste": 2710, + "ges": 2711, + "Array": 2712, + "empl": 2713, + "context": 2714, + "Des": 2715, + "Result": 2716, + "ecut": 2717, + "▁target": 2718, + "▁getting": 2719, + "\"/>": 2720, + "ogle": 2721, + "▁himself": 2722, + "▁wasn": 2723, + "▁block": 2724, + "▁ant": 2725, + "▁York": 2726, + "▁become": 2727, + "iff": 2728, + "ports": 2729, + "reate": 2730, + "='": 2731, + "cd": 2732, + "location": 2733, + "ет": 2734, + "▁access": 2735, + "gress": 2736, + "ros": 2737, + "Up": 2738, + "▁working": 2739, + "▁Am": 2740, + "iqu": 2741, + "cer": 2742, + "▁((": 2743, + "▁Per": 2744, + "▁func": 2745, + "▁girl": 2746, + "▁above": 2747, + "pen": 2748, + "пи": 2749, + "ido": 2750, + "▁version": 2751, + "TY": 2752, + "▁;": 2753, + "mary": 2754, + "abled": 2755, + "annel": 2756, + "▁example": 2757, + "▁context": 2758, + "OP": 2759, + "▁red": 2760, + "▁cir": 2761, + "sm": 2762, + "Log": 2763, + "▁space": 2764, + "▁fut": 2765, + "▁Gener": 2766, + "ills": 2767, + "▁dri": 2768, + "_.": 2769, + "▁felt": 2770, + "▁offic": 2771, + "▁===": 2772, + "ii": 2773, + "▁started": 2774, + "▁Т": 2775, + "▁});": 2776, + "js": 2777, + "▁front": 2778, + "▁almost": 2779, + "irm": 2780, + "!\"": 2781, + "signed": 2782, + "▁yet": 2783, + "▁trad": 2784, + "ients": 2785, + "ama": 2786, + "▁input": 2787, + "lim": 2788, + "па": 2789, + "▁ка": 2790, + "▁camp": 2791, + "ibr": 2792, + "fect": 2793, + "unt": 2794, + "▁half": 2795, + "▁cover": 2796, + "anguage": 2797, + "▁ben": 2798, + "ha": 2799, + "▁diff": 2800, + "_\\": 2801, + "▁об": 2802, + "])": 2803, + "odes": 2804, + "hel": 2805, + "ios": 2806, + "▁О": 2807, + "▁mot": 2808, + "▁social": 2809, + "////////": 2810, + "▁stre": 2811, + "ground": 2812, + "ів": 2813, + "object": 2814, + "ples": 2815, + "reed": 2816, + "▁een": 2817, + "▁based": 2818, + "▁range": 2819, + "An": 2820, + "urg": 2821, + "▁learn": 2822, + "▁exc": 2823, + "▁imp": 2824, + "▁means": 2825, + "▁wur": 2826, + "ends": 2827, + "void": 2828, + "▁std": 2829, + "▁particular": 2830, + "ja": 2831, + "▁source": 2832, + "default": 2833, + "py": 2834, + "▁als": 2835, + "scri": 2836, + "status": 2837, + "▁story": 2838, + "▁begin": 2839, + "▁position": 2840, + "▁special": 2841, + "php": 2842, + "▁bar": 2843, + "▁pract": 2844, + "call": 2845, + "▁das": 2846, + "▁rad": 2847, + "▁close": 2848, + "www": 2849, + "ере": 2850, + "gu": 2851, + "▁Er": 2852, + "▁dom": 2853, + "AM": 2854, + "▁bed": 2855, + "▁several": 2856, + "aul": 2857, + "box": 2858, + "▁low": 2859, + "pack": 2860, + "Reg": 2861, + "Of": 2862, + "atures": 2863, + "én": 2864, + "eder": 2865, + "uilder": 2866, + "cast": 2867, + "conom": 2868, + "raft": 2869, + "▁makes": 2870, + "Loc": 2871, + "http": 2872, + "▁abs": 2873, + "resh": 2874, + "▁Will": 2875, + "break": 2876, + "▁options": 2877, + "fort": 2878, + "▁из": 2879, + "▁anal": 2880, + "▁env": 2881, + "({": 2882, + "event": 2883, + "▁page": 2884, + "ternal": 2885, + "▁distribut": 2886, + "▁food": 2887, + "check": 2888, + "CK": 2889, + "▁во": 2890, + "assert": 2891, + "án": 2892, + "base": 2893, + "▁whole": 2894, + "ación": 2895, + "OD": 2896, + "▁turned": 2897, + "igma": 2898, + "▁response": 2899, + "▁University": 2900, + "▁div": 2901, + "apter": 2902, + "▁results": 2903, + "▁represent": 2904, + "▁everything": 2905, + "▁Cent": 2906, + "utes": 2907, + "rix": 2908, + "▁Some": 2909, + "▁behind": 2910, + "▁creat": 2911, + "place": 2912, + "su": 2913, + "▁Part": 2914, + "umb": 2915, + "mathbb": 2916, + "ping": 2917, + "▁match": 2918, + "Out": 2919, + "dom": 2920, + "▁situ": 2921, + "dr": 2922, + "ara": 2923, + "▁window": 2924, + "ns": 2925, + "lished": 2926, + "▁Ver": 2927, + "▁message": 2928, + "▁Em": 2929, + "▁human": 2930, + "perties": 2931, + "лу": 2932, + "lem": 2933, + "ORT": 2934, + "▁early": 2935, + "▁quick": 2936, + "▁та": 2937, + "roid": 2938, + "▁country": 2939, + "▁due": 2940, + "▁Die": 2941, + "▁trying": 2942, + "▁live": 2943, + "▁press": 2944, + "INT": 2945, + "With": 2946, + "oved": 2947, + "▁specific": 2948, + "▁fall": 2949, + "uk": 2950, + "yl": 2951, + "▁general": 2952, + "му": 2953, + "ну": 2954, + "▁names": 2955, + "where": 2956, + "▁These": 2957, + "▁sil": 2958, + "ét": 2959, + "▁ener": 2960, + "▁Now": 2961, + "▁address": 2962, + "Response": 2963, + "▁Mr": 2964, + "▁answ": 2965, + "▁film": 2966, + "▁strong": 2967, + "▁bring": 2968, + "▁United": 2969, + "▁ge": 2970, + "▁woman": 2971, + "New": 2972, + "ett": 2973, + ".)": 2974, + "ename": 2975, + "▁AN": 2976, + "▁describ": 2977, + "за": 2978, + "ising": 2979, + "EL": 2980, + "ql": 2981, + "▁fur": 2982, + "ying": 2983, + "▁Cal": 2984, + "▁Dr": 2985, + "ERR": 2986, + "▁\\\\": 2987, + "angle": 2988, + "urope": 2989, + "▁city": 2990, + "▁index": 2991, + "▁action": 2992, + "▁However": 2993, + "▁fig": 2994, + "ias": 2995, + "▁question": 2996, + "▁Jan": 2997, + "▁Med": 2998, + "▁Cont": 2999, + "amed": 3000, + "Call": 3001, + "plied": 3002, + "tty": 3003, + "▁individ": 3004, + "page": 3005, + "▁comb": 3006, + "section": 3007, + "▁Comm": 3008, + "uel": 3009, + "▁het": 3010, + "▁Bar": 3011, + "agement": 3012, + "fin": 3013, + "▁major": 3014, + "oper": 3015, + "api": 3016, + "room": 3017, + "▁„": 3018, + "▁hab": 3019, + "зи": 3020, + "▁auf": 3021, + "current": 3022, + "ni": 3023, + "▁include": 3024, + "▁qui": 3025, + "va": 3026, + "UE": 3027, + "▁idea": 3028, + ",'": 3029, + "▁required": 3030, + "▁heart": 3031, + "ibility": 3032, + "iction": 3033, + "Model": 3034, + "write": 3035, + "▁content": 3036, + "▁wer": 3037, + "▁hands": 3038, + "zen": 3039, + "char": 3040, + "}^{": 3041, + "▁mass": 3042, + "ply": 3043, + "▁nat": 3044, + "rel": 3045, + "▁dat": 3046, + "================": 3047, + "imal": 3048, + "▁probably": 3049, + "unch": 3050, + "▁mer": 3051, + "ilar": 3052, + "ires": 3053, + "▁watch": 3054, + "SI": 3055, + "▁cult": 3056, + "▁mother": 3057, + "▁government": 3058, + "ording": 3059, + "▁()": 3060, + "▁pri": 3061, + "▁link": 3062, + "group": 3063, + "OL": 3064, + "▁near": 3065, + "▁Ser": 3066, + "Ser": 3067, + "ito": 3068, + "▁values": 3069, + "▁java": 3070, + "fully": 3071, + "Count": 3072, + "++)": 3073, + "▁vi": 3074, + "▁white": 3075, + "mat": 3076, + "ctx": 3077, + "▁conc": 3078, + "▁stay": 3079, + "ging": 3080, + "▁clear": 3081, + "▁copy": 3082, + "selves": 3083, + "▁provide": 3084, + "▁words": 3085, + "comp": 3086, + "args": 3087, + "▁pick": 3088, + "uly": 3089, + "▁vari": 3090, + "▁believe": 3091, + "▁Co": 3092, + "Property": 3093, + "Group": 3094, + "▁ten": 3095, + "ischen": 3096, + "eturn": 3097, + "ival": 3098, + "System": 3099, + "CL": 3100, + "bed": 3101, + "▁total": 3102, + "▁ist": 3103, + "Input": 3104, + "uments": 3105, + "Manager": 3106, + "ши": 3107, + "▁win": 3108, + "leep": 3109, + "PI": 3110, + "ного": 3111, + "ruction": 3112, + "▁inte": 3113, + "App": 3114, + "avor": 3115, + "▁respect": 3116, + "ators": 3117, + "▁como": 3118, + "▁cut": 3119, + "FA": 3120, + "▁sus": 3121, + "▁App": 3122, + "rect": 3123, + "FI": 3124, + "▁began": 3125, + "oph": 3126, + "▁sort": 3127, + "though": 3128, + "је": 3129, + "icro": 3130, + "Trans": 3131, + "лі": 3132, + "▁Inst": 3133, + "request": 3134, + "ор": 3135, + "▁relations": 3136, + "-\\": 3137, + "Status": 3138, + "жи": 3139, + "▁father": 3140, + "cs": 3141, + "▁sex": 3142, + "isch": 3143, + "vo": 3144, + "}_{": 3145, + "aven": 3146, + "▁Ne": 3147, + "ATE": 3148, + "itten": 3149, + "▁ess": 3150, + "TH": 3151, + "ights": 3152, + "▁hom": 3153, + "▁today": 3154, + "▁zu": 3155, + "ita": 3156, + "▁isn": 3157, + "▁opt": 3158, + "ogn": 3159, + "ér": 3160, + "▁whether": 3161, + "ixed": 3162, + "phi": 3163, + "idence": 3164, + "ald": 3165, + "Client": 3166, + "At": 3167, + "▁death": 3168, + "▁Let": 3169, + "ius": 3170, + "ги": 3171, + "▁ре": 3172, + "ben": 3173, + ")\r": 3174, + "ba": 3175, + ">": 3193, + "▁Just": 3194, + "What": 3195, + "atal": 3196, + "▁Min": 3197, + "▁Cor": 3198, + "▁dark": 3199, + "rl": 3200, + "▁larg": 3201, + "ding": 3202, + "ón": 3203, + "ouch": 3204, + "▁um": 3205, + "▁elect": 3206, + "▁dam": 3207, + "▁needs": 3208, + "▁matter": 3209, + "▁rather": 3210, + "from": 3211, + "ram": 3212, + "▁і": 3213, + "▁taken": 3214, + "▁deal": 3215, + "▁period": 3216, + "▁Mon": 3217, + "▁Л": 3218, + "▁Aug": 3219, + "run": 3220, + "mm": 3221, + "elle": 3222, + "▁export": 3223, + "Sc": 3224, + "vis": 3225, + "abor": 3226, + "▁author": 3227, + "ère": 3228, + "▁remember": 3229, + "▁redu": 3230, + "▁List": 3231, + "▁focus": 3232, + "▁character": 3233, + "Table": 3234, + "▁individual": 3235, + "▁needed": 3236, + "bum": 3237, + "▁style": 3238, + "inary": 3239, + "ersion": 3240, + "oute": 3241, + "▁Pe": 3242, + "▁hon": 3243, + "mut": 3244, + "see": 3245, + "▁became": 3246, + "▁dire": 3247, + "▁document": 3248, + "sec": 3249, + "ening": 3250, + "▁visit": 3251, + "▁fac": 3252, + "tx": 3253, + "down": 3254, + "plit": 3255, + "▁phys": 3256, + "itting": 3257, + "joy": 3258, + "▁hig": 3259, + "This": 3260, + "Ad": 3261, + "▁Brit": 3262, + "▁employ": 3263, + "▁ré": 3264, + "▁т": 3265, + "lambda": 3266, + "▁impro": 3267, + "▁Bo": 3268, + "iding": 3269, + "▁online": 3270, + "mem": 3271, + "atform": 3272, + "▁War": 3273, + "▁cas": 3274, + "asure": 3275, + "▁pur": 3276, + "medi": 3277, + "Dis": 3278, + "▁Germ": 3279, + "pc": 3280, + "са": 3281, + "▁friends": 3282, + "▁Mc": 3283, + "DI": 3284, + "▁plus": 3285, + "▁Set": 3286, + "iddle": 3287, + "itut": 3288, + "▁depend": 3289, + "rest": 3290, + "▁Je": 3291, + "▁hor": 3292, + "▁entire": 3293, + "Query": 3294, + "▁refer": 3295, + "▁hot": 3296, + "▁Aust": 3297, + "▁common": 3298, + "ці": 3299, + "▁pull": 3300, + "▁Add": 3301, + "▁season": 3302, + "▁invol": 3303, + "▁World": 3304, + "client": 3305, + "now": 3306, + "true": 3307, + "append": 3308, + "itted": 3309, + "empt": 3310, + "){": 3311, + "///": 3312, + "▁prop": 3313, + "imate": 3314, + "SC": 3315, + "▁hours": 3316, + "▁hope": 3317, + "andom": 3318, + "ід": 3319, + "istic": 3320, + "▁property": 3321, + "sg": 3322, + ">(": 3323, + "▁write": 3324, + "mark": 3325, + "find": 3326, + "▁personal": 3327, + "][": 3328, + "rown": 3329, + "Ph": 3330, + "▁foot": 3331, + "▁research": 3332, + "ironment": 3333, + "▁nom": 3334, + "▁instance": 3335, + "▁held": 3336, + "De": 3337, + "▁members": 3338, + "▁fire": 3339, + "▁history": 3340, + "▁map": 3341, + "▁discuss": 3342, + "▁espec": 3343, + "▁taking": 3344, + "▁services": 3345, + "▁indust": 3346, + "igen": 3347, + "▁Ass": 3348, + "▁expected": 3349, + "▁wurde": 3350, + "dir": 3351, + "▁among": 3352, + "▁sugg": 3353, + "rec": 3354, + "Inter": 3355, + "block": 3356, + "▁Rep": 3357, + "▁pain": 3358, + "▁five": 3359, + "▁fund": 3360, + "rid": 3361, + "arrow": 3362, + "▁treat": 3363, + "▁heard": 3364, + "▁determ": 3365, + "icult": 3366, + "▁sense": 3367, + "ese": 3368, + "Fun": 3369, + "▁months": 3370, + "json": 3371, + ",”": 3372, + "TI": 3373, + "orage": 3374, + "▁У": 3375, + "▁everyone": 3376, + "▁clos": 3377, + "iers": 3378, + "airs": 3379, + "define": 3380, + "If": 3381, + "osp": 3382, + "▁wonder": 3383, + "NA": 3384, + "query": 3385, + "pg": 3386, + "ites": 3387, + "▁material": 3388, + "yd": 3389, + "Read": 3390, + "html": 3391, + "TE": 3392, + "Pr": 3393, + "^{\\": 3394, + "▁gave": 3395, + "▁IS": 3396, + "▁suggest": 3397, + "Override": 3398, + "rodu": 3399, + "From": 3400, + "▁Europe": 3401, + "PO": 3402, + "▁soon": 3403, + "host": 3404, + "▁Ber": 3405, + "....": 3406, + "▁Har": 3407, + "▁energy": 3408, + "><": 3409, + "aves": 3410, + "▁easy": 3411, + "▁bre": 3412, + "frame": 3413, + "▁ground": 3414, + "with": 3415, + "▁inside": 3416, + "ief": 3417, + "▁mo": 3418, + "pm": 3419, + "pan": 3420, + "igr": 3421, + "▁om": 3422, + "next": 3423, + "omet": 3424, + "▁status": 3425, + "▁}\r": 3426, + "▁music": 3427, + "ora": 3428, + "iles": 3429, + "ki": 3430, + "▁esc": 3431, + "▁bes": 3432, + "▁Dis": 3433, + "▁host": 3434, + "▁comes": 3435, + "used": 3436, + "▁future": 3437, + "lick": 3438, + "aid": 3439, + "▁compet": 3440, + "▁voice": 3441, + "▁load": 3442, + "evel": 3443, + "▁neg": 3444, + "▁command": 3445, + "▁für": 3446, + "▁pie": 3447, + "▁quite": 3448, + "▁blo": 3449, + "agn": 3450, + "ilon": 3451, + "▁claim": 3452, + "▁teach": 3453, + "▁previous": 3454, + "▁site": 3455, + "color": 3456, + "attr": 3457, + "▁accept": 3458, + "▁exact": 3459, + ")}": 3460, + "aft": 3461, + "roller": 3462, + "он": 3463, + "oo": 3464, + "Date": 3465, + "▁ou": 3466, + "sy": 3467, + "▁pretty": 3468, + "▁image": 3469, + "BU": 3470, + "▁terms": 3471, + "▁search": 3472, + "▁è": 3473, + "▁Val": 3474, + "▁‘": 3475, + "▁Dav": 3476, + "MS": 3477, + "src": 3478, + "mar": 3479, + "incip": 3480, + "▁couldn": 3481, + "ados": 3482, + "▁dro": 3483, + "beta": 3484, + "imum": 3485, + "▁minutes": 3486, + "▁grand": 3487, + "▁»": 3488, + "▁Our": 3489, + "Str": 3490, + "VER": 3491, + "maz": 3492, + "▁original": 3493, + "ini": 3494, + "▁coll": 3495, + "loat": 3496, + "▁os": 3497, + "});": 3498, + "summary": 3499, + "▁wall": 3500, + "Color": 3501, + "▁vers": 3502, + "▁della": 3503, + "▁\"\"\"": 3504, + "mathbf": 3505, + "zer": 3506, + "aur": 3507, + "▁track": 3508, + "▁associ": 3509, + "▁suff": 3510, + "▁inde": 3511, + "ague": 3512, + "▁Apr": 3513, + "Le": 3514, + "roups": 3515, + "board": 3516, + "▁attack": 3517, + "▁series": 3518, + "▁instead": 3519, + "ham": 3520, + "book": 3521, + "▁six": 3522, + "▁Rec": 3523, + "▁coming": 3524, + "urt": 3525, + "▁global": 3526, + "▁necess": 3527, + "lege": 3528, + "Pos": 3529, + "▁leave": 3530, + "▁pod": 3531, + "ategory": 3532, + "uz": 3533, + "▁deep": 3534, + "▁km": 3535, + "▁outside": 3536, + "has": 3537, + "options": 3538, + "▁Sm": 3539, + "Sub": 3540, + "rows": 3541, + "▁ви": 3542, + "▁States": 3543, + "▁wrong": 3544, + "▁however": 3545, + "▁sem": 3546, + "▁catch": 3547, + "\"),": 3548, + "model": 3549, + "▁http": 3550, + "▁option": 3551, + "rie": 3552, + "▁ста": 3553, + "▁är": 3554, + "▁enjoy": 3555, + "nu": 3556, + "▁pas": 3557, + "▁amount": 3558, + "▁respons": 3559, + "▁Intern": 3560, + "▁myself": 3561, + "▁opp": 3562, + "▁Sim": 3563, + "▁sens": 3564, + "Ed": 3565, + "▁(\\": 3566, + "▁students": 3567, + "нов": 3568, + "▁points": 3569, + "arning": 3570, + "UP": 3571, + "elling": 3572, + "▁cannot": 3573, + "Be": 3574, + "▁length": 3575, + "null": 3576, + "uint": 3577, + "wise": 3578, + "▁double": 3579, + "ige": 3580, + "ista": 3581, + "▁estab": 3582, + "anch": 3583, + "▁ago": 3584, + "▁bound": 3585, + "▁fa": 3586, + "▁clean": 3587, + "▁simple": 3588, + "mi": 3589, + "########": 3590, + "ifier": 3591, + "▁General": 3592, + "▁seemed": 3593, + "ena": 3594, + "▁age": 3595, + "ной": 3596, + "endif": 3597, + "AA": 3598, + "▁caus": 3599, + "▁educ": 3600, + "▁cell": 3601, + "Gener": 3602, + "space": 3603, + "▁Your": 3604, + "▁beaut": 3605, + "gt": 3606, + "▁limit": 3607, + "▁date": 3608, + "Util": 3609, + "▁National": 3610, + "ows": 3611, + "pat": 3612, + "quad": 3613, + "▁ok": 3614, + "▁И": 3615, + "arth": 3616, + "hat": 3617, + "▁community": 3618, + "oul": 3619, + "▁econom": 3620, + "Component": 3621, + "bor": 3622, + "usion": 3623, + "▁below": 3624, + "earch": 3625, + "ores": 3626, + "ban": 3627, + "▁August": 3628, + "▁further": 3629, + "sigma": 3630, + "▁ha": 3631, + "ji": 3632, + "▁comput": 3633, + "гра": 3634, + "▁None": 3635, + "▁ter": 3636, + "▁anyone": 3637, + "▁task": 3638, + "ente": 3639, + "position": 3640, + "pped": 3641, + "▁aus": 3642, + "Attribute": 3643, + "req": 3644, + "addr": 3645, + "light": 3646, + "ше": 3647, + "▁arm": 3648, + "cover": 3649, + "upport": 3650, + "▁Gl": 3651, + "▁San": 3652, + "▁writing": 3653, + "▁lost": 3654, + "▁Mark": 3655, + "▁gre": 3656, + "TYPE": 3657, + "▁South": 3658, + "▁perfect": 3659, + "▁package": 3660, + "▁infl": 3661, + "haps": 3662, + "▁Ang": 3663, + "respon": 3664, + "ris": 3665, + "ptember": 3666, + "▁building": 3667, + "VAL": 3668, + "free": 3669, + "▁ce": 3670, + "HT": 3671, + "▁From": 3672, + "ds": 3673, + "roy": 3674, + "achine": 3675, + "nown": 3676, + "▁saying": 3677, + "▁бы": 3678, + "oe": 3679, + "Ref": 3680, + "▁network": 3681, + "parent": 3682, + "uge": 3683, + "▁similar": 3684, + ">\r": 3685, + "Builder": 3686, + "▁living": 3687, + "▁continue": 3688, + "anger": 3689, + "▁Red": 3690, + "▁hair": 3691, + "anced": 3692, + "ians": 3693, + "▁dead": 3694, + "▁boolean": 3695, + "ication": 3696, + "▁де": 3697, + "▁client": 3698, + "uct": 3699, + "▁•": 3700, + "SP": 3701, + "older": 3702, + "пе": 3703, + "udio": 3704, + "▁deg": 3705, + "asing": 3706, + "▁step": 3707, + "▁pers": 3708, + "ção": 3709, + "obj": 3710, + "oz": 3711, + "ula": 3712, + "▁round": 3713, + "▁upon": 3714, + "▁resource": 3715, + "▁valid": 3716, + "▁II": 3717, + "bug": 3718, + "std": 3719, + "▁ang": 3720, + "span": 3721, + "pol": 3722, + "ialog": 3723, + "▁phot": 3724, + "?'": 3725, + "DB": 3726, + "▁Fin": 3727, + "VE": 3728, + "Em": 3729, + "▁cam": 3730, + "target": 3731, + "pected": 3732, + "Hel": 3733, + "▁ut": 3734, + "▁Test": 3735, + "▁town": 3736, + "align": 3737, + "▁webs": 3738, + "inner": 3739, + "augh": 3740, + "▁except": 3741, + "▁initial": 3742, + "enty": 3743, + "lich": 3744, + "▁Aut": 3745, + "top": 3746, + "▁fail": 3747, + "ona": 3748, + "▁benef": 3749, + "anks": 3750, + "ische": 3751, + ".*": 3752, + "▁signific": 3753, + "▁contact": 3754, + "Rec": 3755, + "ario": 3756, + "ottom": 3757, + "▁relationship": 3758, + "]);": 3759, + "▁На": 3760, + "Head": 3761, + "format": 3762, + "▁ét": 3763, + "▁More": 3764, + "actory": 3765, + "portun": 3766, + "+\\": 3767, + "▁simply": 3768, + "▁ep": 3769, + "▁Russ": 3770, + "ní": 3771, + "ua": 3772, + "erc": 3773, + "▁longer": 3774, + "inition": 3775, + "ector": 3776, + "aption": 3777, + "▁profess": 3778, + "▁Mus": 3779, + "ilities": 3780, + "ès": 3781, + "▁Act": 3782, + "offset": 3783, + "▁ill": 3784, + "band": 3785, + "▁Ag": 3786, + "▁По": 3787, + "би": 3788, + "content": 3789, + "icon": 3790, + "▁works": 3791, + "ynam": 3792, + "plement": 3793, + "Resource": 3794, + "Action": 3795, + "▁difficult": 3796, + "▁West": 3797, + "▁video": 3798, + "▁THE": 3799, + "▁decl": 3800, + "ondon": 3801, + "ded": 3802, + "}{\\": 3803, + "ocr": 3804, + "▁City": 3805, + "▁я": 3806, + "uer": 3807, + "cz": 3808, + "▁imag": 3809, + "cr": 3810, + "ete": 3811, + "idget": 3812, + "▁Mod": 3813, + "▁forward": 3814, + "▁pict": 3815, + "orge": 3816, + "▁subject": 3817, + "update": 3818, + "attle": 3819, + "sa": 3820, + "▁Ant": 3821, + "▁running": 3822, + "▁sal": 3823, + "conne": 3824, + "▁output": 3825, + "adata": 3826, + "ML": 3827, + "Check": 3828, + "ledge": 3829, + "▁paper": 3830, + "params": 3831, + "avy": 3832, + "▁af": 3833, + "▁eine": 3834, + "▁jour": 3835, + "AY": 3836, + "▁itself": 3837, + "▁Str": 3838, + "style": 3839, + "That": 3840, + "▁million": 3841, + "▁language": 3842, + "OS": 3843, + "ving": 3844, + "▁ма": 3845, + "▁то": 3846, + ")(": 3847, + "▁buy": 3848, + "./": 3849, + "▁...": 3850, + "▁tried": 3851, + "▁compl": 3852, + "▁activ": 3853, + "apped": 3854, + "Button": 3855, + "Token": 3856, + "▁provided": 3857, + "iber": 3858, + "▁created": 3859, + "curity": 3860, + "End": 3861, + "ał": 3862, + "uster": 3863, + "izing": 3864, + "omb": 3865, + "▁sich": 3866, + "▁compon": 3867, + "▁See": 3868, + "▁uint": 3869, + "▁label": 3870, + "vol": 3871, + "ów": 3872, + "ocol": 3873, + "▁received": 3874, + "▁intern": 3875, + "це": 3876, + "Run": 3877, + "▁road": 3878, + "▁Oct": 3879, + "▁Comp": 3880, + "▁study": 3881, + "▁те": 3882, + "Act": 3883, + "▁tour": 3884, + "▁State": 3885, + "▁added": 3886, + "https": 3887, + "stream": 3888, + "▁lower": 3889, + "▁box": 3890, + "▁Sk": 3891, + "▁themselves": 3892, + "▁cross": 3893, + "▁echo": 3894, + "▁device": 3895, + "pose": 3896, + "▁games": 3897, + "PL": 3898, + "Window": 3899, + "ises": 3900, + "title": 3901, + "Stream": 3902, + "zt": 3903, + "▁Sw": 3904, + "▁role": 3905, + "iant": 3906, + "ku": 3907, + "sequ": 3908, + "▁late": 3909, + "▁sold": 3910, + "ря": 3911, + "Comm": 3912, + "▁entre": 3913, + "▁dog": 3914, + "device": 3915, + "Par": 3916, + "▁likely": 3917, + "^{-": 3918, + "▁len": 3919, + "▁Paul": 3920, + "▁tool": 3921, + "Off": 3922, + "▁famil": 3923, + "▁draw": 3924, + "apping": 3925, + "▁events": 3926, + "cret": 3927, + "rought": 3928, + "Content": 3929, + "▁software": 3930, + "ria": 3931, + "msg": 3932, + "gamma": 3933, + "▁hear": 3934, + "Oper": 3935, + "▁yourself": 3936, + "▁liter": 3937, + "emp": 3938, + "▁separ": 3939, + "▁З": 3940, + "▁title": 3941, + "Method": 3942, + "mathrm": 3943, + "▁slow": 3944, + "▁Rom": 3945, + "!!": 3946, + "▁tax": 3947, + "ска": 3948, + "emplate": 3949, + "oi": 3950, + "▁Art": 3951, + "false": 3952, + "astic": 3953, + "сть": 3954, + "ocket": 3955, + "▁ens": 3956, + "TO": 3957, + "amente": 3958, + "local": 3959, + "chie": 3960, + "▁pan": 3961, + "ний": 3962, + "chema": 3963, + "▁North": 3964, + "зо": 3965, + "▁>=": 3966, + "Aut": 3967, + "▁dig": 3968, + "▁seems": 3969, + "▁morning": 3970, + "sole": 3971, + "umer": 3972, + "delta": 3973, + "ité": 3974, + "abase": 3975, + "raf": 3976, + "▁observ": 3977, + "▁Est": 3978, + "▁seg": 3979, + "▁[]": 3980, + "▁Pres": 3981, + "iful": 3982, + "push": 3983, + "▁Off": 3984, + "ipe": 3985, + "ati": 3986, + "▁dim": 3987, + "ceed": 3988, + "Ent": 3989, + "____": 3990, + "entry": 3991, + "▁fight": 3992, + "▁cred": 3993, + "▁OR": 3994, + "▁Dep": 3995, + "${": 3996, + "лен": 3997, + "Create": 3998, + "▁April": 3999, + "ministr": 4000, + "FL": 4001, + "▁Ap": 4002, + "▁Here": 4003, + "private": 4004, + "Instance": 4005, + "iem": 4006, + "▁office": 4007, + "▁third": 4008, + "▁update": 4009, + "Line": 4010, + "tag": 4011, + "▁especially": 4012, + "▁года": 4013, + "▁cu": 4014, + "▁kill": 4015, + "aught": 4016, + "▁swe": 4017, + "Options": 4018, + "IM": 4019, + "CC": 4020, + "▁compan": 4021, + "just": 4022, + "▁While": 4023, + "izer": 4024, + "▁мо": 4025, + "ке": 4026, + "▁auto": 4027, + "▁band": 4028, + "мен": 4029, + "iques": 4030, + "▁ple": 4031, + "NO": 4032, + "▁OF": 4033, + "▁song": 4034, + "▁Acc": 4035, + "EXT": 4036, + "ensor": 4037, + "ining": 4038, + "▁lat": 4039, + "big": 4040, + "▁King": 4041, + "och": 4042, + "si": 4043, + "▁Hist": 4044, + "▁quality": 4045, + "mode": 4046, + "▁opportun": 4047, + "▁wouldn": 4048, + ":**": 4049, + "output": 4050, + "▁feet": 4051, + "▁mis": 4052, + "df": 4053, + "aging": 4054, + "▁ме": 4055, + "▁tro": 4056, + "▁defined": 4057, + "▁review": 4058, + "▁Fil": 4059, + ">>": 4060, + "▁princip": 4061, + "Base": 4062, + "dict": 4063, + "verage": 4064, + "icient": 4065, + "IF": 4066, + "▁hit": 4067, + "Page": 4068, + "▁perm": 4069, + "cel": 4070, + "ít": 4071, + "▁express": 4072, + "▁indic": 4073, + "▁September": 4074, + "image": 4075, + "▁products": 4076, + "▁media": 4077, + "change": 4078, + "igger": 4079, + "▁send": 4080, + "last": 4081, + "ming": 4082, + "pa": 4083, + "uary": 4084, + "▁speak": 4085, + "ный": 4086, + "ще": 4087, + "ysis": 4088, + "lying": 4089, + "▁ч": 4090, + "like": 4091, + "ры": 4092, + "ві": 4093, + "▁Mich": 4094, + "MO": 4095, + "▁Jah": 4096, + "ensive": 4097, + "▁share": 4098, + "▁development": 4099, + "CP": 4100, + "spec": 4101, + "▁fast": 4102, + "het": 4103, + "HO": 4104, + "▁particip": 4105, + "Block": 4106, + "▁viol": 4107, + "▁frame": 4108, + "▁qual": 4109, + "tre": 4110, + "▁Ф": 4111, + "▁toward": 4112, + "fg": 4113, + "Box": 4114, + "Column": 4115, + "▁milit": 4116, + "▁March": 4117, + "▁various": 4118, + "pass": 4119, + "▁Park": 4120, + "▁Ben": 4121, + "Frame": 4122, + "▁normal": 4123, + "open": 4124, + "px": 4125, + "▁phone": 4126, + "▁Even": 4127, + "▁ma": 4128, + "ibrary": 4129, + "Start": 4130, + "idden": 4131, + "rho": 4132, + "graph": 4133, + "acing": 4134, + "'.": 4135, + "arter": 4136, + "mes": 4137, + "inst": 4138, + "▁ir": 4139, + "active": 4140, + "▁fem": 4141, + "▁moved": 4142, + "▁store": 4143, + "▁price": 4144, + "\").": 4145, + "berg": 4146, + "▁nov": 4147, + "▁card": 4148, + "ellow": 4149, + "▁party": 4150, + "▁Mor": 4151, + "ael": 4152, + "▁percent": 4153, + "▁training": 4154, + "▁ing": 4155, + "imer": 4156, + "▁Sam": 4157, + "Default": 4158, + "▁fuck": 4159, + "▁complete": 4160, + "uid": 4161, + "▁details": 4162, + "▁led": 4163, + "Point": 4164, + "▁Count": 4165, + "▁regard": 4166, + "zo": 4167, + "▁Bro": 4168, + "▁recogn": 4169, + "▁Hol": 4170, + "UM": 4171, + "element": 4172, + "Mode": 4173, + "▁exam": 4174, + "▁EX": 4175, + "Image": 4176, + "verse": 4177, + "riter": 4178, + "soft": 4179, + "▁introdu": 4180, + "▁surpr": 4181, + "Buffer": 4182, + "lector": 4183, + "aren": 4184, + "anged": 4185, + "▁Pat": 4186, + "▁Pal": 4187, + "▁contr": 4188, + "Handler": 4189, + "▁features": 4190, + "iple": 4191, + "▁CON": 4192, + "Fil": 4193, + "▁Port": 4194, + "▁thinking": 4195, + "doc": 4196, + "wer": 4197, + "▁worked": 4198, + "PC": 4199, + "cm": 4200, + "dat": 4201, + "PRO": 4202, + "▁Every": 4203, + "▁era": 4204, + "▁First": 4205, + "gn": 4206, + "▁immedi": 4207, + "ovember": 4208, + "apan": 4209, + "▁extra": 4210, + "▁section": 4211, + "▁June": 4212, + "▁via": 4213, + "▁gone": 4214, + "come": 4215, + "▁stri": 4216, + "^\\": 4217, + "antly": 4218, + "▁arch": 4219, + "Source": 4220, + "▁conv": 4221, + "▁London": 4222, + "Number": 4223, + "▁questions": 4224, + "andid": 4225, + "▁played": 4226, + "env": 4227, + "▁School": 4228, + "▁natural": 4229, + "can": 4230, + "▁news": 4231, + "DR": 4232, + "▁chall": 4233, + "▁Soc": 4234, + "▁э": 4235, + "▁attempt": 4236, + "*}": 4237, + "Null": 4238, + "rote": 4239, + "▁bi": 4240, + "▁written": 4241, + "▁blood": 4242, + "▁happened": 4243, + "▁cause": 4244, + "ashing": 4245, + "▁William": 4246, + "adem": 4247, + "▁brought": 4248, + "▁display": 4249, + "ima": 4250, + "▁finally": 4251, + "tab": 4252, + "▁returned": 4253, + "ных": 4254, + "nie": 4255, + "▁q": 4256, + "▁hers": 4257, + "▁Pre": 4258, + "▁dou": 4259, + "buffer": 4260, + "▁effort": 4261, + "aine": 4262, + "xy": 4263, + "▁histor": 4264, + "enu": 4265, + "▁arriv": 4266, + "▁Dem": 4267, + "▁favor": 4268, + "▁handle": 4269, + "SET": 4270, + "▁Public": 4271, + "rupt": 4272, + "▁ur": 4273, + "▁force": 4274, + "▁és": 4275, + "ube": 4276, + "Pre": 4277, + "рі": 4278, + "iny": 4279, + "theta": 4280, + "isf": 4281, + "▁national": 4282, + "Equal": 4283, + "rench": 4284, + "▁wife": 4285, + "▁capt": 4286, + "▁Inter": 4287, + "tau": 4288, + "▁sleep": 4289, + "../../": 4290, + "▁issue": 4291, + "▁member": 4292, + "▁await": 4293, + "▁Dan": 4294, + "zi": 4295, + "inate": 4296, + "▁sym": 4297, + "chan": 4298, + "▁Jack": 4299, + "▁English": 4300, + "▁sz": 4301, + "ributes": 4302, + "▁ign": 4303, + "ál": 4304, + "▁appear": 4305, + "rad": 4306, + "idge": 4307, + "▁couple": 4308, + "▁ship": 4309, + "lig": 4310, + "web": 4311, + "▁usually": 4312, + "▁ready": 4313, + "▁vill": 4314, + "▁Why": 4315, + "ebru": 4316, + "▁grad": 4317, + "ords": 4318, + "▁inf": 4319, + "▁loss": 4320, + "▁od": 4321, + "▁Phil": 4322, + "server": 4323, + "▁Up": 4324, + "▁buff": 4325, + "▁filename": 4326, + "ABLE": 4327, + "iting": 4328, + "efore": 4329, + "()->": 4330, + "▁conditions": 4331, + "vm": 4332, + "eld": 4333, + "itz": 4334, + "▁Trans": 4335, + "▁weight": 4336, + "▁higher": 4337, + "▁rate": 4338, + "▁accom": 4339, + "vider": 4340, + "OM": 4341, + "▁ways": 4342, + "coming": 4343, + "▁lock": 4344, + "▁etc": 4345, + "▁avec": 4346, + "▁takes": 4347, + "▁Char": 4348, + "▁November": 4349, + "method": 4350, + "▁Austral": 4351, + "▁America": 4352, + "long": 4353, + "cember": 4354, + "▁political": 4355, + "flow": 4356, + "▁maybe": 4357, + "▁amb": 4358, + "Layout": 4359, + "iled": 4360, + "omen": 4361, + "ola": 4362, + "icip": 4363, + "partial": 4364, + "True": 4365, + "▁floor": 4366, + "▁Def": 4367, + "▁concern": 4368, + "yr": 4369, + "▁shows": 4370, + "ih": 4371, + "▁answer": 4372, + "acc": 4373, + "▁ball": 4374, + "▁Rev": 4375, + "▁sun": 4376, + "▁quickly": 4377, + "▁somet": 4378, + "mente": 4379, + "▁Mal": 4380, + "undred": 4381, + "▁issues": 4382, + "ecause": 4383, + "pes": 4384, + "▁player": 4385, + "▁parents": 4386, + "▁popular": 4387, + "▁mode": 4388, + "▁mention": 4389, + "NE": 4390, + "Load": 4391, + "▁regular": 4392, + "aved": 4393, + "?:": 4394, + "year": 4395, + "func": 4396, + "▁performance": 4397, + "▁July": 4398, + "thern": 4399, + "▁website": 4400, + "ford": 4401, + "PR": 4402, + "ela": 4403, + "level": 4404, + "uit": 4405, + "flags": 4406, + "▁worth": 4407, + "▁correspon": 4408, + "▁British": 4409, + "sim": 4410, + "▁alone": 4411, + "▁har": 4412, + "▁ones": 4413, + "obile": 4414, + "▁dru": 4415, + "chi": 4416, + "▁David": 4417, + "▁problems": 4418, + "▁column": 4419, + "();\r": 4420, + "ZE": 4421, + "▁relig": 4422, + "ological": 4423, + "▁region": 4424, + "ady": 4425, + "IO": 4426, + "ander": 4427, + "Net": 4428, + "▁built": 4429, + "▁install": 4430, + "▁approach": 4431, + "Cur": 4432, + "▁fine": 4433, + "▁talking": 4434, + "▁changes": 4435, + "Style": 4436, + "▁Mart": 4437, + "лю": 4438, + "response": 4439, + "teger": 4440, + "{\r": 4441, + "irit": 4442, + "▁protected": 4443, + "▁rele": 4444, + "ership": 4445, + "тель": 4446, + "unsigned": 4447, + "ialize": 4448, + "▁https": 4449, + "Tag": 4450, + "▁$(": 4451, + "more": 4452, + "ypes": 4453, + "▁stream": 4454, + "etch": 4455, + "▁engine": 4456, + "KE": 4457, + "cmd": 4458, + "script": 4459, + "ttp": 4460, + "▁avoid": 4461, + "▁terr": 4462, + "▁rock": 4463, + "▁ful": 4464, + "Update": 4465, + "▁environment": 4466, + "▁prec": 4467, + "▁са": 4468, + "▁cases": 4469, + "▁offset": 4470, + "▁rais": 4471, + "lib": 4472, + "ées": 4473, + "aa": 4474, + "yt": 4475, + "▁arr": 4476, + "opyright": 4477, + "first": 4478, + "▁util": 4479, + "▁feature": 4480, + "posed": 4481, + "ffect": 4482, + "жа": 4483, + "itude": 4484, + "ements": 4485, + "asc": 4486, + "ador": 4487, + "lections": 4488, + "▁club": 4489, + "]{": 4490, + "▁*)": 4491, + "ство": 4492, + "▁imm": 4493, + "▁former": 4494, + "▁rights": 4495, + "▁decided": 4496, + "▁rev": 4497, + "▁ment": 4498, + "ani": 4499, + "▁stru": 4500, + "▁attention": 4501, + "artment": 4502, + "▁Ital": 4503, + "alle": 4504, + "▁bis": 4505, + "gener": 4506, + "▁integr": 4507, + "ello": 4508, + "rypt": 4509, + "▁achie": 4510, + "nes": 4511, + "▁stra": 4512, + "sb": 4513, + "▁types": 4514, + "▁RE": 4515, + "Init": 4516, + "▁comment": 4517, + "▁addition": 4518, + "▁ID": 4519, + "ART": 4520, + "FO": 4521, + "щи": 4522, + "Conne": 4523, + "▁squ": 4524, + "▁considered": 4525, + "idad": 4526, + "▁October": 4527, + "cial": 4528, + "▁Of": 4529, + "▁travel": 4530, + "▁boy": 4531, + "').": 4532, + "uy": 4533, + "illa": 4534, + "istry": 4535, + "▁va": 4536, + "▁Che": 4537, + "ERT": 4538, + "ende": 4539, + "ungen": 4540, + "aby": 4541, + "▁Rober": 4542, + "▁playing": 4543, + "ils": 4544, + "▁sam": 4545, + "▁execut": 4546, + "▁Us": 4547, + "▁mut": 4548, + "▁bal": 4549, + "asse": 4550, + "▁kids": 4551, + "▁financ": 4552, + "gor": 4553, + "▁Sec": 4554, + "bert": 4555, + "▁High": 4556, + "▁је": 4557, + "▁kept": 4558, + "button": 4559, + "itory": 4560, + "▁Rem": 4561, + "▁DE": 4562, + "▁reach": 4563, + "▁bur": 4564, + "Label": 4565, + "át": 4566, + "ago": 4567, + "▁passed": 4568, + "▁behav": 4569, + "xFF": 4570, + "▁Return": 4571, + "STR": 4572, + "▁Les": 4573, + "▁ord": 4574, + "ala": 4575, + "inger": 4576, + "▁Since": 4577, + "▁experi": 4578, + "▁shall": 4579, + "▁star": 4580, + "non": 4581, + "▁gun": 4582, + "▁Bel": 4583, + "▁obj": 4584, + "ares": 4585, + "rs": 4586, + "▁weeks": 4587, + "nen": 4588, + "▁Stre": 4589, + "oring": 4590, + "▁î": 4591, + "▁serious": 4592, + "times": 4593, + "▁House": 4594, + "▁roll": 4595, + "▁register": 4596, + "▁module": 4597, + "▁applic": 4598, + "IR": 4599, + "▁cook": 4600, + "aux": 4601, + "▁save": 4602, + "▁Cr": 4603, + ",\r": 4604, + "▁states": 4605, + "▁empty": 4606, + "▁autom": 4607, + "figure": 4608, + "iance": 4609, + "▁happy": 4610, + "▁fn": 4611, + "▁jud": 4612, + "▁hat": 4613, + "ACK": 4614, + "▁Fe": 4615, + "$-": 4616, + "ivil": 4617, + "oted": 4618, + "▁sizeof": 4619, + "▁situation": 4620, + "▁lives": 4621, + "▁feeling": 4622, + "▁risk": 4623, + "▁January": 4624, + "▁Object": 4625, + "▁recomm": 4626, + "▁вы": 4627, + "▁potential": 4628, + "eah": 4629, + "▁complex": 4630, + "printf": 4631, + "istance": 4632, + "irth": 4633, + "lik": 4634, + "aste": 4635, + "▁whose": 4636, + "Arg": 4637, + "▁modern": 4638, + "iones": 4639, + "▁че": 4640, + "▁sett": 4641, + "▁Mag": 4642, + "ae": 4643, + "▁condition": 4644, + "Length": 4645, + "▁fit": 4646, + "ounds": 4647, + "▁changed": 4648, + "▁guy": 4649, + "filter": 4650, + "atever": 4651, + "éd": 4652, + "remove": 4653, + "▁hop": 4654, + "▁Out": 4655, + "▁Rich": 4656, + "child": 4657, + "▁included": 4658, + "$\\": 4659, + "▁Tom": 4660, + "eline": 4661, + "▁sometimes": 4662, + "▁drink": 4663, + "▁quant": 4664, + "▁please": 4665, + "▁Int": 4666, + "rief": 4667, + "▁exactly": 4668, + "cing": 4669, + "▁allowed": 4670, + "build": 4671, + "▁beautiful": 4672, + "▁Well": 4673, + "▁looks": 4674, + "▁ü": 4675, + "▁chance": 4676, + "▁wrote": 4677, + "▁nor": 4678, + "▁failed": 4679, + "Met": 4680, + "▁prior": 4681, + "▁hundred": 4682, + "ской": 4683, + "oria": 4684, + "▁cy": 4685, + "▁web": 4686, + "▁mess": 4687, + "leq": 4688, + "dy": 4689, + "tex": 4690, + "▁anim": 4691, + "atur": 4692, + "▁structure": 4693, + "option": 4694, + "▁actual": 4695, + "▁Franc": 4696, + "enced": 4697, + ".": 4884, + "▁production": 4885, + "iger": 4886, + "▁ст": 4887, + "show": 4888, + "▁population": 4889, + "▁park": 4890, + "▁Ze": 4891, + "▁necessary": 4892, + "▁trust": 4893, + "▁shown": 4894, + "module": 4895, + "GE": 4896, + "▁lay": 4897, + "▁announ": 4898, + "▁className": 4899, + "▁calcul": 4900, + "Function": 4901, + "▁Sal": 4902, + "OK": 4903, + "TP": 4904, + "▁entry": 4905, + "▁Stud": 4906, + "▁items": 4907, + "▁security": 4908, + "Entry": 4909, + "float": 4910, + "ls": 4911, + "ibly": 4912, + "▁contribut": 4913, + "▁Check": 4914, + "MD": 4915, + "▁improve": 4916, + "Part": 4917, + "▁systems": 4918, + "Bl": 4919, + "▁policy": 4920, + "▁screen": 4921, + "▁Any": 4922, + "▁opened": 4923, + "alloc": 4924, + "▁December": 4925, + "▁É": 4926, + "▁email": 4927, + "ader": 4928, + "=>": 4929, + "▁Hen": 4930, + "▁info": 4931, + "▁float": 4932, + "▁switch": 4933, + "ран": 4934, + "urance": 4935, + "▁assum": 4936, + "ustr": 4937, + "▁groups": 4938, + "▁Read": 4939, + "▁wat": 4940, + "Sp": 4941, + "вер": 4942, + "RAN": 4943, + "hib": 4944, + "ALL": 4945, + "▁hus": 4946, + "Spec": 4947, + "\"))": 4948, + "▁French": 4949, + "▁Class": 4950, + "▁president": 4951, + "▁definit": 4952, + "▁Nor": 4953, + "▁Thom": 4954, + "aign": 4955, + "Width": 4956, + "Do": 4957, + "▁{@": 4958, + "agon": 4959, + "▁Lu": 4960, + "▁followed": 4961, + "MM": 4962, + "asons": 4963, + "tmp": 4964, + "▁throws": 4965, + "ITY": 4966, + "ном": 4967, + "▁fair": 4968, + "▁pen": 4969, + "ég": 4970, + "▁interface": 4971, + "▁saf": 4972, + "oon": 4973, + "Back": 4974, + "▁speed": 4975, + "▁extends": 4976, + "empty": 4977, + "▁пере": 4978, + "▁proper": 4979, + "▁driv": 4980, + "фи": 4981, + "▁center": 4982, + "header": 4983, + "▁})": 4984, + "wa": 4985, + "▁middle": 4986, + "▁choose": 4987, + "▁Stad": 4988, + "SO": 4989, + "Factory": 4990, + "Dev": 4991, + "icles": 4992, + "▁application": 4993, + "▁models": 4994, + "pite": 4995, + "cap": 4996, + "xi": 4997, + "ospital": 4998, + "▁dream": 4999, + "END": 5000, + "▁contract": 5001, + "icrosoft": 5002, + "▁thous": 5003, + "izes": 5004, + "▁да": 5005, + "▁CO": 5006, + "▁direction": 5007, + "▁``": 5008, + "▁drive": 5009, + "Max": 5010, + "cia": 5011, + "▁continu": 5012, + "▁Alex": 5013, + "▁gold": 5014, + "▁prep": 5015, + "▁origin": 5016, + "▁rap": 5017, + "Op": 5018, + "ously": 5019, + "▁areas": 5020, + "PORT": 5021, + "она": 5022, + "▁safe": 5023, + "▁professional": 5024, + "apache": 5025, + "▁temper": 5026, + "sz": 5027, + "▁unit": 5028, + "▁cop": 5029, + "eqn": 5030, + "Listener": 5031, + "▁format": 5032, + "select": 5033, + "▁comfort": 5034, + "▁meant": 5035, + "iday": 5036, + "eme": 5037, + "▁active": 5038, + "▁note": 5039, + "▁Mil": 5040, + "only": 5041, + "▁<=": 5042, + "▁neigh": 5043, + "ao": 5044, + "▁blue": 5045, + "▁TV": 5046, + "Child": 5047, + "▁reached": 5048, + "Address": 5049, + "ств": 5050, + "▁closed": 5051, + "inder": 5052, + "olo": 5053, + "▁alt": 5054, + "▁adm": 5055, + "Format": 5056, + "UI": 5057, + "▁Ham": 5058, + "▁frequ": 5059, + "▁independ": 5060, + "▁easily": 5061, + "▁Land": 5062, + "▁tor": 5063, + "ography": 5064, + "infty": 5065, + "▁Work": 5066, + "iven": 5067, + "▁County": 5068, + "▁src": 5069, + "}$,": 5070, + "parse": 5071, + "CD": 5072, + "▁Cour": 5073, + "▁fol": 5074, + "Entity": 5075, + "pgf": 5076, + "▁China": 5077, + "▁Sub": 5078, + "hood": 5079, + "▁fields": 5080, + "▁yes": 5081, + "rend": 5082, + "▁towards": 5083, + "▁staff": 5084, + "▁Air": 5085, + "▁station": 5086, + "atives": 5087, + "▁impact": 5088, + "вы": 5089, + "▁directly": 5090, + "issions": 5091, + "iva": 5092, + "|\\": 5093, + "Ptr": 5094, + "▁Sant": 5095, + "Pol": 5096, + "▁progress": 5097, + "itar": 5098, + "▁parts": 5099, + "▁plant": 5100, + "▁absolut": 5101, + "▁guess": 5102, + "eqref": 5103, + "▁tim": 5104, + "▁Lou": 5105, + "▁cool": 5106, + "alu": 5107, + "▁mouth": 5108, + "них": 5109, + "▁height": 5110, + "gest": 5111, + "▁Post": 5112, + "▁board": 5113, + "▁tit": 5114, + "▁hour": 5115, + "▁server": 5116, + "▁players": 5117, + "rier": 5118, + "Link": 5119, + "▁President": 5120, + "](": 5121, + "▁construct": 5122, + "handle": 5123, + "}$.": 5124, + "rying": 5125, + "▁shop": 5126, + "iana": 5127, + "exp": 5128, + "Helper": 5129, + "Offset": 5130, + "aches": 5131, + "▁connection": 5132, + "▁difference": 5133, + "service": 5134, + "▁gas": 5135, + "▁priv": 5136, + "▁univers": 5137, + "▁wish": 5138, + "Rem": 5139, + "Url": 5140, + "geb": 5141, + "So": 5142, + "ensions": 5143, + "Module": 5144, + "SIZE": 5145, + "▁prem": 5146, + "window": 5147, + "▁dies": 5148, + "del": 5149, + "▁row": 5150, + "▁average": 5151, + "xim": 5152, + "▁pu": 5153, + "anç": 5154, + "Det": 5155, + "ker": 5156, + "ya": 5157, + "▁Det": 5158, + "▁på": 5159, + "▁named": 5160, + "▁decision": 5161, + "win": 5162, + "▁George": 5163, + "arily": 5164, + "▁solution": 5165, + "▁multiple": 5166, + "ategy": 5167, + "▁learning": 5168, + "▁secret": 5169, + "DO": 5170, + "▁nice": 5171, + "////////////////": 5172, + "Su": 5173, + "itation": 5174, + "▁join": 5175, + "▁elements": 5176, + "▁emer": 5177, + "tilde": 5178, + "▁dep": 5179, + "▁shot": 5180, + "▁platform": 5181, + "othing": 5182, + "My": 5183, + "edia": 5184, + "oms": 5185, + "aily": 5186, + "([": 5187, + "▁dress": 5188, + "▁official": 5189, + "estern": 5190, + "▁discover": 5191, + "▁mi": 5192, + "ные": 5193, + "CA": 5194, + "oding": 5195, + "▁Found": 5196, + "▁affect": 5197, + "Vis": 5198, + "stract": 5199, + "iced": 5200, + "debug": 5201, + "▁related": 5202, + "▁spect": 5203, + "ushed": 5204, + "сько": 5205, + "▁bank": 5206, + "▁cele": 5207, + "AND": 5208, + "olf": 5209, + "ем": 5210, + "▁fill": 5211, + "▁gives": 5212, + "▁бу": 5213, + "aron": 5214, + "▁Jes": 5215, + "REG": 5216, + "▁sudd": 5217, + "dated": 5218, + "vi": 5219, + "▁gi": 5220, + "send": 5221, + "cpp": 5222, + "▁spent": 5223, + "ande": 5224, + "▁operation": 5225, + "process": 5226, + "▁inform": 5227, + "▁Free": 5228, + "yond": 5229, + "▁perhaps": 5230, + "▁surv": 5231, + "▁Loc": 5232, + "▁concl": 5233, + "▁раз": 5234, + "▁Over": 5235, + "hol": 5236, + "raz": 5237, + "Write": 5238, + "▁giving": 5239, + "rd": 5240, + "instance": 5241, + "▁released": 5242, + "▁Ro": 5243, + "RA": 5244, + "▁practice": 5245, + "▁graph": 5246, + "▁increase": 5247, + "▁figure": 5248, + "Filter": 5249, + "HECK": 5250, + "idx": 5251, + "▁glass": 5252, + "ski": 5253, + "comes": 5254, + "▁cat": 5255, + "▁cold": 5256, + "goto": 5257, + "ufact": 5258, + "▁Copyright": 5259, + "}}\\": 5260, + "▁streng": 5261, + "▁dir": 5262, + "token": 5263, + "▁occur": 5264, + "arlier": 5265, + "▁measure": 5266, + "▁sec": 5267, + "▁más": 5268, + "▁Net": 5269, + "▁argument": 5270, + "▁sou": 5271, + "▁moving": 5272, + "▁prefer": 5273, + "mask": 5274, + "<<": 5275, + "▁breath": 5276, + "▁physical": 5277, + "▁positive": 5278, + "▁sor": 5279, + "▁depart": 5280, + "▁remove": 5281, + "▁kit": 5282, + "▁meeting": 5283, + "▁Data": 5284, + "ograf": 5285, + "actions": 5286, + "▁parameters": 5287, + "▁Att": 5288, + "esch": 5289, + "▁involved": 5290, + "ät": 5291, + "LL": 5292, + "Bar": 5293, + "▁си": 5294, + "ech": 5295, + "GET": 5296, + "▁prevent": 5297, + "▁beyond": 5298, + "▁Other": 5299, + "än": 5300, + "byte": 5301, + "▁sudden": 5302, + "olve": 5303, + "▁но": 5304, + "LOG": 5305, + "unit": 5306, + "▁truth": 5307, + "rat": 5308, + "SD": 5309, + "▁eat": 5310, + "▁Mad": 5311, + "▁provides": 5312, + "▁session": 5313, + "Dele": 5314, + "▁convers": 5315, + "center": 5316, + "▁continued": 5317, + "otion": 5318, + "cache": 5319, + "display": 5320, + "▁protect": 5321, + "ams": 5322, + "▁pow": 5323, + "CTION": 5324, + "▁Mac": 5325, + "mo": 5326, + "ха": 5327, + "▁distance": 5328, + "▁Time": 5329, + "gi": 5330, + "▁sequ": 5331, + "Target": 5332, + "сле": 5333, + "Server": 5334, + "▁wide": 5335, + "close": 5336, + "▁cru": 5337, + "Ext": 5338, + "▁select": 5339, + "▁pattern": 5340, + "\"));": 5341, + "Provider": 5342, + "URL": 5343, + "▁green": 5344, + "▁waiting": 5345, + "proto": 5346, + "▁immediately": 5347, + "common": 5348, + "azione": 5349, + "river": 5350, + "▁sen": 5351, + "▁!==": 5352, + "▁February": 5353, + "urb": 5354, + "▁Sen": 5355, + "dest": 5356, + ">": 6122, + "command": 6123, + "atz": 6124, + "▁mal": 6125, + "став": 6126, + "▁Press": 6127, + "▁characters": 6128, + "▁zero": 6129, + "AGE": 6130, + "rapper": 6131, + "▁kitchen": 6132, + "aming": 6133, + "▁restr": 6134, + "XX": 6135, + "▁College": 6136, + "▁Array": 6137, + "▁fresh": 6138, + "▁shift": 6139, + "▁specified": 6140, + "plete": 6141, + "ITE": 6142, + "▁Camp": 6143, + "rial": 6144, + "cb": 6145, + "▁TH": 6146, + "IB": 6147, + "osen": 6148, + "▁ú": 6149, + "▁params": 6150, + "ignment": 6151, + "adding": 6152, + "▁degree": 6153, + "Local": 6154, + "Oh": 6155, + "▁zur": 6156, + "▁levels": 6157, + "CS": 6158, + "finished": 6159, + "Case": 6160, + "riage": 6161, + "Vector": 6162, + "▁sea": 6163, + "antic": 6164, + "▁League": 6165, + "▁therefore": 6166, + "One": 6167, + "Return": 6168, + "Access": 6169, + "vas": 6170, + "▁ос": 6171, + "▁rat": 6172, + "Big": 6173, + "▁behavior": 6174, + "kr": 6175, + "▁undefined": 6176, + "▁Es": 6177, + "▁appeared": 6178, + "eles": 6179, + "▁WAR": 6180, + "Stat": 6181, + "▁Google": 6182, + "▁credit": 6183, + "▁File": 6184, + "anging": 6185, + "house": 6186, + "romise": 6187, + "gent": 6188, + "▁habit": 6189, + "▁society": 6190, + "▁encour": 6191, + "▁paint": 6192, + "pet": 6193, + "▁UK": 6194, + "aws": 6195, + "onom": 6196, + "Gl": 6197, + "}_{\\": 6198, + "eless": 6199, + "emy": 6200, + "▁Cong": 6201, + "▁developed": 6202, + "▁images": 6203, + "▁ö": 6204, + "▁font": 6205, + "clear": 6206, + "gin": 6207, + "▁Lord": 6208, + "▁transport": 6209, + "▁::": 6210, + "▁cup": 6211, + "ulate": 6212, + "▁During": 6213, + "priv": 6214, + "▁extrem": 6215, + "▁Di": 6216, + "▁doubt": 6217, + "Py": 6218, + "ifying": 6219, + "split": 6220, + "ego": 6221, + "github": 6222, + "▁),": 6223, + "ROM": 6224, + "▁chair": 6225, + "▁trade": 6226, + "▁nicht": 6227, + "Top": 6228, + "Store": 6229, + "▁parte": 6230, + "project": 6231, + "nia": 6232, + "▁від": 6233, + "war": 6234, + "▁Prof": 6235, + "▁caught": 6236, + "Thread": 6237, + "ства": 6238, + "author": 6239, + "▁doll": 6240, + "▁harm": 6241, + "▁Gen": 6242, + "tree": 6243, + "etime": 6244, + "cfg": 6245, + "▁guys": 6246, + "▁California": 6247, + "▁Green": 6248, + "▁movement": 6249, + "iej": 6250, + "▁statement": 6251, + "▁seeing": 6252, + "▁haven": 6253, + "vention": 6254, + "SL": 6255, + "chedul": 6256, + "iert": 6257, + "▁primary": 6258, + "▁civil": 6259, + "rian": 6260, + "▁button": 6261, + "▁lived": 6262, + "Pass": 6263, + "sor": 6264, + "▁watching": 6265, + "▁skills": 6266, + "tee": 6267, + "Level": 6268, + "▁scient": 6269, + "hs": 6270, + "▁agre": 6271, + "cat": 6272, + "▁tend": 6273, + "▁Mill": 6274, + "▁Cap": 6275, + "ORD": 6276, + "gle": 6277, + "▁сво": 6278, + "»,": 6279, + "▁ahead": 6280, + "vest": 6281, + "▁Jose": 6282, + "ischer": 6283, + "și": 6284, + "▁leaving": 6285, + "▁для": 6286, + "▁south": 6287, + "▁consum": 6288, + "Range": 6289, + "▁activities": 6290, + "Sec": 6291, + "▁sales": 6292, + "▁fix": 6293, + "▁jed": 6294, + "rum": 6295, + "vector": 6296, + "▁spot": 6297, + "▁manufact": 6298, + "кт": 6299, + "orrow": 6300, + "sign": 6301, + "▁college": 6302, + "▁driver": 6303, + "▁definitely": 6304, + "▁spend": 6305, + "mission": 6306, + "зу": 6307, + "atively": 6308, + "bi": 6309, + "Callback": 6310, + "▁particularly": 6311, + "▁hell": 6312, + "▁pool": 6313, + "PRE": 6314, + "▁clearly": 6315, + "PT": 6316, + "othes": 6317, + "▁Id": 6318, + "Location": 6319, + "▁Run": 6320, + "▁fixed": 6321, + "▁Hand": 6322, + "bal": 6323, + "double": 6324, + "Can": 6325, + "Omega": 6326, + "▁challeng": 6327, + "▁standing": 6328, + "iten": 6329, + "▁mechan": 6330, + "▁durch": 6331, + "▁dell": 6332, + "▁raised": 6333, + "▁weak": 6334, + "▁Du": 6335, + "grad": 6336, + "▁scene": 6337, + "poss": 6338, + "▁ton": 6339, + "▁earth": 6340, + "ulations": 6341, + "▁strength": 6342, + "aked": 6343, + "▁remain": 6344, + "▁Bi": 6345, + "▁customer": 6346, + "range": 6347, + "▁interested": 6348, + "ONE": 6349, + "▁coff": 6350, + "require": 6351, + "▁Only": 6352, + "▁Web": 6353, + "▁farm": 6354, + "▁activity": 6355, + "▁rout": 6356, + "bling": 6357, + "SY": 6358, + "▁Richard": 6359, + "▁Ref": 6360, + "▁кон": 6361, + "▁jun": 6362, + "born": 6363, + "ijn": 6364, + "Configuration": 6365, + "uman": 6366, + "EE": 6367, + "▁married": 6368, + "▁За": 6369, + "▁fat": 6370, + "▁kid": 6371, + "▁Tur": 6372, + "▁offered": 6373, + "nic": 6374, + "▁Big": 6375, + "Gamma": 6376, + "▁Health": 6377, + "▁TR": 6378, + "▁się": 6379, + "▁construction": 6380, + "▁Church": 6381, + "▁Bet": 6382, + "bus": 6383, + "▁earn": 6384, + "rict": 6385, + "▁пра": 6386, + "▁brain": 6387, + "▁fra": 6388, + "▁Op": 6389, + "FIG": 6390, + "ema": 6391, + "▁European": 6392, + "▁Saint": 6393, + "ARE": 6394, + "uri": 6395, + "▁River": 6396, + "{}": 6397, + "▁sitting": 6398, + "▁understanding": 6399, + "▁plans": 6400, + "ropri": 6401, + "▁older": 6402, + "▁pressure": 6403, + "Impl": 6404, + "▁peace": 6405, + "Connection": 6406, + "▁fi": 6407, + "rich": 6408, + "▁shut": 6409, + "apers": 6410, + "Port": 6411, + "▁Look": 6412, + "rim": 6413, + "auth": 6414, + "auto": 6415, + "▁highly": 6416, + "▁unless": 6417, + "▁Wal": 6418, + "▁ren": 6419, + "ws": 6420, + "▁core": 6421, + "(-": 6422, + "▁clim": 6423, + "ruit": 6424, + "▁callback": 6425, + "hest": 6426, + "▁Charles": 6427, + "▁Long": 6428, + "}=": 6429, + "ър": 6430, + "▁shared": 6431, + "ulated": 6432, + "gorithm": 6433, + "▁Home": 6434, + "▁village": 6435, + "ees": 6436, + "sv": 6437, + "▁restaur": 6438, + "rey": 6439, + "▁Cast": 6440, + "▁Person": 6441, + "кий": 6442, + "▁organiz": 6443, + "▁Rad": 6444, + "ponents": 6445, + "▁werden": 6446, + "▁bow": 6447, + "sen": 6448, + "ami": 6449, + "Interface": 6450, + "▁basis": 6451, + "▁Company": 6452, + "ernel": 6453, + "itu": 6454, + "Hash": 6455, + "▁aan": 6456, + "▁х": 6457, + "▁smile": 6458, + "xml": 6459, + "▁scen": 6460, + "amm": 6461, + "tool": 6462, + "aria": 6463, + "▁accur": 6464, + "settings": 6465, + "▁Jesus": 6466, + "acement": 6467, + "power": 6468, + "(!": 6469, + "▁calls": 6470, + "▁basic": 6471, + "▁settings": 6472, + "ript": 6473, + "pool": 6474, + "ctors": 6475, + "▁Foundation": 6476, + "▁weap": 6477, + "KEY": 6478, + "foot": 6479, + "▁radio": 6480, + "▁helped": 6481, + "mann": 6482, + "▁jump": 6483, + "▁tick": 6484, + "▁growing": 6485, + "aten": 6486, + "real": 6487, + "▁increasing": 6488, + "Device": 6489, + "varepsilon": 6490, + "▁sets": 6491, + "▁advant": 6492, + "Open": 6493, + "▁reasons": 6494, + "▁supposed": 6495, + "oes": 6496, + "ede": 6497, + "teen": 6498, + "ifdef": 6499, + "▁delete": 6500, + "▁&=": 6501, + "▁Bill": 6502, + "▁aim": 6503, + "▁Ok": 6504, + "▁Av": 6505, + "reci": 6506, + "acks": 6507, + "iste": 6508, + "Properties": 6509, + "▁tmp": 6510, + "▁dei": 6511, + "PER": 6512, + "DC": 6513, + "sta": 6514, + "нии": 6515, + "▁limited": 6516, + "▁greater": 6517, + "description": 6518, + "ori": 6519, + "aints": 6520, + "▁hy": 6521, + "▁Mel": 6522, + "▁CH": 6523, + "cons": 6524, + "▁surround": 6525, + "▁Who": 6526, + "arc": 6527, + "▁telev": 6528, + "itution": 6529, + "▁equal": 6530, + "кі": 6531, + "▁Israel": 6532, + "äh": 6533, + "▁Caption": 6534, + "▁exerc": 6535, + "empor": 6536, + "▁++": 6537, + "▁lib": 6538, + "make": 6539, + "▁MA": 6540, + "copy": 6541, + "friend": 6542, + "▁кото": 6543, + "▁damage": 6544, + "▁\\,": 6545, + "oded": 6546, + "▁none": 6547, + "▁evalu": 6548, + "ston": 6549, + ">,": 6550, + "FOR": 6551, + "▁norm": 6552, + "appe": 6553, + "Session": 6554, + "▁adult": 6555, + "▁hospital": 6556, + "▁recommend": 6557, + "property": 6558, + "stein": 6559, + "final": 6560, + "▁nu": 6561, + "second": 6562, + "▁aspect": 6563, + "\")]": 6564, + "жен": 6565, + "amento": 6566, + "▁rac": 6567, + "save": 6568, + "▁football": 6569, + "Ab": 6570, + "ungs": 6571, + "abil": 6572, + "▁Arch": 6573, + "system": 6574, + "hist": 6575, + "▁luck": 6576, + "render": 6577, + "▁sein": 6578, + "ioni": 6579, + "▁rot": 6580, + "▁corner": 6581, + "▁appropri": 6582, + "▁Software": 6583, + "▁tele": 6584, + "Delete": 6585, + "▁According": 6586, + "▁prison": 6587, + "▁lic": 6588, + "▁ми": 6589, + "term": 6590, + "sets": 6591, + "▁vel": 6592, + "▁rank": 6593, + "▁existing": 6594, + "▁Vir": 6595, + "▁trip": 6596, + "▁му": 6597, + "avax": 6598, + "▁ris": 6599, + "▁define": 6600, + "▁heat": 6601, + "car": 6602, + "▁convert": 6603, + "email": 6604, + "▁Under": 6605, + "▁Ш": 6606, + "▁Grand": 6607, + "▁exists": 6608, + "sys": 6609, + "eff": 6610, + "▁Top": 6611, + "▁č": 6612, + "▁tempor": 6613, + "▁arguments": 6614, + "▁supported": 6615, + "ensed": 6616, + "▁Francis": 6617, + "▁coord": 6618, + "▁achieve": 6619, + "▁Name": 6620, + "▁Jahr": 6621, + "▁Gi": 6622, + "she": 6623, + "▁Dev": 6624, + "▁alla": 6625, + "▁WIT": 6626, + "agment": 6627, + "custom": 6628, + "alls": 6629, + "&&": 6630, + "WE": 6631, + "▁holding": 6632, + "prototype": 6633, + "▁fing": 6634, + "▁bag": 6635, + "▁Party": 6636, + "stack": 6637, + "▁economic": 6638, + "▁Gal": 6639, + "idents": 6640, + "▁Jun": 6641, + "▁showed": 6642, + "osh": 6643, + "▁Bay": 6644, + "mail": 6645, + "▁SO": 6646, + "▁\"<": 6647, + "graphics": 6648, + "▁fu": 6649, + "click": 6650, + "▁battle": 6651, + "{{": 6652, + "▁Event": 6653, + "rior": 6654, + "chaft": 6655, + "▁favorite": 6656, + "usive": 6657, + "support": 6658, + "bm": 6659, + "Kind": 6660, + "▁safety": 6661, + "▁Ent": 6662, + "cup": 6663, + "▁Australia": 6664, + "▁destroy": 6665, + "▁organization": 6666, + "iden": 6667, + "################": 6668, + "dec": 6669, + "▁za": 6670, + "▁seven": 6671, + "arely": 6672, + "▁flag": 6673, + "Dir": 6674, + "▁Carl": 6675, + "▁doctor": 6676, + "▁variety": 6677, + "▁Lin": 6678, + "▁tom": 6679, + "^{(": 6680, + "Bo": 6681, + "antes": 6682, + "▁mine": 6683, + "▁Mit": 6684, + "▁describe": 6685, + "Args": 6686, + "LS": 6687, + "API": 6688, + "▁Luc": 6689, + "phone": 6690, + "▁science": 6691, + "▁Oper": 6692, + "Next": 6693, + "▁investig": 6694, + "▁demonstr": 6695, + "▁Govern": 6696, + "▁objects": 6697, + "▁Louis": 6698, + "▁Returns": 6699, + "▁han": 6700, + "nam": 6701, + "▁comme": 6702, + "▁presence": 6703, + "▁pel": 6704, + "▁detect": 6705, + ")=": 6706, + "▁Chinese": 6707, + "▁rich": 6708, + "▁classes": 6709, + "▁expand": 6710, + "▁Dom": 6711, + "▁Dec": 6712, + "sn": 6713, + "peed": 6714, + "▁Jim": 6715, + "should": 6716, + "▁Smith": 6717, + "▁pages": 6718, + "▁Jean": 6719, + "rics": 6720, + "▁Sund": 6721, + "ads": 6722, + "▁Their": 6723, + "unicip": 6724, + "ву": 6725, + "▁download": 6726, + "▁stress": 6727, + "▁Pet": 6728, + "menu": 6729, + "reme": 6730, + "▁compared": 6731, + "Ste": 6732, + "IND": 6733, + "container": 6734, + "▁Indian": 6735, + "oren": 6736, + "▁ses": 6737, + "▁Whe": 6738, + "▁roku": 6739, + "▁established": 6740, + "▁generally": 6741, + "▁fle": 6742, + "__(": 6743, + "=\"+": 6744, + "Var": 6745, + "▁Make": 6746, + "▁removed": 6747, + "zz": 6748, + "ün": 6749, + "▁mix": 6750, + "erk": 6751, + "iation": 6752, + "outer": 6753, + "SK": 6754, + "▁becomes": 6755, + "▁Hall": 6756, + "scious": 6757, + "▁watched": 6758, + "▁gather": 6759, + "▁Result": 6760, + "proof": 6761, + "pay": 6762, + "▁produced": 6763, + "▁|=": 6764, + "▁border": 6765, + "▁din": 6766, + "▁script": 6767, + "▁actions": 6768, + "▁mas": 6769, + "ща": 6770, + "ooth": 6771, + "▁Techn": 6772, + "Json": 6773, + "▁filled": 6774, + "ден": 6775, + "undle": 6776, + "сту": 6777, + "Tool": 6778, + "▁king": 6779, + "▁ven": 6780, + "stra": 6781, + "▁predict": 6782, + "▁lui": 6783, + "▁WARRAN": 6784, + "▁Fun": 6785, + "Script": 6786, + "▁powerful": 6787, + "▁lose": 6788, + "atically": 6789, + "▁daily": 6790, + "▁ring": 6791, + "▁arrived": 6792, + "Stack": 6793, + "scope": 6794, + "▁Back": 6795, + "elij": 6796, + "▁ze": 6797, + "keys": 6798, + "{\"": 6799, + "VID": 6800, + "▁license": 6801, + "what": 6802, + "▁proced": 6803, + "rant": 6804, + "estival": 6805, + "agram": 6806, + "▁LO": 6807, + "▁Henry": 6808, + "▁flags": 6809, + "Down": 6810, + "scription": 6811, + "▁families": 6812, + "isse": 6813, + "bour": 6814, + "▁Bur": 6815, + "—\"": 6816, + "▁brief": 6817, + "▁creating": 6818, + "▁clients": 6819, + "rangle": 6820, + "▁amazing": 6821, + "▁sind": 6822, + "▁covered": 6823, + "Well": 6824, + "сте": 6825, + "тор": 6826, + "▁Bas": 6827, + "total": 6828, + "▁Init": 6829, + "▁sand": 6830, + "Unit": 6831, + "▁murder": 6832, + "▁bright": 6833, + "▁trav": 6834, + "icans": 6835, + "▁attribute": 6836, + "fc": 6837, + "▁placed": 6838, + "EST": 6839, + "Vari": 6840, + "▁cos": 6841, + "▁attract": 6842, + "anel": 6843, + "}).": 6844, + "bytes": 6845, + "▁parse": 6846, + "▁belong": 6847, + "BN": 6848, + "▁Sol": 6849, + "Po": 6850, + "`,": 6851, + "▁calling": 6852, + "▁?>": 6853, + "▁iter": 6854, + "▁url": 6855, + "▁evening": 6856, + "reek": 6857, + "▁honest": 6858, + "▁director": 6859, + "RC": 6860, + "▁solid": 6861, + "▁phil": 6862, + "iene": 6863, + "FAULT": 6864, + "cope": 6865, + "▁History": 6866, + "▁Team": 6867, + "reedom": 6868, + "▁ru": 6869, + "UB": 6870, + "▁worse": 6871, + "imo": 6872, + "Mat": 6873, + "▁Mex": 6874, + "actor": 6875, + "▁vor": 6876, + "ться": 6877, + "▁experiment": 6878, + "▁Play": 6879, + "▁Another": 6880, + "▁happens": 6881, + "uan": 6882, + "▁patients": 6883, + "▁rend": 6884, + "▁Mo": 6885, + "▁Tex": 6886, + "▁wed": 6887, + "tn": 6888, + "insert": 6889, + "▁па": 6890, + "▁anti": 6891, + "Match": 6892, + "ampionship": 6893, + "▁forces": 6894, + "▁Hot": 6895, + "▁phase": 6896, + "▁template": 6897, + "stop": 6898, + "icated": 6899, + "▁managed": 6900, + "wait": 6901, + "▁*(": 6902, + "GB": 6903, + "▁appoint": 6904, + "ła": 6905, + "▁stick": 6906, + "▁FOR": 6907, + "▁Vis": 6908, + "tor": 6909, + "▁př": 6910, + "quest": 6911, + "uses": 6912, + "\");\r": 6913, + "▁suddenly": 6914, + "éc": 6915, + "ND": 6916, + "urop": 6917, + "ред": 6918, + "▁insurance": 6919, + "access": 6920, + "unfinished": 6921, + "▁tamb": 6922, + "▁sac": 6923, + "▁Court": 6924, + "▁missing": 6925, + "▁Where": 6926, + "▁Sum": 6927, + "}^{\\": 6928, + "▁sua": 6929, + "_,": 6930, + "▁thick": 6931, + "▁Trump": 6932, + "▁operations": 6933, + "FS": 6934, + "▁deux": 6935, + "dz": 6936, + "Template": 6937, + "▁\"/": 6938, + "▁odd": 6939, + "▁reality": 6940, + "▁teams": 6941, + "▁cer": 6942, + "oma": 6943, + "▁și": 6944, + "▁cloud": 6945, + "▁Department": 6946, + "Ne": 6947, + "▁requires": 6948, + "items": 6949, + "▁III": 6950, + "rightarrow": 6951, + ")->": 6952, + "▁writer": 6953, + "replace": 6954, + "▁thr": 6955, + "jen": 6956, + "▁ot": 6957, + "▁occup": 6958, + "▁eventually": 6959, + "▁Math": 6960, + "▁conserv": 6961, + "amer": 6962, + "▁Fort": 6963, + "▁dry": 6964, + "▁sexual": 6965, + "▁costs": 6966, + "▁forms": 6967, + "▁Vict": 6968, + "PAR": 6969, + "framework": 6970, + "▁ди": 6971, + "Operation": 6972, + "зна": 6973, + "which": 6974, + "▁tight": 6975, + "Invalid": 6976, + "▁partner": 6977, + "▁пред": 6978, + "▁thank": 6979, + "▁guard": 6980, + "hem": 6981, + "Body": 6982, + "▁emot": 6983, + "IX": 6984, + "fast": 6985, + "що": 6986, + "ño": 6987, + "night": 6988, + "▁Sci": 6989, + "ника": 6990, + "▁TO": 6991, + "▁individuals": 6992, + "сси": 6993, + "}),": 6994, + "False": 6995, + "(\"%": 6996, + "▁optim": 6997, + "▁-->": 6998, + "▁factor": 6999, + "▁smaller": 7000, + "▁contain": 7001, + "spect": 7002, + "Engine": 7003, + "▁announced": 7004, + "▁Democr": 7005, + "▁rob": 7006, + "▁flat": 7007, + "osoph": 7008, + "Search": 7009, + "ahl": 7010, + "▁Exception": 7011, + "▁Ol": 7012, + "equals": 7013, + "▁unter": 7014, + "shape": 7015, + "NS": 7016, + "Obj": 7017, + "▁species": 7018, + "weight": 7019, + "you": 7020, + "▁este": 7021, + "▁View": 7022, + "▁mission": 7023, + "▁journal": 7024, + "Values": 7025, + "▁einem": 7026, + "ismo": 7027, + "▁projects": 7028, + "▁Das": 7029, + "rible": 7030, + "▁serve": 7031, + "▁opening": 7032, + "▁hur": 7033, + "▁programs": 7034, + "▁USA": 7035, + "iliar": 7036, + "idos": 7037, + "Br": 7038, + "estamp": 7039, + "▁tools": 7040, + "anner": 7041, + "RT": 7042, + "▁Start": 7043, + "▁bath": 7044, + "▁coffee": 7045, + "orter": 7046, + "internal": 7047, + "files": 7048, + "INVAL": 7049, + "ako": 7050, + "dt": 7051, + "▁Second": 7052, + "▁alloc": 7053, + "▁ended": 7054, + "acional": 7055, + "▁manager": 7056, + "▁Sun": 7057, + "agg": 7058, + "▁leader": 7059, + "olved": 7060, + "▁что": 7061, + "▁traditional": 7062, + "shot": 7063, + "rup": 7064, + "CF": 7065, + "▁Each": 7066, + "wr": 7067, + "▁Som": 7068, + "▁materials": 7069, + "▁msg": 7070, + "▁syn": 7071, + "▁produce": 7072, + "▁storage": 7073, + "subsection": 7074, + "▁Sie": 7075, + "▁IP": 7076, + "CESS": 7077, + "▁wa": 7078, + "Record": 7079, + "▁marketing": 7080, + "plet": 7081, + "Dialog": 7082, + "▁mentioned": 7083, + "▁Na": 7084, + "▁Union": 7085, + "▁API": 7086, + "▁negative": 7087, + "txt": 7088, + "▁easier": 7089, + "legal": 7090, + "Dep": 7091, + "▁novel": 7092, + "eur": 7093, + "ació": 7094, + "▁Bud": 7095, + "▁carry": 7096, + "schaft": 7097, + "▁broken": 7098, + "▁trees": 7099, + ">();": 7100, + "▁emb": 7101, + "ieder": 7102, + "▁route": 7103, + "ikel": 7104, + "▁listen": 7105, + "ashion": 7106, + "▁Mrs": 7107, + "▁equipment": 7108, + "agger": 7109, + "▁Thus": 7110, + "▁matrix": 7111, + "alla": 7112, + "▁Tour": 7113, + "▁conversation": 7114, + "Mon": 7115, + "ournal": 7116, + "▁minute": 7117, + "Am": 7118, + "Api": 7119, + "▁forget": 7120, + "Me": 7121, + "levant": 7122, + "temp": 7123, + "▁telling": 7124, + "move": 7125, + "▁independent": 7126, + "toString": 7127, + "edit": 7128, + "▁Jac": 7129, + "azz": 7130, + "react": 7131, + "▁cin": 7132, + "▁Prov": 7133, + "isted": 7134, + "▁hash": 7135, + "onna": 7136, + "iki": 7137, + "▁generated": 7138, + "Render": 7139, + "▁psych": 7140, + "nav": 7141, + "▁entr": 7142, + "пра": 7143, + "rx": 7144, + "ATH": 7145, + "▁assume": 7146, + "Tree": 7147, + "sembly": 7148, + "▁Matt": 7149, + "caption": 7150, + "▁solutions": 7151, + "▁faith": 7152, + "▁digital": 7153, + "▁excell": 7154, + "▁Version": 7155, + "Debug": 7156, + "▁жи": 7157, + "▁carried": 7158, + "reset": 7159, + "▁slowly": 7160, + "ancing": 7161, + "▁owner": 7162, + "▁Ter": 7163, + "▁Did": 7164, + "▁gest": 7165, + "▁été": 7166, + "▁proof": 7167, + "Font": 7168, + "▁nob": 7169, + "Co": 7170, + "▁GNU": 7171, + "▁liber": 7172, + "itness": 7173, + "▁hij": 7174, + "▁vert": 7175, + "ша": 7176, + "FLAG": 7177, + "MENT": 7178, + "▁Son": 7179, + "Mult": 7180, + "▁district": 7181, + "connect": 7182, + "jection": 7183, + "lymp": 7184, + "▁realized": 7185, + "mos": 7186, + "ye": 7187, + "▁render": 7188, + "rio": 7189, + "▁interpret": 7190, + "▁slightly": 7191, + "fix": 7192, + "▁studies": 7193, + "▁rid": 7194, + "atre": 7195, + "▁benefits": 7196, + "▁Face": 7197, + "ivery": 7198, + "рия": 7199, + "document": 7200, + "▁asking": 7201, + "Last": 7202, + "arante": 7203, + "▁Martin": 7204, + "▁Ell": 7205, + "▁vector": 7206, + "▁forced": 7207, + "оло": 7208, + "PH": 7209, + "WR": 7210, + "▁Kl": 7211, + "▁sky": 7212, + "▁strategy": 7213, + "ocked": 7214, + "▁neck": 7215, + "ści": 7216, + "OUT": 7217, + ")),": 7218, + "Custom": 7219, + "▁wie": 7220, + "▁sweet": 7221, + "▁temp": 7222, + "▁foreign": 7223, + "▁hall": 7224, + "astr": 7225, + "Ass": 7226, + "MODE": 7227, + "▁maximum": 7228, + "annels": 7229, + "▁tip": 7230, + "▁seconds": 7231, + "▁stack": 7232, + "iga": 7233, + "▁raise": 7234, + "enable": 7235, + "oir": 7236, + "▁soul": 7237, + "Ke": 7238, + ")$.": 7239, + "▁Tim": 7240, + "ALSE": 7241, + "iser": 7242, + "contin": 7243, + "bel": 7244, + "▁mad": 7245, + "lichen": 7246, + "abe": 7247, + "safe": 7248, + "▁concent": 7249, + "bound": 7250, + "▁Requ": 7251, + "switch": 7252, + "▁stone": 7253, + "▁transl": 7254, + "▁vac": 7255, + "andon": 7256, + "▁Fore": 7257, + "▁sounds": 7258, + "▁Pop": 7259, + "▁HT": 7260, + "lia": 7261, + "enter": 7262, + "▁helps": 7263, + "edy": 7264, + "ствен": 7265, + "anted": 7266, + "▁Its": 7267, + "▁Step": 7268, + "Icon": 7269, + "▁EXPECT": 7270, + "ialized": 7271, + "Post": 7272, + "aze": 7273, + "▁Carol": 7274, + "▁req": 7275, + "▁critical": 7276, + "DS": 7277, + "▁seat": 7278, + "aped": 7279, + "▁upper": 7280, + "▁Sy": 7281, + "▁explain": 7282, + "▁'./": 7283, + "utils": 7284, + "possible": 7285, + "▁dont": 7286, + "Host": 7287, + "▁approxim": 7288, + "Async": 7289, + "▁grab": 7290, + "▁sources": 7291, + "▁Mos": 7292, + "▁Germany": 7293, + "▁rub": 7294, + "CHAN": 7295, + "▁rain": 7296, + "▁truly": 7297, + "▁joined": 7298, + "▁": 8276, + "agnost": 8277, + "▁proposed": 8278, + "▁Game": 8279, + "▁efforts": 8280, + "вя": 8281, + "tc": 8282, + "ск": 8283, + "▁intent": 8284, + "▁Bre": 8285, + "isc": 8286, + "▁protest": 8287, + "▁holds": 8288, + "ometry": 8289, + "▁Have": 8290, + "▁detail": 8291, + "▁WITHOUT": 8292, + "yer": 8293, + "▁Kon": 8294, + "▁noticed": 8295, + "▁requirements": 8296, + "DEBUG": 8297, + "kins": 8298, + "▁Span": 8299, + "▁cars": 8300, + "meta": 8301, + "▁kil": 8302, + "▁Bron": 8303, + "▁experienced": 8304, + "▁remind": 8305, + "ourse": 8306, + "▁Western": 8307, + "tered": 8308, + "▁devices": 8309, + "▁pictures": 8310, + "▁tut": 8311, + "\"`": 8312, + "▁impossible": 8313, + "▁rail": 8314, + "▁feels": 8315, + "icas": 8316, + "illing": 8317, + "▁accident": 8318, + "▁'@": 8319, + "________": 8320, + "▁notes": 8321, + "oman": 8322, + "Parser": 8323, + "▁discovered": 8324, + "▁Roman": 8325, + "▁budget": 8326, + "▁guide": 8327, + "king": 8328, + "▁incred": 8329, + "olar": 8330, + "enden": 8331, + "Desc": 8332, + "▁wave": 8333, + "бли": 8334, + "igt": 8335, + "▁restrict": 8336, + "▁Ret": 8337, + "▁mac": 8338, + "ур": 8339, + "BS": 8340, + "ís": 8341, + "▁generation": 8342, + "dem": 8343, + "alo": 8344, + "бра": 8345, + "▁ordered": 8346, + "drop": 8347, + "▁pp": 8348, + "▁Review": 8349, + "▁literally": 8350, + "▁Sir": 8351, + "▁Yeah": 8352, + "▁density": 8353, + "riz": 8354, + "inde": 8355, + "▁gain": 8356, + "▁panel": 8357, + "jet": 8358, + "▁Times": 8359, + "▁nella": 8360, + "▁previously": 8361, + "points": 8362, + "Send": 8363, + "▁Brown": 8364, + "each": 8365, + "▁trigger": 8366, + "ometimes": 8367, + "icos": 8368, + "GR": 8369, + "Panel": 8370, + "ogen": 8371, + "▁cm": 8372, + "ructions": 8373, + "▁kiss": 8374, + "▁solo": 8375, + "▁famous": 8376, + "ran": 8377, + "про": 8378, + "▁thro": 8379, + "Graph": 8380, + "imit": 8381, + "▁Value": 8382, + "▁starts": 8383, + "ipeline": 8384, + "hd": 8385, + "TC": 8386, + "▁discussion": 8387, + "▁truck": 8388, + "aka": 8389, + "Only": 8390, + "▁Equ": 8391, + "▁kö": 8392, + "▁Bes": 8393, + "▁critic": 8394, + "▁propos": 8395, + "▁batt": 8396, + "▁Section": 8397, + "Show": 8398, + "gp": 8399, + "STATE": 8400, + "POST": 8401, + "▁Nord": 8402, + "▁innov": 8403, + "▁crim": 8404, + "axis": 8405, + "▁Turn": 8406, + "conn": 8407, + "Runtime": 8408, + "▁remaining": 8409, + "oston": 8410, + "▁Э": 8411, + "▁windows": 8412, + "▁Royal": 8413, + "▁vide": 8414, + "PP": 8415, + "chron": 8416, + "▁san": 8417, + "▁rise": 8418, + "▁delle": 8419, + "▁Dur": 8420, + "▁rapid": 8421, + "cert": 8422, + "LA": 8423, + "edge": 8424, + "▁\\]": 8425, + "▁entered": 8426, + "▁laws": 8427, + "▁photo": 8428, + "▁applications": 8429, + "▁Berlin": 8430, + "▁arrest": 8431, + "▁federal": 8432, + "▁Russia": 8433, + "▁usual": 8434, + "▁raw": 8435, + "▁più": 8436, + "être": 8437, + "JSON": 8438, + "SION": 8439, + "xture": 8440, + "istent": 8441, + "▁Power": 8442, + "Bit": 8443, + "▁capacity": 8444, + "▁cards": 8445, + "UID": 8446, + "iments": 8447, + "▁dar": 8448, + "▁Chicago": 8449, + "▁comfortable": 8450, + "tip": 8451, + "bas": 8452, + "▁mu": 8453, + "▁enemy": 8454, + "yan": 8455, + "▁фи": 8456, + "▁updated": 8457, + "ango": 8458, + "Ev": 8459, + "Effect": 8460, + "osing": 8461, + "rence": 8462, + "▁Congress": 8463, + "▁defe": 8464, + "▁ip": 8465, + "▁tout": 8466, + "▁freedom": 8467, + "▁ao": 8468, + "▁Therefore": 8469, + "Edit": 8470, + "▁Virgin": 8471, + "REE": 8472, + "argo": 8473, + "▁Dam": 8474, + "▁traffic": 8475, + "ños": 8476, + "▁alle": 8477, + "▁depth": 8478, + "Now": 8479, + "▁sides": 8480, + "▁годи": 8481, + "Descriptor": 8482, + "▁artikel": 8483, + "▁narrow": 8484, + "___": 8485, + "kw": 8486, + "uto": 8487, + "▁Facebook": 8488, + "tegr": 8489, + "boolean": 8490, + "nik": 8491, + "bd": 8492, + "Track": 8493, + "▁gran": 8494, + "reshold": 8495, + "вет": 8496, + "wrap": 8497, + "▁noise": 8498, + "igu": 8499, + "▁Bon": 8500, + "▁wy": 8501, + "linux": 8502, + "cks": 8503, + "▁fans": 8504, + "▁mach": 8505, + "▁prices": 8506, + "év": 8507, + "outs": 8508, + "standing": 8509, + "▁categ": 8510, + ";\\": 8511, + "▁decre": 8512, + "▁Saturday": 8513, + "▁menu": 8514, + "▁Nov": 8515, + "▁Yet": 8516, + "▁так": 8517, + "liche": 8518, + "▁Academ": 8519, + "▁communication": 8520, + "using": 8521, + "▁Society": 8522, + "▁nuc": 8523, + "pective": 8524, + "orial": 8525, + "▁afraid": 8526, + "▁animal": 8527, + "▁turning": 8528, + "dst": 8529, + "mathfrak": 8530, + "lers": 8531, + "▁lots": 8532, + "▁á": 8533, + "▁Tra": 8534, + "np": 8535, + "▁rose": 8536, + "▁GL": 8537, + "▁helping": 8538, + "▁winter": 8539, + "▁ком": 8540, + "Mock": 8541, + "▁investment": 8542, + "Use": 8543, + "▁Canad": 8544, + "нд": 8545, + "Copy": 8546, + "▁fly": 8547, + "SER": 8548, + "▁Far": 8549, + "▁Ros": 8550, + "amil": 8551, + "▁fighting": 8552, + "▁religious": 8553, + "super": 8554, + "screen": 8555, + "▁furn": 8556, + "▁surprised": 8557, + "▁replied": 8558, + "Activity": 8559, + "▁Down": 8560, + "▁insert": 8561, + "▁Olymp": 8562, + "▁pointed": 8563, + "▁Card": 8564, + "driver": 8565, + "▁Da": 8566, + "!--": 8567, + "roud": 8568, + "undo": 8569, + "▁messages": 8570, + "▁Point": 8571, + "VM": 8572, + "▁plane": 8573, + "xc": 8574, + "▁television": 8575, + "ён": 8576, + "▁thousands": 8577, + "▁cris": 8578, + "▁delay": 8579, + "▁Next": 8580, + "▁nombre": 8581, + "▁tu": 8582, + "▁skip": 8583, + "road": 8584, + "istration": 8585, + "▁tur": 8586, + "▁Develop": 8587, + "▁Па": 8588, + "▁дру": 8589, + "▁wonderful": 8590, + ">&": 8591, + "▁Liber": 8592, + "▁scope": 8593, + "▁manage": 8594, + "▁dass": 8595, + "▁recall": 8596, + "PM": 8597, + "▁relevant": 8598, + "▁Earth": 8599, + "▁как": 8600, + "▁apr": 8601, + "▁ASS": 8602, + "ién": 8603, + "▁SH": 8604, + "oom": 8605, + "itet": 8606, + "none": 8607, + "asi": 8608, + "▁motor": 8609, + "▁Show": 8610, + "nb": 8611, + "▁factors": 8612, + "▁forest": 8613, + "▁вре": 8614, + "thm": 8615, + "▁municip": 8616, + "▁turns": 8617, + "▁Division": 8618, + "EC": 8619, + "▁disappe": 8620, + "structor": 8621, + "▁somewhere": 8622, + "▁African": 8623, + "▁Institute": 8624, + "Grid": 8625, + "▁teacher": 8626, + "uries": 8627, + "▁respectively": 8628, + "▁SD": 8629, + "▁alive": 8630, + "▁pou": 8631, + "▁Water": 8632, + "фе": 8633, + "▁changing": 8634, + "▁afternoon": 8635, + "▁orders": 8636, + "Ret": 8637, + "Pointer": 8638, + "▁sav": 8639, + "erg": 8640, + "oked": 8641, + "essions": 8642, + "▁Fire": 8643, + "aret": 8644, + "imm": 8645, + "▁desire": 8646, + "▁що": 8647, + "▁Design": 8648, + "uture": 8649, + "▁Office": 8650, + "▁cmd": 8651, + "▁eating": 8652, + "Network": 8653, + "▁rough": 8654, + "operator": 8655, + "IGN": 8656, + "▁sports": 8657, + "▁weren": 8658, + "▁noted": 8659, + "▁twice": 8660, + "III": 8661, + "▁anx": 8662, + "▁elim": 8663, + "▁ав": 8664, + "▁io": 8665, + "▁speech": 8666, + "▁condu": 8667, + "elles": 8668, + "idade": 8669, + "▁advance": 8670, + "RI": 8671, + "oca": 8672, + "/\\": 8673, + "apshot": 8674, + "▁tail": 8675, + "models": 8676, + "ogy": 8677, + "▁Jeff": 8678, + "iration": 8679, + "▁Kore": 8680, + "▁leads": 8681, + "bat": 8682, + "Adapter": 8683, + "category": 8684, + "angular": 8685, + "▁saved": 8686, + "▁uniform": 8687, + "▁né": 8688, + "▁businesses": 8689, + "Hist": 8690, + "▁ар": 8691, + "domain": 8692, + "▁Si": 8693, + "raise": 8694, + "▁warn": 8695, + "hetic": 8696, + "▁Gro": 8697, + ")).": 8698, + "}>": 8699, + "зе": 8700, + "▁Amazon": 8701, + "▁Organ": 8702, + "▁Lake": 8703, + "▁agreement": 8704, + "xa": 8705, + "▁perman": 8706, + "▁containing": 8707, + "▁strange": 8708, + "сті": 8709, + "▁stupid": 8710, + "▁speaking": 8711, + "▁Internet": 8712, + "prefix": 8713, + "esc": 8714, + "Assert": 8715, + "prote": 8716, + "▁manner": 8717, + "▁Sz": 8718, + "unte": 8719, + "iot": 8720, + "Profile": 8721, + "oven": 8722, + "▁formed": 8723, + "▁lit": 8724, + "▁economy": 8725, + "▁cz": 8726, + "wid": 8727, + "REQ": 8728, + "▁chosen": 8729, + "▁Produ": 8730, + "oster": 8731, + "stances": 8732, + "awa": 8733, + "▁Ren": 8734, + "▁confirm": 8735, + "▁Бо": 8736, + "▁billion": 8737, + "▁déc": 8738, + "ých": 8739, + "▁illustr": 8740, + "TIES": 8741, + "▁Pub": 8742, + "▁ban": 8743, + "aded": 8744, + "ahn": 8745, + "▁Cath": 8746, + "nonumber": 8747, + "▁worst": 8748, + "▁Ме": 8749, + "▁suggested": 8750, + "stats": 8751, + "▁cant": 8752, + "▁align": 8753, + "kappa": 8754, + "▁hen": 8755, + "▁initi": 8756, + "'])": 8757, + "BI": 8758, + "▁garden": 8759, + "▁secure": 8760, + "▁\\[": 8761, + "handler": 8762, + "elli": 8763, + "ldots": 8764, + "secut": 8765, + "▁extended": 8766, + "}-": 8767, + "anie": 8768, + "▁Find": 8769, + "▁Museum": 8770, + "▁Conne": 8771, + "yy": 8772, + "▁passion": 8773, + "akers": 8774, + "ahr": 8775, + "ologies": 8776, + "▁equation": 8777, + "▁occasion": 8778, + "Let": 8779, + "']['": 8780, + "Print": 8781, + "anes": 8782, + "iente": 8783, + "▁Today": 8784, + "LECT": 8785, + "▁Af": 8786, + ",,": 8787, + "▁Та": 8788, + "▁```": 8789, + "even": 8790, + "sin": 8791, + "urer": 8792, + "▁°": 8793, + "otimes": 8794, + "▁IO": 8795, + "▁poet": 8796, + "()));": 8797, + "▁−": 8798, + "▁adopt": 8799, + "phere": 8800, + "#[": 8801, + "▁centre": 8802, + "oves": 8803, + "▁ans": 8804, + "dp": 8805, + "▁Kir": 8806, + "▁applicable": 8807, + "fp": 8808, + "▁visual": 8809, + "▁okay": 8810, + "oro": 8811, + "▁opportunities": 8812, + "Repository": 8813, + "▁ll": 8814, + "▁Rod": 8815, + "▁shel": 8816, + "▁launch": 8817, + "▁conven": 8818, + "▁Spe": 8819, + "Amer": 8820, + "▁cette": 8821, + "Cond": 8822, + "dep": 8823, + "Own": 8824, + "▁hook": 8825, + "▁dict": 8826, + "▁Those": 8827, + "▁fellow": 8828, + "▁philosoph": 8829, + "vin": 8830, + "ferences": 8831, + "hav": 8832, + "▁adding": 8833, + "iverse": 8834, + "game": 8835, + "▁Blue": 8836, + "▁clin": 8837, + "note": 8838, + "▁Ram": 8839, + "мер": 8840, + "covery": 8841, + "ña": 8842, + "▁би": 8843, + "▁fashion": 8844, + "▁broke": 8845, + "▁'\\": 8846, + "▁reader": 8847, + "ное": 8848, + "ности": 8849, + "▁payment": 8850, + "▁Lic": 8851, + "▁lips": 8852, + "▁academ": 8853, + "▁Mot": 8854, + "ells": 8855, + "CHECK": 8856, + "▁ру": 8857, + "▁MS": 8858, + "Editor": 8859, + "▁zone": 8860, + "iture": 8861, + "▁IT": 8862, + "runtime": 8863, + "▁proceed": 8864, + "лов": 8865, + "▁Maria": 8866, + "olver": 8867, + "▁Thanks": 8868, + "▁shouldn": 8869, + "▁Joh": 8870, + "▁Model": 8871, + "▁Sov": 8872, + "!'": 8873, + "Di": 8874, + "▁cancer": 8875, + "Ident": 8876, + "▁exchange": 8877, + "iller": 8878, + "inf": 8879, + "LEN": 8880, + "(){": 8881, + "aga": 8882, + "\"],": 8883, + "uh": 8884, + "▁Ken": 8885, + "▁photos": 8886, + "▁tiny": 8887, + "▁gent": 8888, + "ül": 8889, + "▁Take": 8890, + "idel": 8891, + "outing": 8892, + "Internal": 8893, + "▁cells": 8894, + "ним": 8895, + "hard": 8896, + "▁Town": 8897, + "obe": 8898, + "plex": 8899, + "тер": 8900, + "tons": 8901, + "▁concentr": 8902, + "mock": 8903, + "vc": 8904, + "áz": 8905, + "▁Championship": 8906, + "▁бе": 8907, + "??": 8908, + "éri": 8909, + "aly": 8910, + "▁Ц": 8911, + "ierte": 8912, + "▁totally": 8913, + "▁Auf": 8914, + "▁ourselves": 8915, + "▁Self": 8916, + "Forms": 8917, + "ighter": 8918, + "▁island": 8919, + "fmt": 8920, + "▁rc": 8921, + "▁tells": 8922, + "BB": 8923, + "dit": 8924, + "▁variables": 8925, + "▁intended": 8926, + "izont": 8927, + "▁plays": 8928, + "dam": 8929, + "seq": 8930, + "▁Sup": 8931, + "▁cultural": 8932, + "▁scream": 8933, + "__,": 8934, + "cipl": 8935, + "Timeout": 8936, + "▁ж": 8937, + "orte": 8938, + "▁replaced": 8939, + "EM": 8940, + "▁abandon": 8941, + "▁Special": 8942, + "ellen": 8943, + "▁Bru": 8944, + "irmed": 8945, + "Te": 8946, + "olt": 8947, + "ju": 8948, + "Argument": 8949, + "▁neut": 8950, + "scape": 8951, + "▁Ray": 8952, + "▁Polit": 8953, + "▁crowd": 8954, + "▁Windows": 8955, + "iego": 8956, + "▁escape": 8957, + "▁Apache": 8958, + "sync": 8959, + "eben": 8960, + "ifies": 8961, + "ether": 8962, + "Meta": 8963, + "▁biggest": 8964, + "Game": 8965, + "▁transaction": 8966, + "Env": 8967, + "▁Мо": 8968, + "▁plenty": 8969, + "▁mel": 8970, + "пре": 8971, + "▁motiv": 8972, + "▁ор": 8973, + "organ": 8974, + "▁mock": 8975, + "▁$_": 8976, + "ене": 8977, + "▁Number": 8978, + "cknow": 8979, + "▁Update": 8980, + "zero": 8981, + "▁surprise": 8982, + "cean": 8983, + "pdf": 8984, + "Global": 8985, + "▁attend": 8986, + "▁fond": 8987, + "▁understood": 8988, + "Nav": 8989, + "▁Mic": 8990, + "=$": 8991, + "oking": 8992, + "▁Stadium": 8993, + "Close": 8994, + "▁competition": 8995, + "▁soldiers": 8996, + "▁OP": 8997, + "agne": 8998, + "▁Anton": 8999, + "Main": 9000, + "ák": 9001, + "▁#[": 9002, + "▁Commit": 9003, + "pyx": 9004, + "▁east": 9005, + "▁Order": 9006, + "Float": 9007, + "▁accepted": 9008, + "▁monitor": 9009, + "▁pad": 9010, + "onic": 9011, + "▁pushed": 9012, + "▁replace": 9013, + "CRE": 9014, + "▁ride": 9015, + "found": 9016, + "=%": 9017, + "вой": 9018, + "▁matches": 9019, + "▁Lie": 9020, + "▁experiences": 9021, + "Pool": 9022, + "ups": 9023, + "AV": 9024, + "▁existence": 9025, + "▁thin": 9026, + "▁magn": 9027, + "COMP": 9028, + "home": 9029, + "▁ni": 9030, + "▁wurden": 9031, + "лав": 9032, + "▁teeth": 9033, + "▁Stan": 9034, + "appro": 9035, + "anny": 9036, + "ifts": 9037, + "▁unknown": 9038, + "▁homes": 9039, + "▁entity": 9040, + "cie": 9041, + "ление": 9042, + "iar": 9043, + "▁compliance": 9044, + "▁focused": 9045, + "uzz": 9046, + "=\\\"": 9047, + "components": 9048, + "Attr": 9049, + "allery": 9050, + "▁identify": 9051, + "Ok": 9052, + "pie": 9053, + "▁Still": 9054, + "▁offering": 9055, + "▁busy": 9056, + "ctl": 9057, + "itors": 9058, + "▁concerned": 9059, + "▁brown": 9060, + "clk": 9061, + "Selected": 9062, + "▁Block": 9063, + "▁egy": 9064, + "icing": 9065, + "▁URL": 9066, + "▁topic": 9067, + "▁Product": 9068, + "▁чи": 9069, + "▁trial": 9070, + "▁weekend": 9071, + "lu": 9072, + "▁IV": 9073, + "▁Egy": 9074, + "xC": 9075, + "▁nove": 9076, + "▁lett": 9077, + "enne": 9078, + "()).": 9079, + ".**": 9080, + "▁promise": 9081, + "election": 9082, + "Auth": 9083, + "rv": 9084, + "ril": 9085, + "▁conduct": 9086, + "▁maintain": 9087, + "▁boat": 9088, + "▁opposite": 9089, + "spin": 9090, + "webpack": 9091, + "anta": 9092, + "▁orient": 9093, + "▁suc": 9094, + "▁exercise": 9095, + "▁efficient": 9096, + "▁tradition": 9097, + "▁zw": 9098, + "▁Sud": 9099, + "going": 9100, + "▁Pier": 9101, + "inv": 9102, + "ipes": 9103, + "ensuremath": 9104, + "▁conver": 9105, + "creen": 9106, + "▁terror": 9107, + "▁Dou": 9108, + "▁invalid": 9109, + "ceived": 9110, + "▁Arab": 9111, + "▁wire": 9112, + "application": 9113, + "shift": 9114, + "Generic": 9115, + "▁Plan": 9116, + "▁Wall": 9117, + "▁directory": 9118, + "▁egg": 9119, + "▁wealth": 9120, + "random": 9121, + "attribute": 9122, + "▁hide": 9123, + "Serial": 9124, + "cam": 9125, + "▁ital": 9126, + "▁Line": 9127, + "▁CHECK": 9128, + "ployment": 9129, + "▁massive": 9130, + "▁extract": 9131, + "chain": 9132, + "Rest": 9133, + "▁Las": 9134, + "▁bear": 9135, + "▁links": 9136, + "▁newsp": 9137, + "▁FC": 9138, + "Card": 9139, + "aks": 9140, + "▁visible": 9141, + "▁Marc": 9142, + "▁Boston": 9143, + "▁reserved": 9144, + "▁roof": 9145, + "licenses": 9146, + "dc": 9147, + "▁Information": 9148, + "▁witness": 9149, + "Sk": 9150, + "*),": 9151, + "Scope": 9152, + "'];": 9153, + "▁Mir": 9154, + "uding": 9155, + "▁trend": 9156, + "rep": 9157, + "▁musical": 9158, + "▁neither": 9159, + "▁Creat": 9160, + "▁positions": 9161, + "LC": 9162, + "ridge": 9163, + "▁officers": 9164, + "▁violence": 9165, + "▁Tem": 9166, + "▁Sus": 9167, + "▁Way": 9168, + "After": 9169, + "acket": 9170, + "▁Sou": 9171, + "acer": 9172, + "||": 9173, + "▁remark": 9174, + "water": 9175, + "ně": 9176, + "▁Са": 9177, + "▁sed": 9178, + "Each": 9179, + "▁photograph": 9180, + "▁letters": 9181, + "▁invent": 9182, + "▁Mas": 9183, + "▁songs": 9184, + "ól": 9185, + "kind": 9186, + "▁Non": 9187, + "▁dust": 9188, + "**:": 9189, + "nabla": 9190, + ".\",": 9191, + "Lock": 9192, + "▁До": 9193, + "▁cluster": 9194, + "loss": 9195, + "▁ASSERT": 9196, + "fall": 9197, + "▁reject": 9198, + "▁Spring": 9199, + "▁wedding": 9200, + "▁grav": 9201, + "ression": 9202, + "limit": 9203, + "RES": 9204, + "]}": 9205, + "▁listed": 9206, + "▁Tele": 9207, + "hline": 9208, + "▁chief": 9209, + "MEM": 9210, + "дар": 9211, + "▁expensive": 9212, + "trace": 9213, + "▁Rog": 9214, + "▁Coll": 9215, + "▁Author": 9216, + "▁Board": 9217, + "▁Capt": 9218, + "TEXT": 9219, + "▁recon": 9220, + "esta": 9221, + "▁properly": 9222, + "▁&\\": 9223, + "leton": 9224, + "iker": 9225, + "Gu": 9226, + "▁Kom": 9227, + "oco": 9228, + "▁anymore": 9229, + "▁taste": 9230, + "▁Santa": 9231, + "gex": 9232, + "▁Secret": 9233, + "▁talent": 9234, + "▁moments": 9235, + "▁Ba": 9236, + "▁extr": 9237, + "▁Commission": 9238, + "▁modify": 9239, + "▁Figure": 9240, + "▁domin": 9241, + "▁plot": 9242, + "enger": 9243, + "utch": 9244, + "▁cities": 9245, + "▁nut": 9246, + "profile": 9247, + "▁Stat": 9248, + "▁nodes": 9249, + "▁ns": 9250, + "essages": 9251, + "impl": 9252, + "icker": 9253, + "▁examples": 9254, + "abeth": 9255, + "▁stated": 9256, + "fire": 9257, + "bul": 9258, + "▁dangerous": 9259, + "▁Pay": 9260, + "▁Gre": 9261, + "▁Monday": 9262, + "esome": 9263, + "igan": 9264, + "rund": 9265, + "prise": 9266, + "fail": 9267, + "▁Never": 9268, + "Av": 9269, + "▁linear": 9270, + "▁ul": 9271, + "WAR": 9272, + "рен": 9273, + "▁AT": 9274, + "▁dop": 9275, + "▁nou": 9276, + "Dest": 9277, + "▁claims": 9278, + "enda": 9279, + "▁crazy": 9280, + "gel": 9281, + "oggle": 9282, + "▁representation": 9283, + "inen": 9284, + "▁alternative": 9285, + "DM": 9286, + "ABILITY": 9287, + "faces": 9288, + "▁doors": 9289, + "ativ": 9290, + "Look": 9291, + "▁JSON": 9292, + "▁appearance": 9293, + "бря": 9294, + "SQL": 9295, + "▁silence": 9296, + "udo": 9297, + "▁Director": 9298, + "Statement": 9299, + "selected": 9300, + "high": 9301, + "prime": 9302, + "▁ignore": 9303, + "▁colors": 9304, + "ushing": 9305, + "▁virt": 9306, + "manager": 9307, + "▁remote": 9308, + "ło": 9309, + "small": 9310, + "▁crime": 9311, + "rb": 9312, + "▁creation": 9313, + "▁flight": 9314, + "▁Sign": 9315, + "ILE": 9316, + "▁DO": 9317, + "comment": 9318, + "▁Cost": 9319, + ".__": 9320, + "▁Cop": 9321, + "▁vom": 9322, + "▁Science": 9323, + "ления": 9324, + "oop": 9325, + "interface": 9326, + "▁WARRANTIES": 9327, + "▁Page": 9328, + "******": 9329, + "ском": 9330, + "TRUE": 9331, + "▁repeated": 9332, + "▁его": 9333, + "шо": 9334, + "▁roz": 9335, + "Pe": 9336, + "▁ISBN": 9337, + "irts": 9338, + "poses": 9339, + "})$": 9340, + "▁І": 9341, + "children": 9342, + "bles": 9343, + "ECT": 9344, + "▁iz": 9345, + "▁builder": 9346, + "▁Media": 9347, + "iat": 9348, + "▁contrast": 9349, + "”,": 9350, + "▁Link": 9351, + "▁Education": 9352, + "▁joint": 9353, + "▁external": 9354, + "▁роз": 9355, + "▁bits": 9356, + "FORM": 9357, + "erman": 9358, + "wp": 9359, + "▁Mike": 9360, + "▁Master": 9361, + "▁senior": 9362, + "▁Nav": 9363, + "▁recorded": 9364, + "eling": 9365, + "esh": 9366, + "fx": 9367, + "кан": 9368, + "▁tall": 9369, + "▁Johnson": 9370, + "▁sono": 9371, + "▁anche": 9372, + "icken": 9373, + "loop": 9374, + "iciency": 9375, + "emporary": 9376, + "▁Does": 9377, + "▁relation": 9378, + "мы": 9379, + "was": 9380, + "low": 9381, + "ichte": 9382, + "▁Jones": 9383, + "▁bedroom": 9384, + "DIS": 9385, + "▁magnet": 9386, + "▁Engine": 9387, + "▁feelings": 9388, + "GC": 9389, + "▁torn": 9390, + "▁relationships": 9391, + "▁Ре": 9392, + "▁proud": 9393, + "▁twe": 9394, + "oval": 9395, + "▁waste": 9396, + "▁reduced": 9397, + "ilton": 9398, + "BP": 9399, + "▁forgot": 9400, + "▁bodies": 9401, + "▁Haw": 9402, + "lag": 9403, + "▁www": 9404, + "door": 9405, + "▁sufficient": 9406, + "▁dollars": 9407, + "Len": 9408, + "▁talked": 9409, + "▁bond": 9410, + "▁Bor": 9411, + "}}{": 9412, + "rod": 9413, + "Password": 9414, + "quare": 9415, + "▁lights": 9416, + "eren": 9417, + "▁thirty": 9418, + "NC": 9419, + "▁TODO": 9420, + "▁respond": 9421, + "ких": 9422, + "direct": 9423, + "ação": 9424, + "▁heav": 9425, + "Media": 9426, + "exit": 9427, + "License": 9428, + "`.": 9429, + "▁mixed": 9430, + "▁desk": 9431, + "▁teaching": 9432, + "▁maj": 9433, + "▁nerv": 9434, + "inations": 9435, + "typeof": 9436, + "▁coast": 9437, + "▁же": 9438, + "▁beside": 9439, + "ummy": 9440, + "Doc": 9441, + "▁schedule": 9442, + "▁recover": 9443, + "▁Further": 9444, + "▁steel": 9445, + "boot": 9446, + "▁Perhaps": 9447, + "▁съ": 9448, + "▁Os": 9449, + "rick": 9450, + "▁Ви": 9451, + "Support": 9452, + "▁(_": 9453, + "nil": 9454, + "pis": 9455, + "xpected": 9456, + "▁processing": 9457, + "Build": 9458, + "arian": 9459, + "▁icon": 9460, + "▁CA": 9461, + "wick": 9462, + "=(": 9463, + "▁algorithm": 9464, + "▁Young": 9465, + "▁Management": 9466, + "▁ancient": 9467, + "ность": 9468, + "oti": 9469, + "▁combination": 9470, + "world": 9471, + "nn": 9472, + "▁dram": 9473, + "enabled": 9474, + "Ac": 9475, + "CCESS": 9476, + "aration": 9477, + "▁blocks": 9478, + "▁Angeles": 9479, + "▁Qual": 9480, + "▁succeed": 9481, + "network": 9482, + "▁oblig": 9483, + "springframework": 9484, + "▁Tre": 9485, + "okes": 9486, + "mun": 9487, + "▁Network": 9488, + "Del": 9489, + "▁estate": 9490, + "▁liqu": 9491, + "▁pob": 9492, + "▁dad": 9493, + "▁distinct": 9494, + "▁Tit": 9495, + "▁Lear": 9496, + "ferred": 9497, + "android": 9498, + "▁subsequ": 9499, + "▁Florida": 9500, + "subset": 9501, + "▁whisper": 9502, + "Vol": 9503, + "ulous": 9504, + "▁crew": 9505, + "▁lug": 9506, + "pid": 9507, + "ocity": 9508, + "skb": 9509, + "▁tea": 9510, + "ун": 9511, + "▁honor": 9512, + "▁Ins": 9513, + "▁gew": 9514, + "Details": 9515, + "eneath": 9516, + "atar": 9517, + "▁_{": 9518, + "amen": 9519, + "▁setup": 9520, + "Transaction": 9521, + "▁blank": 9522, + "Failed": 9523, + "job": 9524, + "▁pret": 9525, + "ße": 9526, + "loor": 9527, + "ří": 9528, + "ncia": 9529, + "▁anywhere": 9530, + "▁Light": 9531, + "▁Ak": 9532, + "BD": 9533, + "▁excited": 9534, + "agers": 9535, + "▁warning": 9536, + "▁processes": 9537, + "hu": 9538, + "▁youth": 9539, + "▁dogs": 9540, + "▁oct": 9541, + "▁nine": 9542, + "Writer": 9543, + "grid": 9544, + "▁importance": 9545, + "estic": 9546, + "▁carefully": 9547, + "master": 9548, + "▁decisions": 9549, + "▁pin": 9550, + "▁crack": 9551, + "TEST": 9552, + "▁Local": 9553, + "▁Right": 9554, + "▁vast": 9555, + "▁faster": 9556, + "▁institut": 9557, + "▁annual": 9558, + "LAN": 9559, + "▁episode": 9560, + "▁XV": 9561, + "▁delivery": 9562, + "tl": 9563, + "FP": 9564, + "circ": 9565, + "▁typically": 9566, + "igo": 9567, + "▁intel": 9568, + "nat": 9569, + "xb": 9570, + "стро": 9571, + ")-": 9572, + "▁Bal": 9573, + "▁Jos": 9574, + "▁gonna": 9575, + "▁Rest": 9576, + "jor": 9577, + "onia": 9578, + "orship": 9579, + "overy": 9580, + "LINE": 9581, + "]:": 9582, + "Queue": 9583, + "▁compare": 9584, + "▁apartment": 9585, + "▁rul": 9586, + "Dr": 9587, + "gency": 9588, + "▁obviously": 9589, + "zie": 9590, + "ycl": 9591, + "fortunately": 9592, + "▁stepped": 9593, + "▁Seg": 9594, + "▁Which": 9595, + "▁PC": 9596, + "▁ast": 9597, + "endor": 9598, + "▁permission": 9599, + "COL": 9600, + "▁TEST": 9601, + "Pay": 9602, + "ères": 9603, + "▁studied": 9604, + "▁accompl": 9605, + "role": 9606, + "Where": 9607, + "protobuf": 9608, + "metadata": 9609, + "Job": 9610, + "▁Four": 9611, + "plements": 9612, + "disable": 9613, + "▁loud": 9614, + "▁happening": 9615, + "▁Using": 9616, + "rog": 9617, + "▁depends": 9618, + "ím": 9619, + "'\\": 9620, + "▁taught": 9621, + "shared": 9622, + "▁attributes": 9623, + "▁Action": 9624, + "▁dess": 9625, + "▁houses": 9626, + "▁reset": 9627, + "▁bien": 9628, + "▁explicit": 9629, + "LOW": 9630, + "->_": 9631, + "▁PM": 9632, + "Category": 9633, + "oice": 9634, + "into": 9635, + "▁mail": 9636, + "▁authority": 9637, + "▁unable": 9638, + "filename": 9639, + "ék": 9640, + "лей": 9641, + "▁sector": 9642, + "appoint": 9643, + "▁hang": 9644, + "▁cel": 9645, + "related": 9646, + "itate": 9647, + "▁'<": 9648, + "amber": 9649, + "▁cheap": 9650, + "▁enabled": 9651, + "▁division": 9652, + "Any": 9653, + "▁hier": 9654, + "▁Head": 9655, + "ntax": 9656, + "uda": 9657, + "▁limitations": 9658, + "▁studio": 9659, + "media": 9660, + "▁circle": 9661, + "нова": 9662, + "▁laug": 9663, + "acts": 9664, + "▁Во": 9665, + "ód": 9666, + "pled": 9667, + "LOC": 9668, + "Expr": 9669, + ">:": 9670, + "▁prés": 9671, + "▁laughed": 9672, + "▁Three": 9673, + "лы": 9674, + "▁ends": 9675, + "▁fundament": 9676, + "▁inher": 9677, + "▁liv": 9678, + "bid": 9679, + "▁responsibility": 9680, + "▁checked": 9681, + "▁Pac": 9682, + "▁fault": 9683, + "▁yellow": 9684, + "▁salt": 9685, + "▁Francisco": 9686, + "▁^": 9687, + "▁ON": 9688, + "▁beauty": 9689, + "yg": 9690, + "▁Aff": 9691, + "▁Eq": 9692, + "▁magic": 9693, + "▁handler": 9694, + "xE": 9695, + "▁numerous": 9696, + "▁hole": 9697, + "▁rooms": 9698, + "cción": 9699, + "▁Arm": 9700, + "person": 9701, + "▁buildings": 9702, + "▁plate": 9703, + "bled": 9704, + "errors": 9705, + "▁Again": 9706, + "▁Default": 9707, + "▁Hard": 9708, + "tó": 9709, + "hus": 9710, + "▁dimension": 9711, + "iale": 9712, + "▁Mult": 9713, + "▁Government": 9714, + "Func": 9715, + "▁blow": 9716, + "▁rect": 9717, + "erra": 9718, + "connection": 9719, + "▁passing": 9720, + "ßen": 9721, + "phas": 9722, + "ensional": 9723, + "record": 9724, + "cohol": 9725, + "▁Harry": 9726, + "izontal": 9727, + "▁finger": 9728, + "▁younger": 9729, + "▁SC": 9730, + "operation": 9731, + "BY": 9732, + "heim": 9733, + "▁Bad": 9734, + "▁storm": 9735, + "▁Nat": 9736, + "▁buying": 9737, + "▁Sometimes": 9738, + "▁Ста": 9739, + "essed": 9740, + "▁damn": 9741, + "▁meg": 9742, + "umes": 9743, + "ünd": 9744, + "тра": 9745, + "▁silver": 9746, + "wd": 9747, + "hidden": 9748, + "ardo": 9749, + "▁communities": 9750, + "▁diet": 9751, + "otted": 9752, + "▁bat": 9753, + "ancer": 9754, + "▁fmt": 9755, + "▁Pen": 9756, + "▁til": 9757, + "Enum": 9758, + "PATH": 9759, + "▁matters": 9760, + "timeout": 9761, + "------------": 9762, + "kan": 9763, + "▁Corpor": 9764, + "=\"../../": 9765, + "▁Ale": 9766, + "hentication": 9767, + "▁complic": 9768, + "▁Security": 9769, + "OFF": 9770, + "Rad": 9771, + "apse": 9772, + "▁dance": 9773, + "▁permissions": 9774, + "▁warrant": 9775, + "▁lad": 9776, + "▁isol": 9777, + "dl": 9778, + "▁Au": 9779, + "yes": 9780, + "▁tv": 9781, + "▁provider": 9782, + "▁terrible": 9783, + "▁department": 9784, + "eral": 9785, + "▁implementation": 9786, + "SR": 9787, + "▁hearing": 9788, + "▁Kn": 9789, + "FR": 9790, + "tv": 9791, + "▁diss": 9792, + "FUN": 9793, + "▁durante": 9794, + "osis": 9795, + "▁tasks": 9796, + "▁Blo": 9797, + "вод": 9798, + "▁branch": 9799, + "▁politics": 9800, + "▁Elle": 9801, + "▁leadership": 9802, + "expr": 9803, + "▁techniques": 9804, + "prec": 9805, + "Sigma": 9806, + "imately": 9807, + "tk": 9808, + "achment": 9809, + "▁Enter": 9810, + "▁creative": 9811, + "▁зна": 9812, + "appy": 9813, + "unched": 9814, + "▁'',": 9815, + "onder": 9816, + "{-": 9817, + "NUM": 9818, + "▁narr": 9819, + "Memory": 9820, + "▁winning": 9821, + "▁Follow": 9822, + "*/\r": 9823, + "vision": 9824, + "resents": 9825, + "zione": 9826, + "▁latter": 9827, + "▁requests": 9828, + "▁margin": 9829, + "▁{\"": 9830, + "video": 9831, + "cn": 9832, + "▁Image": 9833, + "Tim": 9834, + "CONFIG": 9835, + "▁allowing": 9836, + "▁combined": 9837, + "PUT": 9838, + "▁instanceof": 9839, + "igin": 9840, + "▁pero": 9841, + "▁''": 9842, + "▁confidence": 9843, + "▁equivalent": 9844, + "pad": 9845, + "effect": 9846, + "RX": 9847, + "▁lang": 9848, + "strong": 9849, + "▁bridge": 9850, + "aya": 9851, + "▁treated": 9852, + "▁forth": 9853, + "SW": 9854, + "▁accounts": 9855, + "▁PO": 9856, + "▁listening": 9857, + "Route": 9858, + "()))": 9859, + "cpy": 9860, + "▁reform": 9861, + "▁gate": 9862, + "▁Walk": 9863, + "▁somehow": 9864, + "tf": 9865, + "▁layout": 9866, + "umin": 9867, + "▁considering": 9868, + "▁premi": 9869, + "▁Mom": 9870, + "athan": 9871, + "Gen": 9872, + "▁planet": 9873, + "amples": 9874, + "▁MO": 9875, + "shop": 9876, + "▁premier": 9877, + "▁simpl": 9878, + "▁segu": 9879, + "LY": 9880, + "Sum": 9881, + "▁tables": 9882, + "ska": 9883, + "▁ž": 9884, + "pd": 9885, + "▁sous": 9886, + "▁conference": 9887, + "▁Dat": 9888, + "Scroll": 9889, + "▁standards": 9890, + "▁гру": 9891, + "esse": 9892, + "▁citizens": 9893, + "▁occurred": 9894, + "▁democr": 9895, + "▁elev": 9896, + "▁Sem": 9897, + "ensus": 9898, + "headers": 9899, + "▁Chris": 9900, + "imento": 9901, + "kom": 9902, + "Cor": 9903, + "MIN": 9904, + "usher": 9905, + "Database": 9906, + "▁formal": 9907, + "igne": 9908, + "▁organizations": 9909, + "▁Ire": 9910, + "Xml": 9911, + "из": 9912, + "▁pray": 9913, + "▁bomb": 9914, + "▁mand": 9915, + "erts": 9916, + "▁clock": 9917, + "▁buck": 9918, + "вали": 9919, + "ensch": 9920, + "▁volt": 9921, + "▁films": 9922, + "▁plants": 9923, + "inode": 9924, + "Boolean": 9925, + "▁restaurant": 9926, + "ían": 9927, + "▁debut": 9928, + "pages": 9929, + "▁wordt": 9930, + "▁Ба": 9931, + "▁greatest": 9932, + "(\"/": 9933, + "▁copyright": 9934, + "▁rit": 9935, + "sizeof": 9936, + "Trace": 9937, + "uent": 9938, + "тур": 9939, + "▁ko": 9940, + ":\\": 9941, + "▁bigger": 9942, + "▁perfectly": 9943, + "tenance": 9944, + "MASK": 9945, + "ré": 9946, + "▁ett": 9947, + "▁nose": 9948, + "▁craft": 9949, + "iteral": 9950, + "▁discussed": 9951, + "▁Jewish": 9952, + "Cap": 9953, + "▁Unless": 9954, + "▁Jackson": 9955, + "Attributes": 9956, + "▁lunch": 9957, + "öl": 9958, + "atr": 9959, + "▁paying": 9960, + "Parse": 9961, + "()\r": 9962, + "lad": 9963, + "▁rare": 9964, + "▁[];": 9965, + "stone": 9966, + "▁unc": 9967, + "▁defense": 9968, + "}+": 9969, + "▁Global": 9970, + "▁Soviet": 9971, + "▁Australian": 9972, + "▁gli": 9973, + "variant": 9974, + "▁Ron": 9975, + "▁loan": 9976, + "Step": 9977, + "member": 9978, + "Sch": 9979, + "▁Committee": 9980, + "▁spending": 9981, + "▁Tri": 9982, + "▁Journal": 9983, + "▁sugar": 9984, + "elly": 9985, + "HTML": 9986, + "▁advent": 9987, + "wing": 9988, + "▁Whether": 9989, + "oration": 9990, + "▁NE": 9991, + "iveness": 9992, + "▁hav": 9993, + "▁conscious": 9994, + "een": 9995, + "Symbol": 9996, + "▁ку": 9997, + "Logger": 9998, + "▁Little": 9999, + "widet": 10000, + "ocation": 10001, + "pin": 10002, + "▁symmet": 10003, + "▁AD": 10004, + "▁posts": 10005, + "shal": 10006, + "▁Conf": 10007, + "▁chose": 10008, + "mal": 10009, + "ulo": 10010, + "▁Method": 10011, + "▁missed": 10012, + "Remove": 10013, + "Auto": 10014, + "VALUE": 10015, + "thlet": 10016, + "▁Force": 10017, + "pf": 10018, + "▁Я": 10019, + "late": 10020, + "▁pul": 10021, + "Pop": 10022, + "▁advanced": 10023, + "aires": 10024, + "ressed": 10025, + "AME": 10026, + "bell": 10027, + "aching": 10028, + "ić": 10029, + "echo": 10030, + "HS": 10031, + "▁funny": 10032, + "рии": 10033, + "▁eer": 10034, + "▁veget": 10035, + "▁fourth": 10036, + "cf": 10037, + "transform": 10038, + "▁grown": 10039, + "▁McC": 10040, + "site": 10041, + "▁beneath": 10042, + "▁shell": 10043, + "xd": 10044, + "Play": 10045, + "short": 10046, + "Role": 10047, + "▁religion": 10048, + "inator": 10049, + "}<": 10123, + "asp": 10124, + "ajo": 10125, + "exports": 10126, + "▁Node": 10127, + "▁jako": 10128, + "▁ya": 10129, + "▁successfully": 10130, + "▁friendly": 10131, + "buff": 10132, + "DEFAULT": 10133, + "▁pregn": 10134, + "Required": 10135, + "▁binary": 10136, + "isting": 10137, + "▁stared": 10138, + "▁circumstances": 10139, + "▁хо": 10140, + "rei": 10141, + "▁Го": 10142, + "Transform": 10143, + "cnt": 10144, + "▁Ext": 10145, + "report": 10146, + "VERSION": 10147, + "▁analy": 10148, + "▁Marg": 10149, + "▁alleg": 10150, + "builder": 10151, + "ToString": 10152, + "Layer": 10153, + "íst": 10154, + "Prop": 10155, + "▁Emp": 10156, + "}]": 10157, + "▁selling": 10158, + "▁queue": 10159, + "▁seriously": 10160, + "▁Lead": 10161, + "textit": 10162, + "testing": 10163, + "▁Пре": 10164, + "security": 10165, + "iał": 10166, + "ún": 10167, + "chip": 10168, + "▁candidate": 10169, + "▁minister": 10170, + "eria": 10171, + "▁Het": 10172, + "дин": 10173, + "▁Britain": 10174, + "▁barely": 10175, + "▁sty": 10176, + "▁Spanish": 10177, + "▁Ven": 10178, + "timer": 10179, + "ків": 10180, + "▁documents": 10181, + "('.": 10182, + "▁debug": 10183, + "▁contro": 10184, + "стоя": 10185, + "▁joy": 10186, + "Sn": 10187, + "Inv": 10188, + "▁protocol": 10189, + "▁faces": 10190, + "▁Despite": 10191, + "sed": 10192, + "Conf": 10193, + "ARG": 10194, + "▁evolution": 10195, + "▁tod": 10196, + "▁Promise": 10197, + "▁posted": 10198, + "Perm": 10199, + "bet": 10200, + "Ang": 10201, + "Just": 10202, + "▁rum": 10203, + "layer": 10204, + "▁behavi": 10205, + "ipping": 10206, + "▁dynam": 10207, + "▁scheme": 10208, + "▁proto": 10209, + ")/": 10210, + "Collections": 10211, + "riev": 10212, + "▁Click": 10213, + "▁uns": 10214, + "widetilde": 10215, + "▁remembered": 10216, + "гі": 10217, + "inates": 10218, + "▁incorpor": 10219, + "▁Description": 10220, + "▁prepare": 10221, + "▁Final": 10222, + "uation": 10223, + "▁Queen": 10224, + ">;": 10225, + "▁automatically": 10226, + "▁sharp": 10227, + "▁meat": 10228, + "ateur": 10229, + "astern": 10230, + "▁stuck": 10231, + "ASSERT": 10232, + "▁planned": 10233, + "dots": 10234, + "ookie": 10235, + "▁Histor": 10236, + "▁reviews": 10237, + "IMP": 10238, + "▁answered": 10239, + "Total": 10240, + "▁sau": 10241, + "▁Mexico": 10242, + "continue": 10243, + "▁Apple": 10244, + "likely": 10245, + "зва": 10246, + "users": 10247, + "▁identified": 10248, + "▁Lev": 10249, + "▁mol": 10250, + "▁Islam": 10251, + "▁committed": 10252, + "writ": 10253, + "бер": 10254, + "rift": 10255, + "▁interrupt": 10256, + "▁readonly": 10257, + "schema": 10258, + "Sm": 10259, + "Double": 10260, + "aza": 10261, + "▁Hal": 10262, + "Move": 10263, + "▁Series": 10264, + "inline": 10265, + "▁которы": 10266, + "soc": 10267, + "▁tent": 10268, + "▁amer": 10269, + "aki": 10270, + "▁lady": 10271, + "▁tired": 10272, + "ifi": 10273, + "▁même": 10274, + "ouver": 10275, + "▁aside": 10276, + "Did": 10277, + "',\r": 10278, + "▁bringing": 10279, + "Drawing": 10280, + "aro": 10281, + "▁Rh": 10282, + "▁Naz": 10283, + "esso": 10284, + "▁reaction": 10285, + "mitted": 10286, + "▁absolute": 10287, + "haust": 10288, + "(()": 10289, + "▁Task": 10290, + "ERS": 10291, + "▁^{": 10292, + "VD": 10293, + "▁tone": 10294, + "dist": 10295, + "vs": 10296, + "▁wheel": 10297, + "▁administration": 10298, + "▁interests": 10299, + "▁pointer": 10300, + "▁encounter": 10301, + "aver": 10302, + "▁nord": 10303, + "ket": 10304, + "▁beach": 10305, + "▁enjoyed": 10306, + "contains": 10307, + "▁append": 10308, + "Wait": 10309, + "▁squad": 10310, + "zel": 10311, + "▁medium": 10312, + "▁sending": 10313, + "▁Lady": 10314, + "ções": 10315, + "▁destination": 10316, + "nych": 10317, + "▁conflict": 10318, + "▁Ly": 10319, + "▁vul": 10320, + "▁basically": 10321, + "reated": 10322, + "black": 10323, + "ugins": 10324, + "▁calm": 10325, + "érie": 10326, + "har": 10327, + "лан": 10328, + "▁Се": 10329, + "watch": 10330, + "▁Put": 10331, + "▁dump": 10332, + "acher": 10333, + "scroll": 10334, + "▁claimed": 10335, + "▁Control": 10336, + "▁blind": 10337, + "enti": 10338, + "▁Keep": 10339, + "▁Development": 10340, + "images": 10341, + "▁tough": 10342, + "gebra": 10343, + "▁sept": 10344, + "hew": 10345, + "▁skill": 10346, + "▁Tay": 10347, + "▁któ": 10348, + "owner": 10349, + "pare": 10350, + "▁fee": 10351, + "▁continues": 10352, + "▁kan": 10353, + "bes": 10354, + "▁cha": 10355, + "ovo": 10356, + "▁Night": 10357, + "icture": 10358, + "shire": 10359, + "▁essay": 10360, + "▁suppose": 10361, + "etic": 10362, + "Art": 10363, + "acon": 10364, + "lla": 10365, + "words": 10366, + "▁comparison": 10367, + "▁BE": 10368, + "▁challenges": 10369, + "▁ol": 10370, + "citep": 10371, + "▁Foot": 10372, + "▁Such": 10373, + "▁papers": 10374, + "activ": 10375, + "quer": 10376, + "тя": 10377, + "▁То": 10378, + "ський": 10379, + "thur": 10380, + "done": 10381, + "▁shock": 10382, + "▁dedicated": 10383, + "▁correspond": 10384, + "Second": 10385, + "▁bull": 10386, + "life": 10387, + "indent": 10388, + "▁figures": 10389, + "▁Andrew": 10390, + "isp": 10391, + "▁favour": 10392, + "зда": 10393, + "▁Elect": 10394, + "Full": 10395, + "▁nearby": 10396, + "▁Register": 10397, + "Scale": 10398, + "ications": 10399, + "ин": 10400, + "▁AM": 10401, + "pair": 10402, + "▁perspective": 10403, + "▁nos": 10404, + "apa": 10405, + "ostał": 10406, + "▁Pers": 10407, + "icer": 10408, + "▁plastic": 10409, + "дов": 10410, + "ciples": 10411, + "zą": 10412, + "clos": 10413, + "▁уча": 10414, + "▁Á": 10415, + "plugin": 10416, + "▁angle": 10417, + "▁commission": 10418, + "▁funds": 10419, + "▁indu": 10420, + "▁drawn": 10421, + "ám": 10422, + "▁developing": 10423, + "▁segment": 10424, + "isme": 10425, + "scr": 10426, + "▁lies": 10427, + "▁IL": 10428, + "▁api": 10429, + "Extension": 10430, + "▁scal": 10431, + "install": 10432, + "▁Week": 10433, + "▁gentle": 10434, + "▁Canadian": 10435, + "▁dialog": 10436, + "▁articles": 10437, + "Theme": 10438, + "SM": 10439, + "▁Bul": 10440, + "▁leur": 10441, + "▁stom": 10442, + "Plugin": 10443, + "▁после": 10444, + "▁stead": 10445, + "▁ś": 10446, + "ipher": 10447, + "▁prze": 10448, + "▁draft": 10449, + "bottom": 10450, + "▁{};": 10451, + "▁stayed": 10452, + "feature": 10453, + "▁vot": 10454, + "▁fabric": 10455, + "ça": 10456, + "('#": 10457, + "rea": 10458, + "▁reput": 10459, + "▁Cir": 10460, + "▁AL": 10461, + "▁assertEquals": 10462, + "results": 10463, + "▁Cross": 10464, + "ursday": 10465, + "▁audio": 10466, + "▁gap": 10467, + "▁streets": 10468, + "▁scientific": 10469, + "platform": 10470, + "▁auss": 10471, + "▁Cro": 10472, + "▁partial": 10473, + "unc": 10474, + "▁choices": 10475, + "▁или": 10476, + "pred": 10477, + "▁heads": 10478, + "terday": 10479, + "▁Nick": 10480, + "▁weird": 10481, + "asant": 10482, + "▁represented": 10483, + "▁пи": 10484, + "DP": 10485, + "orders": 10486, + "clock": 10487, + "▁Ho": 10488, + "arters": 10489, + "Cmd": 10490, + "oga": 10491, + "Keys": 10492, + "Report": 10493, + "▁Vill": 10494, + "▁Mu": 10495, + "▁owned": 10496, + "SUCCESS": 10497, + "▁typeof": 10498, + "hdr": 10499, + "uable": 10500, + "▁neighborhood": 10501, + "▁AP": 10502, + "▁resulting": 10503, + "▁shadow": 10504, + "STRING": 10505, + "▁videos": 10506, + "лення": 10507, + "expect": 10508, + "▁Valley": 10509, + "▁goto": 10510, + "▁Sher": 10511, + "frastr": 10512, + "▁operating": 10513, + "▁это": 10514, + "▁Licensed": 10515, + "Variable": 10516, + "▁PR": 10517, + "▁Hans": 10518, + "clone": 10519, + "▁Gesch": 10520, + "▁Band": 10521, + "........": 10522, + "uing": 10523, + "▁hundreds": 10524, + "▁ок": 10525, + "▁emotional": 10526, + "▁Indust": 10527, + ")+": 10528, + "▁Egypt": 10529, + "▁franç": 10530, + "▁š": 10531, + "▁fasc": 10532, + "onto": 10533, + "▁Adam": 10534, + "▁laid": 10535, + "▁rig": 10536, + "▁detailed": 10537, + "▁implements": 10538, + "▁university": 10539, + "▁Hy": 10540, + "▁grid": 10541, + "▁regions": 10542, + "Stop": 10543, + "▁slot": 10544, + "▁angry": 10545, + "▁-=": 10546, + "▁waited": 10547, + "Vert": 10548, + "\":\"": 10549, + "▁elem": 10550, + "▁rég": 10551, + "owed": 10552, + "Member": 10553, + "▁ratio": 10554, + "isen": 10555, + "▁Lem": 10556, + "gery": 10557, + "▁cream": 10558, + "▁était": 10559, + "▁geb": 10560, + "unique": 10561, + "▁Deb": 10562, + "▁factory": 10563, + "że": 10564, + "dialog": 10565, + "▁Config": 10566, + "Sync": 10567, + "angers": 10568, + "▁governing": 10569, + "▁Hun": 10570, + "Space": 10571, + "▁jest": 10572, + "icious": 10573, + "▁emphas": 10574, + "umps": 10575, + "▁Esp": 10576, + "▁sul": 10577, + "▁historical": 10578, + "ija": 10579, + "▁lying": 10580, + "▁Steve": 10581, + "▁measures": 10582, + "osto": 10583, + "?”": 10584, + "▁pocket": 10585, + "▁Sat": 10586, + "▁pitch": 10587, + "▁natur": 10588, + "▁humans": 10589, + "▁Simon": 10590, + "adores": 10591, + "(\"\\": 10592, + "inking": 10593, + "▁expos": 10594, + "material": 10595, + "▁apparently": 10596, + "▁Camb": 10597, + "▁Box": 10598, + "▁spaces": 10599, + "exists": 10600, + "▁acting": 10601, + "ORY": 10602, + "зова": 10603, + "Good": 10604, + "ienne": 10605, + "▁Williams": 10606, + "▁fruit": 10607, + "iera": 10608, + "▁Lim": 10609, + "▁trait": 10610, + "▁artists": 10611, + "▁absor": 10612, + "rait": 10613, + "LOAD": 10614, + "▁movies": 10615, + "▁dynamic": 10616, + "asts": 10617, + "▁Integer": 10618, + "▁smoke": 10619, + "пі": 10620, + "angel": 10621, + ">(\"": 10622, + "▁instrument": 10623, + "▁fuel": 10624, + "ної": 10625, + "atalogue": 10626, + "▁serial": 10627, + "Files": 10628, + "▁bathroom": 10629, + "ilo": 10630, + "esto": 10631, + "▁pm": 10632, + "entials": 10633, + "▁Online": 10634, + "white": 10635, + "▁tips": 10636, + "▁capable": 10637, + "Fig": 10638, + "TV": 10639, + "▁он": 10640, + "ké": 10641, + "bitr": 10642, + "Mapping": 10643, + "▁tak": 10644, + "ющи": 10645, + "вля": 10646, + ")\",": 10647, + "▁Karl": 10648, + "▁Human": 10649, + "▁Pot": 10650, + "▁represents": 10651, + "▁consistent": 10652, + "_(": 10653, + "wen": 10654, + "▁Rose": 10655, + "law": 10656, + "▁FROM": 10657, + "▁begins": 10658, + "▁edit": 10659, + "▁mountain": 10660, + "▁chapter": 10661, + "▁wondered": 10662, + "▁industrial": 10663, + "▁Major": 10664, + "▁ges": 10665, + "▁directed": 10666, + "eros": 10667, + "▁Wild": 10668, + "liament": 10669, + "Book": 10670, + "username": 10671, + "hot": 10672, + "▁nam": 10673, + "▁league": 10674, + "bra": 10675, + "кон": 10676, + "▁Tal": 10677, + "▁Ва": 10678, + "▁exports": 10679, + "(@": 10680, + "▁sharing": 10681, + "▁Tro": 10682, + "ść": 10683, + "uesday": 10684, + "ylv": 10685, + "▁guitar": 10686, + "elen": 10687, + "Selection": 10688, + "▁confident": 10689, + "rypto": 10690, + "▁hors": 10691, + "editor": 10692, + "▁shoulders": 10693, + "getName": 10694, + "encing": 10695, + "SELECT": 10696, + "вши": 10697, + "▁kinds": 10698, + "▁Wel": 10699, + "▁purposes": 10700, + "Matrix": 10701, + "invalid": 10702, + "▁owners": 10703, + "▁Records": 10704, + "▁Process": 10705, + "▁chat": 10706, + "▁Dor": 10707, + "▁bin": 10708, + "redit": 10709, + "oire": 10710, + "▁Total": 10711, + "▁Family": 10712, + "ARY": 10713, + "▁bread": 10714, + "▁compre": 10715, + "▁shoes": 10716, + "▁raz": 10717, + "▁trace": 10718, + "nej": 10719, + "orted": 10720, + "hn": 10721, + "▁procedure": 10722, + "properties": 10723, + "plier": 10724, + "▁hero": 10725, + "panel": 10726, + "▁marked": 10727, + "▁worried": 10728, + "\\|": 10729, + "pts": 10730, + "▁Support": 10731, + "▁serving": 10732, + "Fail": 10733, + "▁disappoint": 10734, + "▁Scot": 10735, + "▁pleasure": 10736, + "▁judge": 10737, + "zeich": 10738, + "▁forever": 10739, + "▁Zeit": 10740, + "uous": 10741, + "inent": 10742, + "▁dw": 10743, + "▁waren": 10744, + "▁flash": 10745, + "▁troops": 10746, + "▁drugs": 10747, + "▁diam": 10748, + ".~": 10749, + "imp": 10750, + "inned": 10751, + "▁EV": 10752, + "Struct": 10753, + "▁justice": 10754, + "▁officials": 10755, + "ffff": 10756, + "▁Common": 10757, + "▁Cat": 10758, + "▁tomorrow": 10759, + "▁él": 10760, + "Texture": 10761, + "qpoint": 10762, + "▁Fried": 10763, + "▁Term": 10764, + "pgfqpoint": 10765, + "▁nem": 10766, + "norm": 10767, + "▁hardly": 10768, + "oda": 10769, + "zeta": 10770, + "emic": 10771, + "▁полу": 10772, + "▁loaded": 10773, + "kes": 10774, + "ció": 10775, + "▁fool": 10776, + "▁trick": 10777, + "▁dst": 10778, + "Find": 10779, + "▁все": 10780, + "}},": 10781, + "▁framework": 10782, + "▁merely": 10783, + "▁union": 10784, + "▁Edward": 10785, + "rif": 10786, + "Flag": 10787, + "▁crisis": 10788, + "▁finite": 10789, + "▁lol": 10790, + "▁Kim": 10791, + "ната": 10792, + "since": 10793, + "▁compat": 10794, + "▁pert": 10795, + "ibilities": 10796, + "▁también": 10797, + "ibli": 10798, + "▁teen": 10799, + "▁sympt": 10800, + "oral": 10801, + "ders": 10802, + "otte": 10803, + "при": 10804, + "▁Jane": 10805, + "▁originally": 10806, + "▁throat": 10807, + "mag": 10808, + "sup": 10809, + "uni": 10810, + "$$": 10811, + "▁Library": 10812, + "▁attacks": 10813, + "ingen": 10814, + "('/": 10815, + "▁hes": 10816, + "coin": 10817, + "ounce": 10818, + "▁Academy": 10819, + "MODULE": 10820, + "isms": 10821, + "▁Adv": 10822, + "▁Bol": 10823, + "▁incident": 10824, + ")^{": 10825, + "▁bij": 10826, + "▁Rome": 10827, + "▁Italy": 10828, + "events": 10829, + "▁Fern": 10830, + "▁ber": 10831, + "▁silent": 10832, + "▁pier": 10833, + "▁YO": 10834, + "▁plain": 10835, + "Bas": 10836, + "▁pill": 10837, + "rase": 10838, + "▁carrying": 10839, + "▁resp": 10840, + "ную": 10841, + "▁typical": 10842, + "Wrapper": 10843, + "▁gau": 10844, + "▁chemical": 10845, + "▁hal": 10846, + "throw": 10847, + "Cluster": 10848, + "▁Gab": 10849, + "▁Girl": 10850, + "quir": 10851, + "▁Arg": 10852, + "▁relief": 10853, + "▁Ве": 10854, + "dm": 10855, + "▁frustr": 10856, + "\\%": 10857, + "▁stores": 10858, + "▁bottle": 10859, + "▁Lew": 10860, + "two": 10861, + "stad": 10862, + "▁cheek": 10863, + "▁concerns": 10864, + "▁helpful": 10865, + "▁coverage": 10866, + "isi": 10867, + "ADD": 10868, + "async": 10869, + "▁approximately": 10870, + "iffer": 10871, + "hook": 10872, + "▁enum": 10873, + "ová": 10874, + "▁evil": 10875, + "▁constantly": 10876, + "apply": 10877, + "▁siè": 10878, + "▁practices": 10879, + "▁teachers": 10880, + "▁Sn": 10881, + "▁Awards": 10882, + "▁substant": 10883, + "▁$.": 10884, + "dk": 10885, + "▁mob": 10886, + "▁ingred": 10887, + "vere": 10888, + "Multi": 10889, + "пер": 10890, + "stal": 10891, + "yard": 10892, + "required": 10893, + "vement": 10894, + "▁intelligence": 10895, + "▁thinks": 10896, + "▁personally": 10897, + "▁trained": 10898, + "orney": 10899, + ")\\": 11266, + "anal": 11267, + "Section": 11268, + "plus": 11269, + "üt": 11270, + "▁embed": 11271, + "▁strings": 11272, + "Before": 11273, + "proc": 11274, + "▁спо": 11275, + "trl": 11276, + "vr": 11277, + "Background": 11278, + "logger": 11279, + "agraph": 11280, + "iest": 11281, + "▁goods": 11282, + "batch": 11283, + "▁optional": 11284, + "▁Taylor": 11285, + "▁recognize": 11286, + "walk": 11287, + "▁Hit": 11288, + "▁Elizabeth": 11289, + "}:": 11290, + "▁careful": 11291, + "краї": 11292, + "▁locations": 11293, + "▁structures": 11294, + "▁disk": 11295, + "▁ships": 11296, + "▁suo": 11297, + "▁sowie": 11298, + "▁Ess": 11299, + "▁Hash": 11300, + "▁reasonable": 11301, + "▁Moreover": 11302, + "▁formula": 11303, + "▁Centre": 11304, + "▁residents": 11305, + "RS": 11306, + "Ids": 11307, + "▁Know": 11308, + "▁trib": 11309, + "▁rés": 11310, + "▁stable": 11311, + "▁Would": 11312, + "▁breaking": 11313, + "▁meal": 11314, + "▁phen": 11315, + "▁fel": 11316, + "▁Fred": 11317, + "Author": 11318, + "▁capture": 11319, + "opts": 11320, + "▁everywhere": 11321, + "▁sque": 11322, + "▁moder": 11323, + "setup": 11324, + "▁Supp": 11325, + "▁whenever": 11326, + "{(": 11327, + "wart": 11328, + "▁toe": 11329, + "Prefix": 11330, + "hou": 11331, + "gage": 11332, + ">\"": 11333, + "▁frag": 11334, + "▁Theorem": 11335, + "memory": 11336, + "▁contents": 11337, + "docs": 11338, + "}'": 11339, + "▁Irish": 11340, + "Then": 11341, + "aats": 11342, + "Save": 11343, + "▁agency": 11344, + "▁име": 11345, + "дова": 11346, + "▁Function": 11347, + "NN": 11348, + "destroy": 11349, + "▁Message": 11350, + "▁cancel": 11351, + "▁superior": 11352, + "▁ec": 11353, + "▁literature": 11354, + "▁PART": 11355, + "Il": 11356, + "▁Cab": 11357, + "engine": 11358, + "▁basket": 11359, + "worth": 11360, + "▁Sel": 11361, + "fetch": 11362, + "▁Stadt": 11363, + "▁Ки": 11364, + "▁conj": 11365, + "▁seiner": 11366, + "▁confirmed": 11367, + "▁Argent": 11368, + "amar": 11369, + "pgfpath": 11370, + "▁struggle": 11371, + "Pattern": 11372, + "▁Middle": 11373, + "itan": 11374, + "▁moon": 11375, + "orough": 11376, + "▁Catholic": 11377, + "▁struck": 11378, + "]->": 11379, + "▁weapon": 11380, + "▁subst": 11381, + "▁instructions": 11382, + "▁occas": 11383, + "protected": 11384, + "▁Less": 11385, + "▁batch": 11386, + "▁contra": 11387, + "▁deck": 11388, + "▁ignored": 11389, + "▁refused": 11390, + "trigger": 11391, + "▁criminal": 11392, + "GA": 11393, + "olly": 11394, + "▁Bell": 11395, + "▁Ю": 11396, + "forward": 11397, + "▁prefix": 11398, + "▁immediate": 11399, + "▁assigned": 11400, + "▁elected": 11401, + "▁tonight": 11402, + "▁Dies": 11403, + "▁Beach": 11404, + "▁preced": 11405, + "ował": 11406, + "▁galax": 11407, + "▁logic": 11408, + "enza": 11409, + "▁Captain": 11410, + "▁Hay": 11411, + "▁facts": 11412, + "▁ни": 11413, + "té": 11414, + "▁sb": 11415, + "oped": 11416, + "▁combat": 11417, + "▁explore": 11418, + "▁(-": 11419, + "Loader": 11420, + "▁Wilson": 11421, + "▁locked": 11422, + ":)": 12970, + "▁quel": 12971, + "▁Га": 12972, + "Ty": 12973, + "▁temps": 12974, + "▁ghost": 12975, + "Material": 12976, + "ERCHANT": 12977, + "pointer": 12978, + "жда": 12979, + "aha": 12980, + "ulf": 12981, + "▁supplement": 12982, + "▁dismiss": 12983, + "▁closing": 12984, + "▁vulner": 12985, + "▁après": 12986, + "▁overwhel": 12987, + "ское": 12988, + "▁disag": 12989, + "acia": 12990, + "oured": 12991, + "ruption": 12992, + "▁PS": 12993, + "Endpoint": 12994, + "Real": 12995, + "▁Tag": 12996, + "▁stairs": 12997, + "lyn": 12998, + "▁eleg": 12999, + "▁veter": 13000, + "factory": 13001, + "anne": 13002, + "▁Bat": 13003, + "▁franc": 13004, + "lung": 13005, + "▁\"'": 13006, + ".',": 13007, + "▁Country": 13008, + "^{[": 13009, + "▁yours": 13010, + "ailability": 13011, + "Clear": 13012, + "ätt": 13013, + "пис": 13014, + "▁joke": 13015, + "▁annoy": 13016, + "▁rag": 13017, + "vari": 13018, + "лекс": 13019, + "▁Psy": 13020, + "ilty": 13021, + "mount": 13022, + "▁cual": 13023, + "▁solar": 13024, + "}^{(": 13025, + "Short": 13026, + "▁taxes": 13027, + "Append": 13028, + "Win": 13029, + "estyle": 13030, + "▁facil": 13031, + "вро": 13032, + "▁sought": 13033, + "▁bare": 13034, + "▁react": 13035, + "jar": 13036, + "MAC": 13037, + "lov": 13038, + "warn": 13039, + "▁crucial": 13040, + "▁museum": 13041, + "ниц": 13042, + "▁Kent": 13043, + "Maybe": 13044, + "▁bike": 13045, + "▁Address": 13046, + "XML": 13047, + "▁admitted": 13048, + "▁$(\\": 13049, + "▁spell": 13050, + "▁vic": 13051, + "gre": 13052, + "▁proc": 13053, + "theless": 13054, + "▁Nom": 13055, + "▁Rail": 13056, + "▁acceler": 13057, + "▁convin": 13058, + "▁Property": 13059, + "▁DA": 13060, + "▁clip": 13061, + "▁plugin": 13062, + "Limit": 13063, + "views": 13064, + "bru": 13065, + "▁pra": 13066, + "▁ak": 13067, + "▁ej": 13068, + "▁opts": 13069, + "▁slip": 13070, + "▁gang": 13071, + "asted": 13072, + "uals": 13073, + "▁dying": 13074, + "Coll": 13075, + "ammen": 13076, + "▁Policy": 13077, + "ERCHANTABILITY": 13078, + "▁Collection": 13079, + "▁vec": 13080, + "▁Dick": 13081, + "stud": 13082, + "▁layers": 13083, + "▁tied": 13084, + "}\\\\": 13085, + "▁alors": 13086, + "▁jou": 13087, + "▁chicken": 13088, + "▁permanent": 13089, + "▁Everything": 13090, + "▁Low": 13091, + "▁Cook": 13092, + "▁peak": 13093, + "▁PARTICULAR": 13094, + "▁dear": 13095, + "ič": 13096, + "▁introduce": 13097, + "▁causing": 13098, + "писа": 13099, + "Bound": 13100, + "hund": 13101, + "multi": 13102, + "▁pare": 13103, + "annt": 13104, + "▁breat": 13105, + "▁commitment": 13106, + "▁increasingly": 13107, + "кой": 13108, + "▁Friend": 13109, + "▁statistics": 13110, + "▁Manager": 13111, + "plicate": 13112, + "Cloud": 13113, + "aci": 13114, + "▁Conference": 13115, + "Span": 13116, + "▁CEO": 13117, + "▁Wait": 13118, + "▁Ober": 13119, + "ifting": 13120, + "imiento": 13121, + "getElement": 13122, + "▁gle": 13123, + "лия": 13124, + "▁wieder": 13125, + "▁instruction": 13126, + "gly": 13127, + "▁blame": 13128, + "▁listade": 13129, + "▁aapt": 13130, + "▁Lewis": 13131, + "Fragment": 13132, + "▁gear": 13133, + "mill": 13134, + "prod": 13135, + "▁burning": 13136, + "ється": 13137, + "▁mé": 13138, + "ène": 13139, + "▁complicated": 13140, + "bh": 13141, + "▁Justice": 13142, + "▁tested": 13143, + "▁staring": 13144, + "▁survive": 13145, + "▁cous": 13146, + "▁rib": 13147, + "aml": 13148, + "▁Trust": 13149, + "▁cad": 13150, + "▁Terr": 13151, + "▁mapping": 13152, + "▁twelve": 13153, + "▁grant": 13154, + "▁thorough": 13155, + "▁Ü": 13156, + "▁folks": 13157, + "▁Content": 13158, + "▁childhood": 13159, + "cker": 13160, + "сно": 13161, + "RECT": 13162, + "▁finale": 13163, + "▁shower": 13164, + "éric": 13165, + "▁spat": 13166, + "odge": 13167, + "рь": 13168, + "▁pes": 13169, + "eda": 13170, + "Db": 13171, + "▁Antonio": 13172, + "▁engaged": 13173, + "▁vess": 13174, + "vals": 13175, + "▁electronic": 13176, + "lemma": 13177, + "▁Wy": 13178, + "mad": 13179, + "merge": 13180, + "apon": 13181, + "▁privile": 13182, + "▁novembre": 13183, + "▁Sports": 13184, + "will": 13185, + "▁controls": 13186, + "▁categories": 13187, + "▁Georgia": 13188, + "ipedia": 13189, + "▁AV": 13190, + "atori": 13191, + "▁___": 13192, + "▁À": 13193, + "▁Ryan": 13194, + "▁Charlie": 13195, + "▁исто": 13196, + "▁emotion": 13197, + "▁cooking": 13198, + "▁attempts": 13199, + "▁FITNESS": 13200, + "äter": 13201, + "Enable": 13202, + "DT": 13203, + "▁Change": 13204, + "AspNet": 13205, + "▁га": 13206, + "▁ordinary": 13207, + "▁SQL": 13208, + "plane": 13209, + "%.": 13210, + "▁Summer": 13211, + "▁avait": 13212, + "upp": 13213, + "▁illness": 13214, + "UINT": 13215, + ">{": 13216, + "▁zwischen": 13217, + "▁hardware": 13218, + "▁sounded": 13219, + "equiv": 13220, + "▁piano": 13221, + "uset": 13222, + "kn": 13223, + "TRY": 13224, + "▁bab": 13225, + "нен": 13226, + "▁reliable": 13227, + "▁Bronnen": 13228, + "▁Store": 13229, + "Az": 13230, + "▁»,": 13231, + "Static": 13232, + "dw": 13233, + "green": 13234, + "▁'';": 13235, + "lij": 13236, + "eva": 13237, + "ній": 13238, + "▁Syd": 13239, + "inois": 13240, + "convert": 13241, + "▁declare": 13242, + "bres": 13243, + "INK": 13244, + "itled": 13245, + "▁accord": 13246, + "▁mars": 13247, + "Sequence": 13248, + "zip": 13249, + "▁Brazil": 13250, + "▁meetings": 13251, + "▁accuracy": 13252, + "▁Machine": 13253, + "▁autor": 13254, + "▁ainsi": 13255, + "Simple": 13256, + "Resources": 13257, + "каза": 13258, + "▁MP": 13259, + "they": 13260, + "▁Bang": 13261, + "▁eing": 13262, + "ateful": 13263, + "▁Something": 13264, + "▁upset": 13265, + "History": 13266, + "dimensional": 13267, + "▁explanation": 13268, + "▁civ": 13269, + "▁conce": 13270, + "▁köz": 13271, + "▁promised": 13272, + "жду": 13273, + "wed": 13274, + "Fore": 13275, + "Amount": 13276, + "abb": 13277, + "▁clothing": 13278, + "лись": 13279, + "oen": 13280, + "▁Print": 13281, + "▁sizes": 13282, + "▁banks": 13283, + "ribed": 13284, + "▁'../": 13285, + "FIX": 13286, + "▁Hug": 13287, + "▁zn": 13288, + "▁INT": 13289, + "▁instances": 13290, + "▁alongside": 13291, + "Namespace": 13292, + "▁renew": 13293, + "▁asc": 13294, + "▁waves": 13295, + "▁pom": 13296, + "Duration": 13297, + "days": 13298, + "$(": 13299, + "▁grabbed": 13300, + "▁surgery": 13301, + "▁restore": 13302, + "Normal": 13303, + "▁Leb": 13304, + "▁analyt": 13305, + "Literal": 13306, + "HA": 13307, + "▁shares": 13308, + "illet": 13309, + "ols": 13310, + "▁Dog": 13311, + "orno": 13312, + "▁manip": 13313, + "jav": 13314, + "▁essentially": 13315, + "▁casual": 13316, + "opl": 13317, + "▁р": 13318, + "▁SU": 13319, + "▁engineering": 13320, + "▁Prime": 13321, + "▁SW": 13322, + "▁reaching": 13323, + "▁вла": 13324, + "▁Росси": 13325, + "▁Kre": 13326, + "erry": 13327, + "▁oppon": 13328, + "program": 13329, + "emper": 13330, + "isEmpty": 13331, + "▁Unit": 13332, + "INTER": 13333, + "ethe": 13334, + "zd": 13335, + "CUR": 13336, + "▁vm": 13337, + "conv": 13338, + "ropol": 13339, + "▁Coast": 13340, + "▁Select": 13341, + "▁была": 13342, + "▁Ve": 13343, + "owy": 13344, + "▁myth": 13345, + "ceptions": 13346, + "classes": 13347, + "▁worden": 13348, + "▁assault": 13349, + "▁dual": 13350, + "ORK": 13351, + "▁inches": 13352, + "▁FA": 13353, + "▁Station": 13354, + "▁personality": 13355, + "▁scar": 13356, + "▁regime": 13357, + "▁noten": 13358, + "▁rural": 13359, + "iza": 13360, + "Audio": 13361, + "▁disput": 13362, + "▁aver": 13363, + "▁obst": 13364, + "▁Region": 13365, + "utf": 13366, + "▁Cass": 13367, + "hspace": 13368, + "▁shipping": 13369, + "iko": 13370, + "icked": 13371, + "numer": 13372, + "дна": 13373, + "riel": 13374, + "disabled": 13375, + "opol": 13376, + "looking": 13377, + "▁classical": 13378, + "▁constructed": 13379, + "▁referenties": 13380, + "]+": 13381, + "▁captured": 13382, + "▁minimal": 13383, + "▁sock": 13384, + "father": 13385, + "isión": 13386, + "▁equally": 13387, + "▁reduction": 13388, + "Ant": 13389, + "aison": 13390, + "▁argue": 13391, + "circle": 13392, + "▁toler": 13393, + "}\",": 13394, + "▁primarily": 13395, + "usal": 13396, + "▁algebra": 13397, + "▁gathered": 13398, + "▁Remember": 13399, + "_);": 13400, + "UTE": 13401, + "▁Kit": 13402, + "Sy": 13403, + "HEAD": 13404, + "▁recipe": 13405, + "▁scenario": 13406, + "▁Following": 13407, + "VAR": 13408, + "▁yard": 13409, + "▁stad": 13410, + "*(": 13411, + "▁validate": 13412, + "DEX": 13413, + "▁committee": 13414, + "▁temporary": 13415, + "▁consequences": 13416, + "▁également": 13417, + "ктив": 13418, + "▁ra": 13419, + "▁displ": 13420, + "▁apps": 13421, + "▁Teil": 13422, + "▁».": 13423, + "▁adopted": 13424, + "tensor": 13425, + "▁femin": 13426, + "▁мар": 13427, + "логи": 13428, + "tech": 13429, + "▁Rot": 13430, + "▁knees": 13431, + "phys": 13432, + "owej": 13433, + "▁Oxford": 13434, + "анд": 13435, + "hell": 13436, + "ografia": 13437, + "▁exposed": 13438, + "ktop": 13439, + "oby": 13440, + "lower": 13441, + "▁Senate": 13442, + "▁sword": 13443, + "Flow": 13444, + "▁Unfortunately": 13445, + "▁boxes": 13446, + "▁cuando": 13447, + "▁pilot": 13448, + "▁Album": 13449, + "Bal": 13450, + "Sort": 13451, + "FIELD": 13452, + "▁desert": 13453, + "COMM": 13454, + "rons": 13455, + "adows": 13456, + "▁loyal": 13457, + "▁asset": 13458, + "▁mud": 13459, + "фа": 13460, + "▁secondary": 13461, + "▁Ар": 13462, + "▁cul": 13463, + "▁Asian": 13464, + "▁staying": 13465, + "▁dataset": 13466, + "▁USE": 13467, + "▁loves": 13468, + "▁velocity": 13469, + "áv": 13470, + "▁purchased": 13471, + "SOC": 13472, + "▁competitive": 13473, + "▁Football": 13474, + "iska": 13475, + "▁knock": 13476, + "stairs": 13477, + "azy": 13478, + "▁vend": 13479, + "▁arts": 13480, + "▁Bras": 13481, + "uela": 13482, + "кто": 13483, + "trim": 13484, + "▁dirty": 13485, + "▁websites": 13486, + "▁Indep": 13487, + "▁стра": 13488, + "sr": 13489, + "▁ticket": 13490, + "atile": 13491, + "▁implemented": 13492, + "▁время": 13493, + "▁bowl": 13494, + "DATE": 13495, + "▁alter": 13496, + "▁Space": 13497, + "▁accompan": 13498, + "ordon": 13499, + "▁doctors": 13500, + "istas": 13501, + "Cast": 13502, + "дом": 13503, + "CTL": 13504, + "urers": 13505, + "▁ingredients": 13506, + "▁calculated": 13507, + "▁leather": 13508, + "▁sensitive": 13509, + "▁suspic": 13510, + "stan": 13511, + "▁anni": 13512, + "await": 13513, + "▁Franç": 13514, + "▁abort": 13515, + "▁Spirit": 13516, + "▁Walter": 13517, + "unkt": 13518, + "▁vertical": 13519, + "ORS": 13520, + "best": 13521, + "▁Client": 13522, + "itated": 13523, + "▁ва": 13524, + "▁Č": 13525, + "▁ville": 13526, + "▁diplom": 13527, + "orne": 13528, + "▁bars": 13529, + "Uri": 13530, + "APTER": 13531, + "pons": 13532, + "utz": 13533, + "Proto": 13534, + "▁stir": 13535, + "▁це": 13536, + "▁primer": 13537, + "igible": 13538, + "extra": 13539, + "▁Books": 13540, + "▁Bos": 13541, + "▁Et": 13542, + "▁Welt": 13543, + "▁Korea": 13544, + "рито": 13545, + "▁vibr": 13546, + "Self": 13547, + "linear": 13548, + "об": 13549, + "▁Lang": 13550, + "▁deeper": 13551, + "▁termin": 13552, + "enschaft": 13553, + "▁році": 13554, + "ammed": 13555, + "visible": 13556, + "▁IOException": 13557, + "▁Wind": 13558, + "usqu": 13559, + "▁Stop": 13560, + "▁орга": 13561, + "INVALID": 13562, + "▁cub": 13563, + "▁jew": 13564, + "▁captain": 13565, + "зі": 13566, + "chunk": 13567, + "apture": 13568, + "ashboard": 13569, + "▁divided": 13570, + "▁extensive": 13571, + "▁suffer": 13572, + "▁heading": 13573, + "created": 13574, + "▁quietly": 13575, + "▁ny": 13576, + "▁пол": 13577, + "\"+": 13578, + "ikan": 13579, + "▁designs": 13580, + "zu": 13581, + "}+\\": 13582, + "Operator": 13583, + "▁Lemma": 13584, + "▁нау": 13585, + "acji": 13586, + "лове": 13587, + "Servlet": 13588, + "▁Kevin": 13589, + "stage": 13590, + "bn": 13591, + "textwidth": 13592, + "failed": 13593, + "▁Staff": 13594, + "▁enem": 13595, + "unde": 13596, + "ень": 13597, + "Packet": 13598, + "▁Als": 13599, + "kar": 13600, + "]['": 13601, + "ked": 13602, + "Pers": 13603, + ">::": 13604, + "▁arc": 13605, + "▁synt": 13606, + "SPE": 13607, + "▁Да": 13608, + "▁Mi": 13609, + "▁Moh": 13610, + "▁Death": 13611, + "browser": 13612, + "▁Dave": 13613, + "▁succ": 13614, + "toggle": 13615, + "▁tack": 13616, + "Comment": 13617, + "eron": 13618, + "▁awareness": 13619, + "▁hug": 13620, + "▁contemporary": 13621, + "ulating": 13622, + "▁Title": 13623, + "▁THIS": 13624, + "havior": 13625, + "rank": 13626, + "▁dozen": 13627, + "▁cheese": 13628, + "coln": 13629, + "▁radius": 13630, + "▁dimensions": 13631, + "roduction": 13632, + "▁adds": 13633, + "▁household": 13634, + "▁Davis": 13635, + "pkg": 13636, + "{$": 13637, + "▁casino": 13638, + "▁Pierre": 13639, + "▁objective": 13640, + "train": 13641, + "▁Michigan": 13642, + "payload": 13643, + "▁rug": 13644, + "▁severe": 13645, + "mean": 13646, + "▁toss": 13647, + "▁embarrass": 13648, + "▁Very": 13649, + "▁appeal": 13650, + "▁Comput": 13651, + "▁forgotten": 13652, + "▁kernel": 13653, + "▁carbon": 13654, + "fw": 13655, + "▁Су": 13656, + "▁Empire": 13657, + "▁quote": 13658, + "etz": 13659, + "▁mini": 13660, + "▁pipe": 13661, + "▁nous": 13662, + "▁Move": 13663, + "▁ду": 13664, + "▁nervous": 13665, + "▁Мар": 13666, + "*\r": 13667, + "▁Bush": 13668, + "▁peer": 13669, + "▁Writ": 13670, + "▁satisfied": 13671, + "▁pulling": 13672, + "▁Pur": 13673, + "▁Miller": 13674, + "▁FL": 13675, + "amaz": 13676, + "▁mile": 13677, + "▁Need": 13678, + "▁supplies": 13679, + "▁año": 13680, + "▁pace": 13681, + "▁Victoria": 13682, + "▁ought": 13683, + "▁Player": 13684, + "agnostic": 13685, + "▁viv": 13686, + "▁Patrick": 13687, + "▁Š": 13688, + "▁Story": 13689, + "aca": 13690, + "▁mountains": 13691, + "CLASS": 13692, + "▁fragment": 13693, + "▁settlement": 13694, + "▁Furthermore": 13695, + "▁drivers": 13696, + "▁Ju": 13697, + "▁были": 13698, + "Rows": 13699, + "▁impression": 13700, + "▁infer": 13701, + "▁Expl": 13702, + "olute": 13703, + "ovan": 13704, + "arance": 13705, + "CAP": 13706, + "▁enforce": 13707, + "▁Burn": 13708, + "Reset": 13709, + "mother": 13710, + "▁Battle": 13711, + "padding": 13712, + "iate": 13713, + "▁cried": 13714, + "AK": 13715, + "uns": 13716, + "▁siècle": 13717, + "▁Contin": 13718, + "bank": 13719, + "junit": 13720, + "objects": 13721, + "Rot": 13722, + "issa": 13723, + "▁begun": 13724, + "*-": 13725, + "▁visiting": 13726, + "жде": 13727, + "▁targets": 13728, + "▁Latin": 13729, + "ут": 13730, + "▁Esc": 13731, + "*;": 13732, + "ång": 13733, + "▁({": 13734, + "▁diagram": 13735, + "Models": 13736, + "▁partnership": 13737, + "▁från": 13738, + "ulty": 13739, + "Pod": 13740, + "CALL": 13741, + "modal": 13742, + "sig": 13743, + "itzer": 13744, + "itel": 13745, + "▁convinced": 13746, + "abl": 13747, + "стве": 13748, + "▁cot": 13749, + "▁repeat": 13750, + "▁lists": 13751, + "sound": 13752, + "▁royal": 13753, + "▁grace": 13754, + "▁oraz": 13755, + "Notification": 13756, + "prite": 13757, + "▁arrival": 13758, + "ancell": 13759, + "hentic": 13760, + "decode": 13761, + "▁fantastic": 13762, + "progress": 13763, + "proxy": 13764, + "ző": 13765, + "kel": 13766, + "▁convenient": 13767, + "aque": 13768, + "riet": 13769, + "▁Digital": 13770, + "iors": 13771, + "▁Budd": 13772, + "andra": 13773, + "addy": 13774, + "▁overs": 13775, + "▁consumers": 13776, + "pn": 13777, + "mouse": 13778, + "▁BC": 13779, + "deg": 13780, + "perm": 13781, + "ités": 13782, + "▁испо": 13783, + "heast": 13784, + "hour": 13785, + "PARAM": 13786, + "conscious": 13787, + "▁wing": 13788, + "▁atmosphere": 13789, + "▁gig": 13790, + "▁contre": 13791, + "▁drama": 13792, + "ят": 13793, + "▁Front": 13794, + "▁philosophy": 13795, + "▁Hart": 13796, + "▁nurs": 13797, + "uras": 13798, + "▁Tru": 13799, + "▁sud": 13800, + "▁performing": 13801, + "пы": 13802, + "▁confused": 13803, + "▁checks": 13804, + "amt": 13805, + "Make": 13806, + "▁RO": 13807, + "▁df": 13808, + "izations": 13809, + "▁degli": 13810, + "▁architecture": 13811, + "Renderer": 13812, + "▁Ла": 13813, + "▁ptr": 13814, + "▁dieser": 13815, + "submit": 13816, + "▁topics": 13817, + "▁principles": 13818, + "vars": 13819, + "sock": 13820, + "▁tongue": 13821, + "▁percentage": 13822, + "▁SS": 13823, + "▁dol": 13824, + "▁rice": 13825, + "ío": 13826, + "▁Eastern": 13827, + "▁recognition": 13828, + "▁Ern": 13829, + "▁Ut": 13830, + "▁caut": 13831, + "▁Cloud": 13832, + "▁conversion": 13833, + "▁Ohio": 13834, + "▁ME": 13835, + "▁surely": 13836, + "▁gard": 13837, + "puis": 13838, + "▁urg": 13839, + "imi": 13840, + "▁absence": 13841, + "▁winner": 13842, + "Language": 13843, + "▁HTTP": 13844, + "wt": 13845, + "▁translation": 13846, + "сс": 13847, + "▁Kind": 13848, + "Two": 13849, + "▁Revolution": 13850, + "Insert": 13851, + "Every": 13852, + "orient": 13853, + "▁тра": 13854, + "▁emotions": 13855, + "details": 13856, + "▁flu": 13857, + "▁operate": 13858, + "Ag": 13859, + "unning": 13860, + "▁partie": 13861, + "tri": 13862, + "▁golden": 13863, + "▁Би": 13864, + "▁foundation": 13865, + "isten": 13866, + "▁Carlos": 13867, + "Children": 13868, + "▁neighb": 13869, + "▁Cart": 13870, + "Begin": 13871, + "гда": 13872, + "▁scheduled": 13873, + "'>": 13874, + "▁observations": 13875, + "▁producer": 13876, + "athers": 13877, + "ному": 13878, + "▁expectations": 13879, + "oso": 13880, + "zh": 13881, + "mutable": 13882, + "▁writes": 13883, + "▁pushing": 13884, + "▁seats": 13885, + "▁breast": 13886, + "aping": 13887, + "▁Simple": 13888, + "▁socket": 13889, + "▁slave": 13890, + "iley": 13891, + "▁assistant": 13892, + "▁trim": 13893, + "▁landscape": 13894, + "▁association": 13895, + "quant": 13896, + "▁Palest": 13897, + "▁sweat": 13898, + "engers": 13899, + "?_": 13900, + "ép": 13901, + ">.": 13902, + "▁curious": 13903, + "▁Component": 13904, + "▁replacement": 13905, + "раль": 13906, + "▁Track": 13907, + "▁Remove": 13908, + "▁Size": 13909, + "peror": 13910, + "▁calculate": 13911, + "▁sessions": 13912, + "▁typed": 13913, + "▁submit": 13914, + "!!!": 13915, + "▁partition": 13916, + "eding": 13917, + "-----": 13918, + "azioni": 13919, + "ließ": 13920, + "onal": 13921, + "▁shru": 13922, + "▁REG": 13923, + "▁Fac": 13924, + "configuration": 13925, + "▁было": 13926, + "▁Among": 13927, + "__);": 13928, + "▁Server": 13929, + "▁LOG": 13930, + "▁cand": 13931, + "']);": 13932, + "gov": 13933, + "▁Six": 13934, + "undefined": 13935, + "▁ty": 13936, + "asa": 13937, + "▁particles": 13938, + "▁фор": 13939, + "``": 13940, + "Tube": 13941, + "eland": 13942, + "fold": 13943, + "ogo": 13944, + "▁approaches": 13945, + "onda": 13946, + "agr": 13947, + ",$": 13948, + "▁{{": 13949, + "▁Modern": 13950, + "▁Winter": 13951, + "available": 13952, + "▁Lud": 13953, + "▁casa": 13954, + "▁Could": 13955, + "▁fifteen": 13956, + "▁potentially": 13957, + "^^": 13958, + "▁seit": 13959, + "Animation": 13960, + "кого": 13961, + "Zone": 13962, + "elif": 13963, + "▁acknowled": 13964, + "▁ownership": 13965, + "▁describes": 13966, + "▁reverse": 13967, + "▁contest": 13968, + "▁scored": 13969, + "▁opposed": 13970, + "flex": 13971, + "kre": 13972, + "▁merge": 13973, + "▁covering": 13974, + "▁honestly": 13975, + "▁Mess": 13976, + "▁rarely": 13977, + "▁incredible": 13978, + "itage": 13979, + "▁victims": 13980, + "ными": 13981, + "wl": 13982, + "izza": 13983, + "dn": 13984, + "onde": 13985, + "▁przy": 13986, + "▁HTML": 13987, + "▁payload": 13988, + "Bus": 13989, + "usb": 13990, + "Fn": 13991, + "▁displayed": 13992, + "▁ocean": 13993, + "▁Avenue": 13994, + "acion": 13995, + "ghan": 13996, + "metric": 13997, + "ieties": 13998, + "▁attractive": 13999, + "▁fö": 14000, + "Creat": 14001, + "verter": 14002, + "▁Alice": 14003, + "пол": 14004, + "▁fraction": 14005, + "▁behaviour": 14006, + "▁Jersey": 14007, + "▁revenue": 14008, + "▁tres": 14009, + "ILD": 14010, + "▁Ét": 14011, + "▁sync": 14012, + "wich": 14013, + "▁ancest": 14014, + "ът": 14015, + "omo": 14016, + "▁Ide": 14017, + "▁gained": 14018, + "▁momentum": 14019, + "▁Ko": 14020, + "ieu": 14021, + "ielt": 14022, + "▁bonus": 14023, + "▁texture": 14024, + "Modal": 14025, + "NEXT": 14026, + "▁године": 14027, + "▁languages": 14028, + "vt": 14029, + "▁representing": 14030, + "▁Dream": 14031, + "curr": 14032, + "qual": 14033, + "▁js": 14034, + "burn": 14035, + "▁contributions": 14036, + "▁ric": 14037, + "}-\\": 14038, + "={{": 14039, + "cart": 14040, + "FB": 14041, + "jud": 14042, + "esp": 14043, + "▁electron": 14044, + "▁ell": 14045, + "▁Runtime": 14046, + "achel": 14047, + "\\_": 14048, + "week": 14049, + "packet": 14050, + "▁Secretary": 14051, + "▁Jahrhund": 14052, + "▁threshold": 14053, + "bage": 14054, + "▁concer": 14055, + "▁bone": 14056, + "▁Hollywood": 14057, + "Cursor": 14058, + "▁awarded": 14059, + "▁summary": 14060, + "aggio": 14061, + "▁stell": 14062, + "▁flesh": 14063, + "Pair": 14064, + "▁Age": 14065, + "ington": 14066, + "▁'.": 14067, + "aser": 14068, + "кова": 14069, + "▁quart": 14070, + "ryption": 14071, + "Alloc": 14072, + "ften": 14073, + "Operand": 14074, + "▁indicated": 14075, + "($_": 14076, + "getString": 14077, + "▁listener": 14078, + "spir": 14079, + ")_": 14080, + "vens": 14081, + "▁foods": 14082, + "anza": 14083, + "teil": 14084, + "DESC": 14085, + "▁notion": 14086, + "▁employment": 14087, + "▁swing": 14088, + "nbsp": 14089, + "▁pounds": 14090, + "tools": 14091, + "▁participate": 14092, + "▁Tax": 14093, + "▁скла": 14094, + "apol": 14095, + "▁fost": 14096, + "compat": 14097, + "▁publication": 14098, + "▁rapidly": 14099, + "▁Wis": 14100, + "EventListener": 14101, + "▁première": 14102, + "uso": 14103, + "extend": 14104, + "▁MERCHANTABILITY": 14105, + "UTF": 14106, + "▁experiments": 14107, + "single": 14108, + "zk": 14109, + "▁naj": 14110, + "}}}": 14111, + "Lin": 14112, + "▁interact": 14113, + "▁cms": 14114, + "▁Roger": 14115, + "▁Ру": 14116, + ">'": 14117, + "commit": 14118, + "лось": 14119, + "▁outcome": 14120, + "▁hits": 14121, + "▁им": 14122, + "▁spark": 14123, + "console": 14124, + "▁verw": 14125, + "▁като": 14126, + "agnostics": 14127, + "▁soci": 14128, + "▁dining": 14129, + "▁tech": 14130, + "št": 14131, + "folio": 14132, + "ultane": 14133, + "ктор": 14134, + "▁Brand": 14135, + "Join": 14136, + "▁ию": 14137, + "▁pros": 14138, + "▁posit": 14139, + "Public": 14140, + "AspNetCore": 14141, + "▁Shop": 14142, + "▁coinc": 14143, + "нием": 14144, + "▁references": 14145, + "about": 14146, + "namespace": 14147, + "DL": 14148, + "▁IR": 14149, + "▁cada": 14150, + "▁Jordan": 14151, + "▁gep": 14152, + "▁bron": 14153, + "andidate": 14154, + "EXPECT": 14155, + "amo": 14156, + "▁Deutsch": 14157, + "auc": 14158, + "▁райо": 14159, + "▁Labor": 14160, + "▁surrounded": 14161, + "тро": 14162, + "▁nome": 14163, + "▁underlying": 14164, + "▁educational": 14165, + "RIGHT": 14166, + "COUNT": 14167, + "inch": 14168, + "Typ": 14169, + "umph": 14170, + "four": 14171, + "Controls": 14172, + "▁cp": 14173, + "cost": 14174, + "▁mechanism": 14175, + "eness": 14176, + "équ": 14177, + "▁acquired": 14178, + "▁falls": 14179, + "▁Hou": 14180, + "▁LE": 14181, + "forEach": 14182, + "▁vertex": 14183, + "▁IF": 14184, + "curs": 14185, + "'=>": 14186, + "тери": 14187, + "▁SA": 14188, + "riers": 14189, + "▁uw": 14190, + "▁marks": 14191, + "▁energ": 14192, + "hof": 14193, + "ylvania": 14194, + "▁Allen": 14195, + "umpy": 14196, + "ого": 14197, + "ству": 14198, + "voice": 14199, + "▁engage": 14200, + "▁mant": 14201, + "orse": 14202, + "===": 14203, + "▁improvement": 14204, + "Opt": 14205, + "▁arrested": 14206, + "тия": 14207, + "▁сле": 14208, + "itched": 14209, + "socket": 14210, + "▁cycl": 14211, + "▁SM": 14212, + "▁Sex": 14213, + "▁neutral": 14214, + "вав": 14215, + "▁Jess": 14216, + "▁dip": 14217, + "▁opposition": 14218, + "▁borrow": 14219, + "спе": 14220, + "▁avant": 14221, + "кола": 14222, + "▁ta": 14223, + "Anim": 14224, + "▁Gall": 14225, + "rgb": 14226, + "▁guilty": 14227, + "▁buried": 14228, + "▁gy": 14229, + "Initial": 14230, + "▁accomp": 14231, + "▁breathing": 14232, + "berry": 14233, + "GRO": 14234, + "▁subsequent": 14235, + "roupe": 14236, + "ulpt": 14237, + "tb": 14238, + "▁ä": 14239, + "Pi": 14240, + "argv": 14241, + "▁Must": 14242, + ":'": 14243, + "svg": 14244, + "oup": 14245, + "▁precisely": 14246, + "▁Ta": 14247, + "rena": 14248, + "▁folder": 14249, + "▁Channel": 14250, + "▁revol": 14251, + "Miss": 14252, + "лом": 14253, + "reddit": 14254, + "adelph": 14255, + "▁discrim": 14256, + "▁ave": 14257, + "pleted": 14258, + "▁gently": 14259, + "FFFF": 14260, + "ropy": 14261, + "▁dial": 14262, + "NotFound": 14263, + "▁\"[": 14264, + "Home": 14265, + "onte": 14266, + "▁relie": 14267, + "▁Context": 14268, + "▁stats": 14269, + "▁Energy": 14270, + "ounced": 14271, + "▁grave": 14272, + "▁recip": 14273, + "лин": 14274, + "blog": 14275, + "▁naam": 14276, + "▁wo": 14277, + "▁directions": 14278, + "▁Lincoln": 14279, + "!)": 14280, + "unci": 14281, + "neq": 14282, + "Tags": 14283, + "▁tum": 14284, + "▁saving": 14285, + "aille": 14286, + "itemize": 14287, + "▁Famil": 14288, + "msm": 14289, + "news": 14290, + "FFER": 14291, + "▁Dead": 14292, + "▁territory": 14293, + "▁Kat": 14294, + "ocker": 14295, + "integer": 14296, + "▁sne": 14297, + "▁fails": 14298, + "▁français": 14299, + "▁introduction": 14300, + "▁Grant": 14301, + "ycle": 14302, + "'].": 14303, + "▁vier": 14304, + "native": 14305, + "▁Kle": 14306, + "quote": 14307, + "Users": 14308, + "▁advis": 14309, + "▁gym": 14310, + "▁protein": 14311, + "ال": 14312, + "▁Mai": 14313, + "▁providers": 14314, + "▁soil": 14315, + "gui": 14316, + "▁Nation": 14317, + "reation": 14318, + "▁Tab": 14319, + "ensis": 14320, + "inas": 14321, + "▁Scotland": 14322, + "▁dispatch": 14323, + "union": 14324, + "▁bere": 14325, + "▁Pow": 14326, + "▁Hig": 14327, + "▁studying": 14328, + "REF": 14329, + "SSL": 14330, + "▁fright": 14331, + "▁SORT": 14332, + "▁compr": 14333, + "▁Madrid": 14334, + "rowned": 14335, + "opes": 14336, + "pdev": 14337, + "▁wash": 14338, + "▁'../../": 14339, + "}}_": 14340, + "▁accum": 14341, + "rolling": 14342, + "▁NC": 14343, + "▁fiction": 14344, + "ipt": 14345, + "connected": 14346, + "limits": 14347, + "▁lap": 14348, + "▁whereas": 14349, + "prom": 14350, + "▁appointment": 14351, + "Program": 14352, + "▁Пер": 14353, + "nah": 14354, + "Validation": 14355, + "icons": 14356, + "äll": 14357, + "▁radical": 14358, + "▁exclusive": 14359, + "emony": 14360, + "▁challenging": 14361, + "▁ms": 14362, + "▁Private": 14363, + "▁vida": 14364, + "▁други": 14365, + "▁campus": 14366, + "forms": 14367, + "дно": 14368, + "plaat": 14369, + "bst": 14370, + "ATED": 14371, + "▁Abstract": 14372, + "▁intense": 14373, + "▁Ltd": 14374, + "▁controvers": 14375, + "óg": 14376, + "▁să": 14377, + "▁landing": 14378, + "!=": 14379, + "▁scenes": 14380, + "▁Chap": 14381, + "▁spoken": 14382, + "cred": 14383, + "▁pride": 14384, + "quet": 14385, + "▁meter": 14386, + "▁deutsch": 14387, + "uum": 14388, + "▁bless": 14389, + "▁Hann": 14390, + "▁inputs": 14391, + "▁Row": 14392, + "▁withdraw": 14393, + "Pal": 14394, + "acles": 14395, + "assets": 14396, + "▁vl": 14397, + "веде": 14398, + "▁Got": 14399, + "▁airport": 14400, + "wind": 14401, + "▁Columbia": 14402, + "▁chocolate": 14403, + "▁hö": 14404, + "▁alarm": 14405, + "FTWARE": 14406, + "▁Jay": 14407, + "▁sake": 14408, + "▁registration": 14409, + "vid": 14410, + "▁lake": 14411, + "▁username": 14412, + "▁hack": 14413, + "indexOf": 14414, + "cx": 14415, + "▁festival": 14416, + "▁clubs": 14417, + "cases": 14418, + "CTRL": 14419, + "];\r": 14420, + "▁Aud": 14421, + "▁primera": 14422, + "ват": 14423, + "▁brilliant": 14424, + "uther": 14425, + "▁difficulty": 14426, + "itals": 14427, + "▁scores": 14428, + "▁polít": 14429, + "database": 14430, + "aska": 14431, + "▁######": 14432, + "▁acid": 14433, + "aton": 14434, + "atomic": 14435, + "freq": 14436, + "▁WARRANTY": 14437, + "▁reporting": 14438, + ".),": 14439, + "▁nights": 14440, + "▁programme": 14441, + ")}{": 14442, + "xic": 14443, + "▁spo": 14444, + "lined": 14445, + "quarters": 14446, + "eree": 14447, + "mers": 14448, + "▁serves": 14449, + "cow": 14450, + "лько": 14451, + "enso": 14452, + "▁environ": 14453, + "Like": 14454, + "anche": 14455, + "▁crash": 14456, + "▁Kap": 14457, + "noindent": 14458, + "Conn": 14459, + "▁авто": 14460, + "▁infrastructure": 14461, + "IME": 14462, + "▁Room": 14463, + "need": 14464, + "orer": 14465, + "▁Dest": 14466, + "▁Domin": 14467, + "atherine": 14468, + "▁Sydney": 14469, + "▁gauge": 14470, + "▁jet": 14471, + "bably": 14472, + "▁commonly": 14473, + "▁stations": 14474, + "iah": 14475, + "nl": 14476, + "жу": 14477, + "eten": 14478, + "_)": 14479, + "iac": 14480, + "amos": 14481, + "nement": 14482, + "kon": 14483, + "Interval": 14484, + "▁cabin": 14485, + "▁eg": 14486, + "▁shots": 14487, + "▁Area": 14488, + "smith": 14489, + "parameter": 14490, + "'}": 14491, + "▁hem": 14492, + "▁singing": 14493, + "▁accessible": 14494, + "▁Prin": 14495, + "optional": 14496, + "ancial": 14497, + "ships": 14498, + "▁canvas": 14499, + "spe": 14500, + "▁addresses": 14501, + "▁xml": 14502, + "▁'\"": 14503, + "▁kar": 14504, + "öff": 14505, + "▁ages": 14506, + "ёр": 14507, + "zing": 14508, + "▁över": 14509, + "▁Clean": 14510, + "▁Silver": 14511, + "▁осо": 14512, + "health": 14513, + "Ali": 14514, + "▁ts": 14515, + "atern": 14516, + "▁choosing": 14517, + "▁burned": 14518, + "brid": 14519, + "rooms": 14520, + "ött": 14521, + "KERN": 14522, + "▁dish": 14523, + "Sa": 14524, + "Detail": 14525, + "▁Hind": 14526, + "▁Dans": 14527, + "ię": 14528, + "▁Jahren": 14529, + "extension": 14530, + "allas": 14531, + "▁Billy": 14532, + "usammen": 14533, + "itud": 14534, + "geon": 14535, + "Temp": 14536, + "Leg": 14537, + "ittel": 14538, + "addle": 14539, + "▁muscle": 14540, + "▁scared": 14541, + "sson": 14542, + "▁denote": 14543, + "ieurs": 14544, + "▁orange": 14545, + "▁hub": 14546, + "▁reb": 14547, + "edi": 14548, + "▁voices": 14549, + "Folder": 14550, + "▁suspend": 14551, + "▁Heart": 14552, + "▁scrap": 14553, + "▁aggreg": 14554, + "▁Guide": 14555, + "transaction": 14556, + "▁riding": 14557, + "▁vá": 14558, + "▁breed": 14559, + "▁concert": 14560, + "approx": 14561, + "▁chances": 14562, + "Tok": 14563, + "Eq": 14564, + "parts": 14565, + "▁scholar": 14566, + "offs": 14567, + "flush": 14568, + "!”": 14569, + "▁login": 14570, + "▁soort": 14571, + "▁Mand": 14572, + "▁functional": 14573, + "▁Bou": 14574, + "▁subjects": 14575, + "mys": 14576, + "▁extraord": 14577, + "▁Building": 14578, + "ikt": 14579, + "Bad": 14580, + "iami": 14581, + "Driver": 14582, + "ête": 14583, + "▁kv": 14584, + "▁timer": 14585, + "itionally": 14586, + "▁athlet": 14587, + "▁\");": 14588, + "wy": 14589, + "CFG": 14590, + "▁heaven": 14591, + "ов": 14592, + "▁experimental": 14593, + "▁bounds": 14594, + "ICK": 14595, + "▁excit": 14596, + "▁quit": 14597, + "▁universal": 14598, + "дь": 14599, + "▁SP": 14600, + "▁stub": 14601, + "▁kle": 14602, + "▁Bart": 14603, + "▁\"@": 14604, + "pel": 14605, + "▁(!(": 14606, + "▁selector": 14607, + "EB": 14608, + "▁coc": 14609, + "eted": 14610, + "ють": 14611, + "▁possess": 14612, + "▁Rick": 14613, + "▁unusual": 14614, + "termin": 14615, + "▁bags": 14616, + "▁loading": 14617, + "▁tf": 14618, + "▁)\r": 14619, + "provider": 14620, + "pletion": 14621, + "▁cursor": 14622, + "▁paused": 14623, + "им": 14624, + "▁counsel": 14625, + "]<": 14626, + "zech": 14627, + "▁tie": 14628, + "▁Moon": 14629, + "▁armed": 14630, + "▁observe": 14631, + "▁permet": 14632, + "▁Job": 14633, + "för": 14634, + "argument": 14635, + "▁eggs": 14636, + "ást": 14637, + "▁incredibly": 14638, + "werken": 14639, + "izard": 14640, + "▁painted": 14641, + "▁Vietnam": 14642, + "▁violent": 14643, + "Est": 14644, + "ierra": 14645, + "reader": 14646, + "weise": 14647, + "▁Josh": 14648, + "▁Him": 14649, + "ashes": 14650, + "origin": 14651, + "▁spir": 14652, + "▁Tree": 14653, + "▁niet": 14654, + "WIN": 14655, + "margin": 14656, + "▁involves": 14657, + "▁organis": 14658, + "▁Nacional": 14659, + "bara": 14660, + "▁depuis": 14661, + "pio": 14662, + "features": 14663, + "stru": 14664, + "▁Disney": 14665, + "▁restaurants": 14666, + "Mill": 14667, + "))\r": 14668, + "сла": 14669, + "remote": 14670, + "▁Third": 14671, + "▁baseball": 14672, + "▁algun": 14673, + "]$": 14674, + "▁employed": 14675, + "pot": 14676, + "▁UnityEngine": 14677, + "▁integration": 14678, + "▁risks": 14679, + "▁stro": 14680, + "▁agosto": 14681, + "including": 14682, + "▁Mind": 14683, + "▁stroke": 14684, + "▁deals": 14685, + "ajax": 14686, + "ёт": 14687, + "▁\\|": 14688, + "tar": 14689, + "adelphia": 14690, + "▁sab": 14691, + "pur": 14692, + "▁screw": 14693, + "▁inev": 14694, + "▁\\;": 14695, + "▁Donald": 14696, + "öd": 14697, + "cca": 14698, + "esis": 14699, + "▁separated": 14700, + "DBG": 14701, + "agent": 14702, + "▁packed": 14703, + "ння": 14704, + "intern": 14705, + "▁Monte": 14706, + "▁province": 14707, + "▁expanded": 14708, + "▁approached": 14709, + "▁Ep": 14710, + "CLK": 14711, + "▁ore": 14712, + "Batch": 14713, + "▁impressive": 14714, + "RM": 14715, + "▁Location": 14716, + "▁shame": 14717, + "wrapper": 14718, + "unwrap": 14719, + "peer": 14720, + "Bits": 14721, + "▁SN": 14722, + "scar": 14723, + "Come": 14724, + "▁council": 14725, + "▁shouted": 14726, + "making": 14727, + "▁Maur": 14728, + "▁wis": 14729, + "LETE": 14730, + "▁fs": 14731, + "▁dz": 14732, + "unque": 14733, + "uego": 14734, + "Random": 14735, + "Html": 14736, + "zem": 14737, + "▁Dutch": 14738, + "▁Golden": 14739, + "▁Tar": 14740, + "▁Herm": 14741, + "▁stretch": 14742, + "vard": 14743, + "▁tries": 14744, + "WI": 14745, + "▁disappeared": 14746, + "▁crusher": 14747, + "▁Kan": 14748, + "Mag": 14749, + "ør": 14750, + "▁Cambridge": 14751, + "▁dopo": 14752, + "atura": 14753, + "heart": 14754, + "▁Spiel": 14755, + "/**\r": 14756, + "Direction": 14757, + "atting": 14758, + "wig": 14759, + "▁codes": 14760, + "▁powder": 14761, + "alert": 14762, + "sembl": 14763, + "▁ye": 14764, + "Star": 14765, + "▁roots": 14766, + "▁Holl": 14767, + "Rele": 14768, + "▁constitu": 14769, + "nc": 14770, + "“.": 14771, + "reference": 14772, + "ificial": 14773, + "closure": 14774, + "▁figured": 14775, + "▁assumption": 14776, + "getElementById": 14777, + "▁AG": 14778, + "oses": 14779, + "▁_\"": 14780, + "epper": 14781, + "obre": 14782, + "enumerate": 14783, + "ографи": 14784, + "▁lessons": 14785, + "▁qualified": 14786, + "Person": 14787, + "anse": 14788, + "▁Mort": 14789, + "sylvania": 14790, + "▁cré": 14791, + "Binding": 14792, + "іс": 14793, + "▁Vari": 14794, + "▁reminded": 14795, + "▁membership": 14796, + "iper": 14797, + "zte": 14798, + "▁cref": 14799, + "▁PA": 14800, + "plaatst": 14801, + "▁Environment": 14802, + "boy": 14803, + "▁phrase": 14804, + "rivial": 14805, + "rag": 14806, + "води": 14807, + "▁pse": 14808, + "▁tournament": 14809, + ")},": 14810, + "▁Sound": 14811, + "▁Vel": 14812, + "▁Berg": 14813, + "elson": 14814, + "▁refuge": 14815, + "▁elsewhere": 14816, + "quality": 14817, + "▁abandoned": 14818, + "▁Flo": 14819, + "ibil": 14820, + "UAL": 14821, + "▁Platz": 14822, + "▁delta": 14823, + "▁Buy": 14824, + "rière": 14825, + "▁flour": 14826, + "▁laughing": 14827, + "▁Looking": 14828, + "Agent": 14829, + "▁wx": 14830, + "▁Wales": 14831, + "Ctx": 14832, + "▁cake": 14833, + "▁crate": 14834, + "▁кла": 14835, + "anga": 14836, + "Zero": 14837, + "▁amounts": 14838, + "Tra": 14839, + "ometric": 14840, + "▁constraints": 14841, + "▁temple": 14842, + "▁installation": 14843, + "stroke": 14844, + "▁Neder": 14845, + "ți": 14846, + "▁Ibid": 14847, + "▁obs": 14848, + "entries": 14849, + "▁jusqu": 14850, + "ORM": 14851, + "▁Sky": 14852, + "ikes": 14853, + "nak": 14854, + "▁modes": 14855, + "▁Hitler": 14856, + "▁belt": 14857, + "▁pointing": 14858, + "▁Ban": 14859, + "ignore": 14860, + "▁persu": 14861, + "▁Besides": 14862, + "ynom": 14863, + "▁legis": 14864, + "▁CPU": 14865, + "anded": 14866, + "uis": 14867, + "bsite": 14868, + "▁Euro": 14869, + "▁utter": 14870, + "eclipse": 14871, + "▁irre": 14872, + "▁Document": 14873, + "▁Meanwhile": 14874, + "▁familie": 14875, + "verify": 14876, + "▁Jason": 14877, + "▁Ort": 14878, + "▁ciudad": 14879, + "▁technologies": 14880, + "▁части": 14881, + "nica": 14882, + "cancel": 14883, + "Virtual": 14884, + "▁evident": 14885, + "aman": 14886, + "▁Supreme": 14887, + "atoes": 14888, + "▁steady": 14889, + "▁monthly": 14890, + "▁SOFTWARE": 14891, + "Die": 14892, + "▁applying": 14893, + "Dig": 14894, + "vier": 14895, + "▁горо": 14896, + "▁WH": 14897, + "▁minds": 14898, + "▁kam": 14899, + "▁expertise": 14900, + "▁notification": 14901, + ".-": 14902, + "▁deliber": 14903, + "▁HE": 14904, + "▁resist": 14905, + "outes": 14906, + "▁Howard": 14907, + "special": 14908, + "▁presentation": 14909, + "▁YouTube": 14910, + "mir": 14911, + "▁rust": 14912, + "▁nations": 14913, + "▁Gets": 14914, + "▁responses": 14915, + "arded": 14916, + "immer": 14917, + "▁reveal": 14918, + "▁Meg": 14919, + "▁todos": 14920, + "▁ade": 14921, + "ategories": 14922, + "▁payments": 14923, + "ôt": 14924, + "Enumer": 14925, + "▁platforms": 14926, + "▁lifetime": 14927, + "Complete": 14928, + "Quest": 14929, + "enders": 14930, + "▁cum": 14931, + "pler": 14932, + "▁appl": 14933, + "ährend": 14934, + "зь": 14935, + "enez": 14936, + "overty": 14937, + "ynchron": 14938, + "▁argued": 14939, + "▁Kath": 14940, + "▁synchron": 14941, + "▁Builder": 14942, + "Border": 14943, + "Plan": 14944, + "rieb": 14945, + "nm": 14946, + "FORMAT": 14947, + "usk": 14948, + "▁jumped": 14949, + "charg": 14950, + "▁contribute": 14951, + "Mesh": 14952, + "Univers": 14953, + "rell": 14954, + "▁polar": 14955, + "▁trois": 14956, + "icio": 14957, + "Groups": 14958, + "▁(%": 14959, + "Loop": 14960, + "▁gaz": 14961, + "dbg": 14962, + "LAY": 14963, + "John": 14964, + "blocks": 14965, + "▁lung": 14966, + "▁kön": 14967, + "through": 14968, + "▁fifth": 14969, + "lisher": 14970, + "▁involving": 14971, + "▁Deep": 14972, + "▁области": 14973, + "▁sull": 14974, + "Export": 14975, + "▁Kate": 14976, + "period": 14977, + "charge": 14978, + "GT": 14979, + "\">\r": 14980, + "тин": 14981, + "▁Ott": 14982, + "▁interactions": 14983, + "▁Toronto": 14984, + "TRACE": 14985, + "▁difer": 14986, + "▁liberal": 14987, + "▁particle": 14988, + "▁surve": 14989, + "alous": 14990, + "reason": 14991, + "▁depression": 14992, + "ал": 14993, + "▁flower": 14994, + "▁waar": 14995, + "▁hade": 14996, + "▁centuries": 14997, + "uty": 14998, + "party": 14999, + "▁approval": 15000, + "generate": 15001, + "▁Barn": 15002, + "▁marg": 15003, + "▁monde": 15004, + "▁ook": 15005, + "▁Clark": 15006, + "▁theoret": 15007, + "viously": 15008, + "?)": 15009, + "▁Rud": 15010, + "stmt": 15011, + "inction": 15012, + "▁tun": 15013, + "▁roads": 15014, + "▁rotation": 15015, + "ppen": 15016, + "sensor": 15017, + "▁Kol": 15018, + "idelines": 15019, + "▁є": 15020, + "▁composed": 15021, + "▁virus": 15022, + "'$": 15023, + "SN": 15024, + "▁Von": 15025, + "mont": 15026, + "lar": 15027, + "▁opinions": 15028, + "uction": 15029, + "rupal": 15030, + "underline": 15031, + "▁horror": 15032, + "Must": 15033, + "otto": 15034, + "Should": 15035, + "▁statist": 15036, + "▁gem": 15037, + "▁secre": 15038, + "▁strip": 15039, + "▁dirt": 15040, + "amazon": 15041, + "▁Round": 15042, + "▁discovery": 15043, + "▁GO": 15044, + "▁substantial": 15045, + "ibt": 15046, + "▁demands": 15047, + "▁everyday": 15048, + "▁besch": 15049, + "▁Bridge": 15050, + "▁HD": 15051, + "▁Dol": 15052, + "▁très": 15053, + "anni": 15054, + "roit": 15055, + "());\r": 15056, + "far": 15057, + "timestamp": 15058, + "▁bulk": 15059, + "Black": 15060, + "▁gan": 15061, + "setting": 15062, + "retval": 15063, + "ване": 15064, + "nung": 15065, + "▁talks": 15066, + "▁scientists": 15067, + "▁vig": 15068, + "▁quantity": 15069, + "▁Gard": 15070, + "▁movements": 15071, + "ähr": 15072, + "lings": 15073, + "▁Те": 15074, + "team": 15075, + "rito": 15076, + "▁assembly": 15077, + "ilst": 15078, + "▁happiness": 15079, + "▁leaf": 15080, + "▁assessment": 15081, + "Coord": 15082, + "irs": 15083, + "sam": 15084, + "▁attorney": 15085, + "▁geme": 15086, + "IDE": 15087, + "▁Vere": 15088, + "▁Anthony": 15089, + "amiento": 15090, + "▁Ast": 15091, + "▁circul": 15092, + "▁Frances": 15093, + "▁pent": 15094, + "▁mate": 15095, + "▁Transport": 15096, + "owo": 15097, + "чу": 15098, + "istes": 15099, + "TRAN": 15100, + "IMPORT": 15101, + "▁Break": 15102, + "▁sons": 15103, + "▁investors": 15104, + "▁Philipp": 15105, + "THOD": 15106, + "▁panic": 15107, + "▁:)": 15108, + "▁detection": 15109, + "▁simultane": 15110, + "nte": 15111, + "▁listened": 15112, + "кре": 15113, + "▁Brig": 15114, + "Optional": 15115, + "▁abund": 15116, + "▁criteria": 15117, + "▁chip": 15118, + "▁окру": 15119, + "▁Constant": 15120, + "▁mining": 15121, + "тал": 15122, + "mates": 15123, + "▁worship": 15124, + "router": 15125, + "CN": 15126, + "▁Match": 15127, + "▁Cole": 15128, + "▁downt": 15129, + "▁holes": 15130, + "▁grateful": 15131, + "RESULT": 15132, + "▁Europa": 15133, + "▁consent": 15134, + "lä": 15135, + "opter": 15136, + "▁colleagues": 15137, + "orous": 15138, + "▁enemies": 15139, + "hang": 15140, + "actual": 15141, + "Objects": 15142, + "▁як": 15143, + "▁fluid": 15144, + "fixed": 15145, + "▁Graph": 15146, + "▁scratch": 15147, + "cers": 15148, + "ribu": 15149, + "▁validation": 15150, + "▁completion": 15151, + "▁Begin": 15152, + "endpoint": 15153, + "rient": 15154, + "CM": 15155, + "▁Site": 15156, + "▁explains": 15157, + "tres": 15158, + "▁anybody": 15159, + "foreach": 15160, + "lon": 15161, + "Chain": 15162, + "▁Buff": 15163, + "ocal": 15164, + "▁Morgan": 15165, + "▁sang": 15166, + "▁passes": 15167, + "@@": 15168, + "ijd": 15169, + "Word": 15170, + "▁Hung": 15171, + "▁Fer": 15172, + "▁vý": 15173, + "bast": 15174, + "▁entertainment": 15175, + "hin": 15176, + "▁grat": 15177, + "▁Member": 15178, + "▁Minn": 15179, + "▁printed": 15180, + "▁Franklin": 15181, + "▁Imp": 15182, + "Machine": 15183, + "columns": 15184, + "▁deleted": 15185, + "▁manufacturing": 15186, + "▁rely": 15187, + "▁conse": 15188, + "▁fishing": 15189, + "blo": 15190, + "-$": 15191, + "▁.\"": 15192, + "▁clinical": 15193, + "▁Studies": 15194, + "▁Бу": 15195, + "definition": 15196, + "▁evaluation": 15197, + "▁attacked": 15198, + "▁frozen": 15199, + "zent": 15200, + "▁últ": 15201, + "▁rational": 15202, + "othe": 15203, + "Cancel": 15204, + "history": 15205, + "setText": 15206, + "▁alc": 15207, + "▁hydro": 15208, + "▁Theatre": 15209, + "▁Material": 15210, + "IOException": 15211, + "******/": 15212, + "spl": 15213, + "NODE": 15214, + "attrs": 15215, + "▁mie": 15216, + "▁offices": 15217, + "ró": 15218, + "▁jam": 15219, + "▁Ident": 15220, + "vé": 15221, + "Setting": 15222, + "▁Several": 15223, + "▁decay": 15224, + "Android": 15225, + "▁Save": 15226, + "unted": 15227, + "▁Mountain": 15228, + "usc": 15229, + "▁marzo": 15230, + "▁asleep": 15231, + "▁soldier": 15232, + "▁Double": 15233, + "PK": 15234, + "▁contrad": 15235, + "▁wins": 15236, + "ceiver": 15237, + "▁seasons": 15238, + "▁Chall": 15239, + "▁healthcare": 15240, + "ład": 15241, + "от": 15242, + "▁Five": 15243, + "▁Hell": 15244, + "▁worldwide": 15245, + "▁',": 15246, + "ян": 15247, + "made": 15248, + "▁responded": 15249, + "▁ay": 15250, + "▁procedures": 15251, + "тера": 15252, + "▁cleared": 15253, + "\"].": 15254, + "▁Target": 15255, + "▁Side": 15256, + "omin": 15257, + "▁deploy": 15258, + "▁Tell": 15259, + "▁ongoing": 15260, + "floor": 15261, + "▁bones": 15262, + "▁Delete": 15263, + "▁shrugged": 15264, + "Our": 15265, + "Der": 15266, + "▁initialize": 15267, + "▁Ted": 15268, + "MAGE": 15269, + "▁hire": 15270, + "▁tracking": 15271, + "▁ash": 15272, + "▁ceiling": 15273, + "ках": 15274, + "etti": 15275, + "▁courage": 15276, + "enschapp": 15277, + "ются": 15278, + "More": 15279, + "▁folg": 15280, + "▁Grace": 15281, + "▁Kelly": 15282, + "▁reven": 15283, + "▁Ali": 15284, + "▁disp": 15285, + "▁defeat": 15286, + "▁creature": 15287, + "▁Kennedy": 15288, + "▁Diego": 15289, + "EMP": 15290, + "▁steam": 15291, + "endance": 15292, + "rig": 15293, + "▁ignor": 15294, + "emen": 15295, + "▁Gru": 15296, + "▁proposal": 15297, + "▁weiter": 15298, + "▁лі": 15299, + "ibles": 15300, + "▁consideration": 15301, + "▁believes": 15302, + "▁Soph": 15303, + "“,": 15304, + "▁Matthew": 15305, + "▁circuit": 15306, + "▁singer": 15307, + "▁Square": 15308, + "ço": 15309, + "Edge": 15310, + "▁astr": 15311, + "▁representative": 15312, + "▁comprehensive": 15313, + "liga": 15314, + "▁mere": 15315, + "tbl": 15316, + "▁continuing": 15317, + "ographer": 15318, + "LED": 15319, + "▁/***/": 15320, + "▁sear": 15321, + "▁enormous": 15322, + "izi": 15323, + "Dit": 15324, + "there": 15325, + "ін": 15326, + "сите": 15327, + "▁guerra": 15328, + "▁endpoint": 15329, + "▁lesson": 15330, + "zon": 15331, + "variable": 15332, + "ис": 15333, + "▁researchers": 15334, + "▁attempted": 15335, + "▁enf": 15336, + "тура": 15337, + "▁defin": 15338, + "вест": 15339, + "▁awful": 15340, + "▁lowest": 15341, + "rules": 15342, + "▁unlike": 15343, + "interval": 15344, + "▁producing": 15345, + "▁Kam": 15346, + "▁IMP": 15347, + "General": 15348, + "▁faire": 15349, + "▁maxim": 15350, + "assemb": 15351, + "acent": 15352, + "?>": 15353, + "plica": 15354, + "▁ram": 15355, + "mate": 15356, + "цу": 15357, + "mn": 15358, + "▁Hi": 15359, + "▁stages": 15360, + "▁Editor": 15361, + "▁tang": 15362, + "RD": 15363, + "▁ich": 15364, + "▁dependent": 15365, + "lifer": 15366, + "ascript": 15367, + "▁exposure": 15368, + "рез": 15369, + "▁mart": 15370, + "▁Barcel": 15371, + "xspace": 15372, + "SESSION": 15373, + "▁prest": 15374, + "URCE": 15375, + "-.": 15376, + "▁село": 15377, + "have": 15378, + "▁observation": 15379, + "▁commands": 15380, + "▁eager": 15381, + "▁outdoor": 15382, + "▁DEBUG": 15383, + "▁hr": 15384, + "AX": 15385, + "▁puzz": 15386, + "blank": 15387, + "бур": 15388, + "▁kennis": 15389, + "▁regarded": 15390, + "▁}),": 15391, + "volume": 15392, + "▁произ": 15393, + "▁Training": 15394, + "añ": 15395, + "▁fois": 15396, + "▁три": 15397, + "вня": 15398, + "▁optimal": 15399, + "▁subscription": 15400, + "bridge": 15401, + "imental": 15402, + "▁Think": 15403, + "▁\";": 15404, + "▁legisl": 15405, + "▁Hop": 15406, + "▁branches": 15407, + "▁Veg": 15408, + "▁sprint": 15409, + "▁flux": 15410, + "▁Freder": 15411, + "sis": 15412, + "notify": 15413, + "▁Фран": 15414, + "som": 15415, + "nym": 15416, + "▁Ré": 15417, + "lett": 15418, + "ingham": 15419, + "▁Farm": 15420, + "DOM": 15421, + "▁shield": 15422, + "Here": 15423, + "▁Treat": 15424, + "▁Luke": 15425, + "▁unsafe": 15426, + "anton": 15427, + "▁Imper": 15428, + "▁telephone": 15429, + "▁unlock": 15430, + "Owner": 15431, + "collection": 15432, + "▁snd": 15433, + "▁suiv": 15434, + "▁entering": 15435, + "шен": 15436, + "▁Label": 15437, + "selector": 15438, + "▁GET": 15439, + "▁quando": 15440, + "▁fed": 15441, + "jQuery": 15442, + "Origin": 15443, + "▁Alan": 15444, + "mathscr": 15445, + "▁pregnant": 15446, + "Expect": 15447, + "resources": 15448, + "▁ersten": 15449, + "alia": 15450, + "▁retired": 15451, + "ût": 15452, + "Cred": 15453, + "▁méd": 15454, + "▁erh": 15455, + "Framework": 15456, + "Slot": 15457, + "duration": 15458, + "sal": 15459, + "▁composition": 15460, + "article": 15461, + "gpu": 15462, + "▁permitted": 15463, + "▁Font": 15464, + "▁Much": 15465, + "▁pending": 15466, + "▁agencies": 15467, + "Columns": 15468, + "▁klik": 15469, + "▁rating": 15470, + "mind": 15471, + "▁Pennsylvania": 15472, + "Java": 15473, + "abstract": 15474, + "▁dumb": 15475, + "▁VI": 15476, + "usa": 15477, + "Remote": 15478, + "▁YOU": 15479, + "▁Creek": 15480, + "мати": 15481, + "Bottom": 15482, + "▁rolling": 15483, + "▁bundle": 15484, + "▁golf": 15485, + "gpio": 15486, + "▁Chair": 15487, + "▁cls": 15488, + "$}": 15489, + "▁Parliament": 15490, + "führ": 15491, + "Many": 15492, + "▁Sep": 15493, + "▁badly": 15494, + "igi": 15495, + "▁Gemeinde": 15496, + "Ill": 15497, + "▁Ан": 15498, + "uart": 15499, + "itempty": 15500, + "▁Niger": 15501, + "▁immigr": 15502, + "Super": 15503, + "vá": 15504, + "istribute": 15505, + "Helpers": 15506, + "▁waters": 15507, + "▁joining": 15508, + "omitempty": 15509, + "▁Otherwise": 15510, + "▁Host": 15511, + "▁redd": 15512, + "▁dy": 15513, + "▁converted": 15514, + "▁prayer": 15515, + "▁Украї": 15516, + "▁elections": 15517, + "reb": 15518, + "erie": 15519, + "▁свя": 15520, + "Abs": 15521, + "iembre": 15522, + "holders": 15523, + "▁Rol": 15524, + "utschen": 15525, + "▁Gh": 15526, + "tery": 15527, + "анг": 15528, + "▁narrative": 15529, + "minus": 15530, + "▁Iron": 15531, + "=\"#": 15532, + "▁wand": 15533, + "▁wished": 15534, + "icode": 15535, + "orr": 15536, + "[[": 15537, + "▁detected": 15538, + "▁municipal": 15539, + "▁Pour": 15540, + "▁Serv": 15541, + "citet": 15542, + "▁grey": 15543, + "▁Rap": 15544, + "▁voy": 15545, + "▁lleg": 15546, + "▁currency": 15547, + "▁Script": 15548, + "strument": 15549, + "▁expecting": 15550, + "▁tickets": 15551, + "▁bucket": 15552, + "egr": 15553, + "▁jacket": 15554, + "drv": 15555, + "▁loans": 15556, + "▁kann": 15557, + "▁integral": 15558, + "▁characteristics": 15559, + "(\".": 15560, + "▁manual": 15561, + "▁dynamics": 15562, + ":*": 15563, + "sha": 15564, + "reens": 15565, + "onical": 15566, + "▁toile": 15567, + "aña": 15568, + "▁distant": 15569, + "▁handled": 15570, + "Bool": 15571, + "▁penal": 15572, + "▁Things": 15573, + "▁prominent": 15574, + "▁exped": 15575, + "▁Help": 15576, + "▁asp": 15577, + "lap": 15578, + "▁Auth": 15579, + "Basic": 15580, + "achuset": 15581, + "▁Bild": 15582, + "▁entitled": 15583, + "▁jag": 15584, + "▁rejected": 15585, + "▁memor": 15586, + "orts": 15587, + "▁applies": 15588, + "▁Language": 15589, + "specific": 15590, + "achusetts": 15591, + "HAND": 15592, + "▁Route": 15593, + "market": 15594, + "▁Ky": 15595, + "▁pose": 15596, + "ACHE": 15597, + "poll": 15598, + "▁rocks": 15599, + "bone": 15600, + "▁DIS": 15601, + "Watch": 15602, + "▁smiling": 15603, + "рио": 15604, + "Month": 15605, + "▁efter": 15606, + "construct": 15607, + "▁bands": 15608, + "▁collaboration": 15609, + "ними": 15610, + "glas": 15611, + "▁vy": 15612, + "▁engagement": 15613, + "__)": 15614, + "▁wings": 15615, + "ким": 15616, + "netje": 15617, + "ativa": 15618, + "▁Duke": 15619, + "лее": 15620, + "▁Within": 15621, + "▁dove": 15622, + "▁cb": 15623, + "yers": 15624, + "pow": 15625, + "[(": 15626, + "▁evaluate": 15627, + "Points": 15628, + "▁рі": 15629, + "odigd": 15630, + "onomy": 15631, + "▁Illinois": 15632, + "▁Typ": 15633, + "▁coordinates": 15634, + "pisode": 15635, + "ucked": 15636, + "▁flav": 15637, + "▁brands": 15638, + "▁calendar": 15639, + "Lib": 15640, + "▁uitgen": 15641, + "▁tale": 15642, + "▁briefly": 15643, + "▁mic": 15644, + "RESS": 15645, + "▁später": 15646, + "▁integrated": 15647, + "▁cookies": 15648, + "▁uitgenodigd": 15649, + "▁Priv": 15650, + "▁phenomen": 15651, + "▁voegen": 15652, + "Supp": 15653, + "▁refers": 15654, + "пад": 15655, + "▁Clinton": 15656, + "▁assignment": 15657, + "inals": 15658, + "▁asym": 15659, + "cycle": 15660, + "▁Anderson": 15661, + "▁binding": 15662, + "rique": 15663, + "hind": 15664, + "▁behalf": 15665, + "▁Fle": 15666, + "▁breaks": 15667, + "▁soap": 15668, + "вар": 15669, + "▁vä": 15670, + "▁crying": 15671, + "▁→": 15672, + "▁msm": 15673, + "▁boots": 15674, + "owing": 15675, + "▁bell": 15676, + "suite": 15677, + "▁Bundes": 15678, + "Year": 15679, + "ndef": 15680, + "Other": 15681, + "▁google": 15682, + "ENCE": 15683, + "WER": 15684, + "Les": 15685, + "Shared": 15686, + "▁ED": 15687, + "IFT": 15688, + "▁floating": 15689, + "ým": 15690, + "{},": 15691, + "Binary": 15692, + "▁roce": 15693, + "raj": 15694, + "▁bewerken": 15695, + "BF": 15696, + "▁Hur": 15697, + "cen": 15698, + "▁ere": 15699, + "▁camb": 15700, + "▁Pakistan": 15701, + "▁greatly": 15702, + "▁logging": 15703, + "/.": 15704, + "Tensor": 15705, + "▁opens": 15706, + "▁Rio": 15707, + "▁klikken": 15708, + "▁sculpt": 15709, + "apore": 15710, + "wx": 15711, + "▁Nich": 15712, + "nan": 15713, + "▁injured": 15714, + "compare": 15715, + "tha": 15716, + "Sample": 15717, + "Shell": 15718, + "▁commander": 15719, + "▁receiver": 15720, + "▁hopes": 15721, + "▁byl": 15722, + "▁proxy": 15723, + "▁gall": 15724, + "getId": 15725, + "▁Bab": 15726, + "feld": 15727, + "▁\"_": 15728, + "▁Hab": 15729, + "simple": 15730, + "▁executed": 15731, + "▁ate": 15732, + "▁animation": 15733, + "▁inhab": 15734, + "▁боль": 15735, + "▁router": 15736, + "▁glob": 15737, + "Geplaatst": 15738, + "▁beginnetje": 15739, + "▁Kur": 15740, + "▁Ха": 15741, + "aligned": 15742, + "▁certificate": 15743, + "▁Å": 15744, + ".).": 15745, + "▁soll": 15746, + "▁Import": 15747, + "реди": 15748, + "▁pandemic": 15749, + "▁nic": 15750, + "vä": 15751, + "▁Gree": 15752, + "▁Say": 15753, + "▁ді": 15754, + "▁Num": 15755, + "▁roughly": 15756, + "▁después": 15757, + "▁​": 15758, + "▁specify": 15759, + "Mapper": 15760, + "licht": 15761, + "▁thumb": 15762, + "wie": 15763, + "▁unlikely": 15764, + "▁Edd": 15765, + "Hey": 15766, + "▁Opt": 15767, + "BLOCK": 15768, + "вор": 15769, + "▁×": 15770, + "▁ba": 15771, + "▁periods": 15772, + "▁titles": 15773, + "Med": 15774, + "▁fon": 15775, + "▁bast": 15776, + "▁Forest": 15777, + "▁№": 15778, + "onds": 15779, + "▁fal": 15780, + "▁gesch": 15781, + "direction": 15782, + "IFY": 15783, + "▁LA": 15784, + "▁(((": 15785, + "GTH": 15786, + "itudes": 15787, + "▁destruction": 15788, + "▁Ja": 15789, + "▁stake": 15790, + "ifferent": 15791, + "▁identical": 15792, + "▁fog": 15793, + "▁Reb": 15794, + "ские": 15795, + "ступ": 15796, + "jax": 15797, + "▁Mars": 15798, + "▁historic": 15799, + "▁Vo": 15800, + "▁entrepre": 15801, + "▁tension": 15802, + "▁WHERE": 15803, + "▁Philadelphia": 15804, + "Counter": 15805, + "▁frames": 15806, + "▁muy": 15807, + "ej": 15808, + "öt": 15809, + "eu": 15810, + "▁челове": 15811, + "PROC": 15812, + "▁resolved": 15813, + "▁tape": 15814, + "цион": 15815, + "▁singular": 15816, + "▁personnel": 15817, + "▁Mun": 15818, + "▁Occ": 15819, + "▁scalar": 15820, + "dess": 15821, + "▁cable": 15822, + "being": 15823, + "▁Jenn": 15824, + "▁erst": 15825, + "Actions": 15826, + "Environment": 15827, + "via": 15828, + "▁struggling": 15829, + "▁DVD": 15830, + "whe": 15831, + "▁throwing": 15832, + "Bounds": 15833, + "▁MD": 15834, + "▁\"../": 15835, + "▁satisfy": 15836, + "▁Colorado": 15837, + "▁Active": 15838, + "Tasks": 15839, + "<>();": 15840, + "▁slipped": 15841, + "▁poison": 15842, + "zb": 15843, + "Dispatch": 15844, + "warning": 15845, + "▁ultimate": 15846, + "picture": 15847, + "expression": 15848, + "▁Talk": 15849, + "▁flick": 15850, + "▁raising": 15851, + "▁transactions": 15852, + "▁glance": 15853, + "▁gri": 15854, + "▁през": 15855, + "selection": 15856, + "ња": 15857, + "endl": 15858, + "▁Abb": 15859, + "▁bold": 15860, + "▁maintained": 15861, + "Exists": 15862, + "▁encouraged": 15863, + "Qual": 15864, + "▁essere": 15865, + "▁hired": 15866, + "letter": 15867, + "itches": 15868, + "others": 15869, + "▁woj": 15870, + "▁injuries": 15871, + "▁dil": 15872, + "execut": 15873, + "▁Steel": 15874, + "▁Garden": 15875, + "зя": 15876, + "\\,\\": 15877, + "▁Angel": 15878, + "prim": 15879, + ">:]<": 15880, + "gb": 15881, + "peat": 15882, + "inte": 15883, + "▁apolog": 15884, + "▁regulations": 15885, + "Src": 15886, + "kh": 15887, + "Upload": 15888, + "mapping": 15889, + "▁presents": 15890, + "▁poetry": 15891, + "▁stops": 15892, + "▁Tol": 15893, + "▁tower": 15894, + "▁OUT": 15895, + "Thank": 15896, + "▁organic": 15897, + "▁drei": 15898, + "▁pound": 15899, + "century": 15900, + "▁modules": 15901, + "▁дере": 15902, + "▁worn": 15903, + "▁parad": 15904, + "▁Cos": 15905, + "fic": 15906, + "▁без": 15907, + "▁Jimmy": 15908, + "▁lands": 15909, + "▁minist": 15910, + "vspace": 15911, + "▁lighting": 15912, + "▁naked": 15913, + "▁designer": 15914, + "▁Stream": 15915, + "TMP": 15916, + "Center": 15917, + "resentation": 15918, + "ONT": 15919, + "▁ers": 15920, + "▁measurement": 15921, + "▁muscles": 15922, + "▁Ign": 15923, + "▁COM": 15924, + "▁fru": 15925, + "▁genre": 15926, + "▁alpha": 15927, + "▁retirement": 15928, + "▁Gon": 15929, + "ől": 15930, + "contents": 15931, + "▁healing": 15932, + "▁sido": 15933, + "incipal": 15934, + "Permission": 15935, + "рак": 15936, + "▁Gordon": 15937, + "▁Rank": 15938, + "▁Autom": 15939, + "Constructor": 15940, + "wiki": 15941, + "▁concerning": 15942, + "rizona": 15943, + "▁variant": 15944, + "▁arranged": 15945, + "▁Spr": 15946, + "BPACK": 15947, + "Timestamp": 15948, + "restore": 15949, + "aware": 15950, + "▁Observ": 15951, + "▁SV": 15952, + "ipp": 15953, + "▁Executive": 15954, + "▁colleg": 15955, + "▁explicitly": 15956, + "written": 15957, + "▁Kön": 15958, + "irus": 15959, + "▁Hold": 15960, + "▁Pract": 15961, + "Character": 15962, + "▁redistribute": 15963, + "uerto": 15964, + "▁Student": 15965, + "▁elder": 15966, + "▁Dop": 15967, + "vp": 15968, + "▁Hub": 15969, + "▁grounds": 15970, + "▁Ry": 15971, + "▁signals": 15972, + "▁gifts": 15973, + "▁strengthen": 15974, + "▁Lyn": 15975, + "commun": 15976, + "▁най": 15977, + "▁finance": 15978, + "noc": 15979, + "helm": 15980, + "▁cuts": 15981, + "▁adventure": 15982, + "▁Ric": 15983, + "▁intellectual": 15984, + "▁Output": 15985, + "▁awk": 15986, + "▁concentration": 15987, + "▁guidance": 15988, + "Buff": 15989, + "▁filling": 15990, + "▁regul": 15991, + "▁delicious": 15992, + "([]": 15993, + "ших": 15994, + "▁tons": 15995, + "activity": 15996, + "GP": 15997, + "LOB": 15998, + "stadt": 15999, + "tal": 16000, + "▁img": 16001, + "▁rush": 16002, + "attice": 16003, + "▁pok": 16004, + "steps": 16005, + "▁lid": 16006, + "▁DNA": 16007, + "Browser": 16008, + "▁ladies": 16009, + "▁années": 16010, + "▁rescue": 16011, + "avity": 16012, + "rock": 16013, + "▁glasses": 16014, + "▁Bey": 16015, + ")}$": 16016, + "detail": 16017, + "▁dés": 16018, + "tax": 16019, + "▁favourite": 16020, + "▁precision": 16021, + "▁conoc": 16022, + "Ms": 16023, + "▁Native": 16024, + "▁Pil": 16025, + "InputStream": 16026, + "orp": 16027, + "▁Pap": 16028, + "▁picking": 16029, + "iph": 16030, + "Loading": 16031, + "▁priest": 16032, + "Hook": 16033, + "▁pist": 16034, + "▁Une": 16035, + "%,": 16036, + "▁bil": 16037, + "▁conservative": 16038, + "eval": 16039, + "iking": 16040, + "'},": 16041, + "▁sauce": 16042, + "▁Due": 16043, + "assen": 16044, + "▁occasionally": 16045, + "▁Дж": 16046, + "unknown": 16047, + "DED": 16048, + "▁drum": 16049, + "▁dub": 16050, + "ATURE": 16051, + "usage": 16052, + "getType": 16053, + "reply": 16054, + "▁strategic": 16055, + "▁kap": 16056, + "design": 16057, + "datetime": 16058, + "▁Prim": 16059, + "Master": 16060, + "▁Corps": 16061, + "▁considerable": 16062, + "▁Tu": 16063, + "▁ла": 16064, + "▁tous": 16065, + "▁clar": 16066, + "▁poem": 16067, + "album": 16068, + "]*": 16069, + "loaded": 16070, + "▁traveling": 16071, + "вые": 16072, + "▁Ferr": 16073, + "▁pharm": 16074, + "abi": 16075, + "▁}\\": 16076, + "collect": 16077, + "▁Bour": 16078, + "OC": 16079, + "▁measurements": 16080, + "▁Professional": 16081, + "▁sensor": 16082, + "utsche": 16083, + "▁demanded": 16084, + "▁accompanied": 16085, + "▁prend": 16086, + "▁encoding": 16087, + "▁Geschichte": 16088, + "▁mig": 16089, + "▁Gib": 16090, + "▁Reich": 16091, + "▁myster": 16092, + "▁Mock": 16093, + "▁physically": 16094, + "▁Bau": 16095, + "▁Single": 16096, + "▁managing": 16097, + "▁Kil": 16098, + "▁Temple": 16099, + "▁lev": 16100, + "▁lí": 16101, + "CPU": 16102, + "▁Premier": 16103, + "▁Give": 16104, + "iri": 16105, + "NV": 16106, + "▁AI": 16107, + "▁fp": 16108, + "лександ": 16109, + "▁tant": 16110, + "▁fot": 16111, + "Nullable": 16112, + "▁guards": 16113, + "Once": 16114, + "▁chamber": 16115, + "film": 16116, + "▁bias": 16117, + "▁Tai": 16118, + "insic": 16119, + "▁ml": 16120, + "▁Ka": 16121, + "вал": 16122, + "▁SER": 16123, + "▁Someone": 16124, + "}}_{": 16125, + "Fixed": 16126, + "▁bent": 16127, + "▁prohib": 16128, + "▁bid": 16129, + "▁fewer": 16130, + "кры": 16131, + "▁lugar": 16132, + "▁deserve": 16133, + "ssl": 16134, + "▁cfg": 16135, + "reck": 16136, + "▁stability": 16137, + "resize": 16138, + "▁assertThat": 16139, + "Trigger": 16140, + "▁станов": 16141, + "plugins": 16142, + "▁lets": 16143, + "хід": 16144, + "▁Laura": 16145, + "нер": 16146, + "▁brut": 16147, + "▁FI": 16148, + "isons": 16149, + "▁dyn": 16150, + "icher": 16151, + "rayed": 16152, + "▁frequent": 16153, + "▁jedoch": 16154, + "▁Marine": 16155, + "strings": 16156, + "▁Util": 16157, + "▁bos": 16158, + "Mus": 16159, + "▁Portugal": 16160, + "Strategy": 16161, + "▁посе": 16162, + "▁slice": 16163, + "▁insight": 16164, + "▁widget": 16165, + "▁général": 16166, + "messages": 16167, + "▁Hu": 16168, + "▁requirement": 16169, + "Side": 16170, + "emplates": 16171, + "▁ceremony": 16172, + "▁physics": 16173, + "▁graduate": 16174, + "para": 16175, + "▁preserv": 16176, + "▁shops": 16177, + "zek": 16178, + "▁ub": 16179, + "prepare": 16180, + "▁Oil": 16181, + "▁fib": 16182, + "▁runtime": 16183, + "▁hogy": 16184, + "Warning": 16185, + "▁Convert": 16186, + "bourne": 16187, + "▁emerged": 16188, + "▁Ди": 16189, + "ighth": 16190, + "guard": 16191, + "kal": 16192, + "validation": 16193, + "ência": 16194, + "▁drinks": 16195, + "theorem": 16196, + "HR": 16197, + "iev": 16198, + "ployee": 16199, + "Usage": 16200, + "▁спе": 16201, + "dispatch": 16202, + "▁instantly": 16203, + "obi": 16204, + "▁justify": 16205, + "▁Nev": 16206, + "▁явля": 16207, + "agra": 16208, + "▁transmission": 16209, + "fly": 16210, + ";';": 17021, + "▁cousin": 17022, + "createElement": 17023, + "Could": 17024, + "▁capac": 17025, + "▁pause": 17026, + "ArrayList": 17027, + "kte": 17028, + "ordered": 17029, + "▁shaking": 17030, + "labels": 17031, + "▁reducing": 17032, + "вых": 17033, + "USED": 17034, + "▁voting": 17035, + "▁Ministry": 17036, + "▁Mig": 17037, + "▁Chen": 17038, + "▁accompany": 17039, + "ulle": 17040, + "▁ga": 17041, + "▁equipped": 17042, + "▁nun": 17043, + "Bet": 17044, + "▁licensed": 17045, + "ARCH": 17046, + "FN": 17047, + "▁engines": 17048, + "▁ster": 17049, + "▁locale": 17050, + "▁въ": 17051, + "links": 17052, + "▁Capital": 17053, + "▁alien": 17054, + "Wr": 17055, + "ръ": 17056, + "Cart": 17057, + "▁Marketing": 17058, + "▁RT": 17059, + "FileName": 17060, + "▁ti": 17061, + "iji": 17062, + "▁versus": 17063, + "live": 17064, + "Sym": 17065, + "kor": 17066, + "▁emission": 17067, + "umm": 17068, + "ycz": 17069, + "▁climbed": 17070, + "▁plusieurs": 17071, + "кри": 17072, + "yar": 17073, + "osten": 17074, + "▁usb": 17075, + "▁crossing": 17076, + "▁polynom": 17077, + "▁removal": 17078, + "▁Adams": 17079, + "▁ihre": 17080, + "anden": 17081, + "▁Benj": 17082, + "▁Phill": 17083, + "▁wounded": 17084, + "▁Castle": 17085, + "bild": 17086, + "Annotation": 17087, + "Processor": 17088, + "▁tin": 17089, + "folg": 17090, + "▁Students": 17091, + "▁Mexican": 17092, + "▁administrative": 17093, + "ILED": 17094, + "▁conqu": 17095, + "▁cheer": 17096, + "▁Ces": 17097, + "Because": 17098, + "▁Juni": 17099, + "▁encontr": 17100, + "avi": 17101, + "VI": 17102, + "aku": 17103, + "▁Ton": 17104, + "▁smoking": 17105, + "▁bay": 17106, + "works": 17107, + "ат": 17108, + "attered": 17109, + "▁Boolean": 17110, + "▁Balt": 17111, + "defer": 17112, + "pathy": 17113, + "Ah": 17114, + "▁akt": 17115, + "▁governor": 17116, + "Pad": 17117, + "▁sisters": 17118, + "Lat": 17119, + "▁revel": 17120, + "▁SY": 17121, + "itos": 17122, + "▁filters": 17123, + "Chunk": 17124, + "consum": 17125, + "▁removing": 17126, + "▁Herr": 17127, + "▁generator": 17128, + "▁Cra": 17129, + "▁farmers": 17130, + "▁Members": 17131, + "▁overcome": 17132, + "▁Cin": 17133, + "igkeit": 17134, + "criptions": 17135, + "Tests": 17136, + "▁клу": 17137, + "▁shake": 17138, + "▁yy": 17139, + "placement": 17140, + "▁awards": 17141, + "▁episodes": 17142, + "▁Blood": 17143, + "▁bullet": 17144, + "▁viene": 17145, + "▁Financial": 17146, + "Future": 17147, + "▁rou": 17148, + "▁biologie": 17149, + "▁useState": 17150, + "iani": 17151, + "piece": 17152, + "▁speaker": 17153, + "▁refr": 17154, + "ARK": 17155, + "▁MIT": 17156, + "▁Tan": 17157, + "▁Based": 17158, + "▁cultiv": 17159, + "▁hungry": 17160, + "▁Ay": 17161, + "▁Hey": 17162, + "▁excitement": 17163, + "ibraries": 17164, + "Hit": 17165, + "▁Ende": 17166, + "NG": 17167, + "FIL": 17168, + ".\")": 17169, + "Family": 17170, + "inery": 17171, + "necess": 17172, + "velope": 17173, + "▁Bot": 17174, + "porter": 17175, + "▁climb": 17176, + "▁Eli": 17177, + "urent": 17178, + "▁mistakes": 17179, + "ában": 17180, + "marks": 17181, + "pkt": 17182, + "Library": 17183, + "sted": 17184, + "ublice": 17185, + "▁Administration": 17186, + "▁shapes": 17187, + "публи": 17188, + "God": 17189, + "innen": 17190, + "коло": 17191, + "<<<<": 17192, + "ibe": 17193, + "ês": 17194, + "▁США": 17195, + "▁Foreign": 17196, + "▁Margaret": 17197, + "▁gene": 17198, + "▁disturb": 17199, + "▁тер": 17200, + "▁onClick": 17201, + "▁Engineering": 17202, + "▁stopping": 17203, + "▁restrictions": 17204, + ",*": 17205, + "BUF": 17206, + "▁shadows": 17207, + "hci": 17208, + "▁Christians": 17209, + "▁fence": 17210, + "▁luxury": 17211, + "akh": 17212, + "coord": 17213, + "▁investigate": 17214, + "▁conventional": 17215, + "\"—": 17216, + "▁visits": 17217, + "isé": 17218, + "▁Sac": 17219, + "className": 17220, + "▁Psych": 17221, + "▁reflected": 17222, + "▁пло": 17223, + "▁Vice": 17224, + "ław": 17225, + "________________": 17226, + "▁Wolf": 17227, + "rente": 17228, + "▁Champion": 17229, + "▁simulation": 17230, + "esota": 17231, + "▁Soon": 17232, + "▁Cel": 17233, + "▁theories": 17234, + "▁STR": 17235, + "▁collective": 17236, + "▁coordinate": 17237, + "querySelector": 17238, + "emed": 17239, + "Break": 17240, + "▁gef": 17241, + "▁electricity": 17242, + "▁gathering": 17243, + "aters": 17244, + "exper": 17245, + "▁Roma": 17246, + "▁Cooper": 17247, + "SYMBOL": 17248, + "vd": 17249, + "iversary": 17250, + "aines": 17251, + "▁Grad": 17252, + "▁independence": 17253, + "woh": 17254, + "▁consequence": 17255, + "▁conversations": 17256, + "▁Rou": 17257, + "▁andere": 17258, + "▁Systems": 17259, + "гар": 17260, + "▁moist": 17261, + "flu": 17262, + "ція": 17263, + "ниш": 17264, + "▁rode": 17265, + "▁perd": 17266, + "▁szer": 17267, + "▁flood": 17268, + "▁intim": 17269, + "stderr": 17270, + "▁reflection": 17271, + "Scan": 17272, + "▁disaster": 17273, + "akespe": 17274, + "▁Invalid": 17275, + "▁humor": 17276, + "▁Friedrich": 17277, + "▁suggestions": 17278, + "uvud": 17279, + "Delay": 17280, + "brief": 17281, + "▁ис": 17282, + "glied": 17283, + "fas": 17284, + "▁Smart": 17285, + "▁medi": 17286, + "sdk": 17287, + "▁seus": 17288, + "▁Arizona": 17289, + "▁innocent": 17290, + "Warn": 17291, + "acious": 17292, + "▁Moscow": 17293, + "▁caps": 17294, + "Delegate": 17295, + "▁dramatic": 17296, + "books": 17297, + "▁shore": 17298, + "uki": 17299, + "▁Russell": 17300, + "▁correlation": 17301, + "Help": 17302, + "▁pubblic": 17303, + "zym": 17304, + "comb": 17305, + "EY": 17306, + "LENGTH": 17307, + "▁Mün": 17308, + "▁_.": 17309, + "▁ferm": 17310, + "▁Ian": 17311, + "▁Studio": 17312, + "▁affairs": 17313, + "los": 17314, + "Rules": 17315, + "running": 17316, + "▁Posted": 17317, + "Pixel": 17318, + "▁dancing": 17319, + "▁agreements": 17320, + "▁Pic": 17321, + "ancia": 17322, + "▁má": 17323, + "ationToken": 17324, + "descriptor": 17325, + "▁Carter": 17326, + "Release": 17327, + "************": 17328, + "▁outstanding": 17329, + "changes": 17330, + "ARRAY": 17331, + "▁Barbara": 17332, + "▁nurse": 17333, + "(\r": 17334, + "▁Douglas": 17335, + "▁nucle": 17336, + "ouri": 17337, + "▁Style": 17338, + "avo": 17339, + "▁painful": 17340, + "▁slic": 17341, + "▁seinem": 17342, + "SUPPORT": 17343, + "ogene": 17344, + "▁satell": 17345, + "tagon": 17346, + "▁collapse": 17347, + "velle": 17348, + "MON": 17349, + "aughters": 17350, + "▁threatened": 17351, + "▁Illegal": 17352, + "▁desperate": 17353, + "strict": 17354, + "rus": 17355, + "ститу": 17356, + "\\\":": 17357, + "▁conflic": 17358, + "download": 17359, + "atos": 17360, + "▁Position": 17361, + ".*;": 17362, + "▁theater": 17363, + "▁pleasant": 17364, + "▁Cette": 17365, + "▁Singapore": 17366, + "heet": 17367, + "▁pir": 17368, + "▁acquis": 17369, + "▁назва": 17370, + "теля": 17371, + "▁recru": 17372, + "жения": 17373, + "ёл": 17374, + "версите": 17375, + "▁respective": 17376, + "▁tunnel": 17377, + "▁Dean": 17378, + "Du": 17379, + "▁uncle": 17380, + "▁offensive": 17381, + "colo": 17382, + "▁Unlike": 17383, + "series": 17384, + "▁Arn": 17385, + "minute": 17386, + "▁descriptor": 17387, + "▁stones": 17388, + "ICATION": 17389, + "▁Pad": 17390, + "▁iPhone": 17391, + "ei": 17392, + "▁fantasy": 17393, + "▁Korean": 17394, + "\"}": 17395, + "▁orth": 17396, + "halten": 17397, + "deep": 17398, + "▁Kay": 17399, + "requency": 17400, + "▁duties": 17401, + "awt": 17402, + "▁nearest": 17403, + "▁disorder": 17404, + "стру": 17405, + "▁Chile": 17406, + "▁seq": 17407, + "▁transportation": 17408, + "OO": 17409, + "▁Dez": 17410, + "iju": 17411, + "▁Results": 17412, + "jed": 17413, + "ivel": 17414, + "HOST": 17415, + "▁€": 17416, + "▁Î": 17417, + "▁chin": 17418, + "▁matt": 17419, + "▁voted": 17420, + "▁gehör": 17421, + "▁▁▁▁▁▁▁▁▁▁▁": 17422, + "▁sue": 17423, + "▁legacy": 17424, + "вся": 17425, + "SOURCE": 17426, + "WORK": 17427, + "itis": 17428, + "▁$|": 17429, + "▁обо": 17430, + "▁nr": 17431, + "▁Tamb": 17432, + "▁snap": 17433, + "▁impressed": 17434, + "▁deposit": 17435, + "▁divid": 17436, + "Segment": 17437, + "▁кар": 17438, + "▁Gas": 17439, + "▁crimes": 17440, + "▁insult": 17441, + "▁Hum": 17442, + "▁bounded": 17443, + "▁kicked": 17444, + "▁Му": 17445, + "▁|\\": 17446, + "added": 17447, + "Produ": 17448, + "▁./": 17449, + "▁awkward": 17450, + "▁Кра": 17451, + "▁ї": 17452, + "▁CONTR": 17453, + "▁beim": 17454, + "▁placeholder": 17455, + "spi": 17456, + "▁Bei": 17457, + "▁Pf": 17458, + "ientes": 17459, + "disk": 17460, + "blk": 17461, + "neo": 17462, + "itarian": 17463, + "▁cogn": 17464, + "▁sout": 17465, + "▁trash": 17466, + "▁Rab": 17467, + "▁decline": 17468, + "tat": 17469, + "▁combine": 17470, + "▁Tot": 17471, + "▁drops": 17472, + "Times": 17473, + "cheduler": 17474, + "▁governments": 17475, + "Tex": 17476, + "▁Used": 17477, + "зан": 17478, + "▁pd": 17479, + "мет": 17480, + "▁&=&": 17481, + "▁Nag": 17482, + "▁дол": 17483, + "▁Always": 17484, + "rtc": 17485, + "ске": 17486, + "▁performances": 17487, + "rupted": 17488, + "▁два": 17489, + "▁managers": 17490, + "▁Pitt": 17491, + "▁mystery": 17492, + "▁settle": 17493, + "ulse": 17494, + "cross": 17495, + "question": 17496, + "asha": 17497, + "seed": 17498, + "urable": 17499, + "Final": 17500, + "++++": 17501, + "inputs": 17502, + "▁backup": 17503, + "▁Learning": 17504, + "▁*,": 17505, + "logo": 17506, + "▁seinen": 17507, + "▁vulnerable": 17508, + "directory": 17509, + "ië": 17510, + "▁friendship": 17511, + "tu": 17512, + "▁Vec": 17513, + "rifice": 17514, + "▁бра": 17515, + "▁involve": 17516, + "TON": 17517, + "▁corrid": 17518, + "separ": 17519, + "Destroy": 17520, + "▁jul": 17521, + "▁inequality": 17522, + "▁ain": 17523, + "hex": 17524, + "▁wider": 17525, + "тели": 17526, + "▁jack": 17527, + "▁quot": 17528, + "▁Glen": 17529, + "initely": 17530, + "ihood": 17531, + "▁waist": 17532, + "▁Manchester": 17533, + "regular": 17534, + "▁(&": 17535, + "▁masses": 17536, + "▁DEFAULT": 17537, + "▁chairs": 17538, + "▁Fast": 17539, + "▁citt": 17540, + "_{{\\": 17541, + "oa": 17542, + "▁$\\{": 17543, + "▁seeds": 17544, + "▁Ald": 17545, + "▁Batt": 17546, + "fab": 17547, + "▁democracy": 17548, + "DTO": 17549, + "▁Hij": 17550, + "PTR": 17551, + "Na": 17552, + "▁Harvard": 17553, + "sid": 17554, + "Pred": 17555, + "fers": 17556, + "▁spare": 17557, + "AMP": 17558, + "▁groupe": 17559, + "▁sender": 17560, + "▁Christopher": 17561, + "▁prisoners": 17562, + "▁Ker": 17563, + "▁Crist": 17564, + "▁ALL": 17565, + "rice": 17566, + "▁antes": 17567, + "natural": 17568, + "▁Susan": 17569, + "▁Juli": 17570, + "▁diab": 17571, + "ixon": 17572, + "icator": 17573, + "▁flexible": 17574, + "▁reserve": 17575, + "Contains": 17576, + "▁Hil": 17577, + "▁Isa": 17578, + "▁towns": 17579, + "GS": 17580, + "▁Trad": 17581, + "▁Lock": 17582, + "▁Grund": 17583, + "▁criticism": 17584, + "ню": 17585, + "▁că": 17586, + "▁politician": 17587, + "stable": 17588, + "Accept": 17589, + "Summary": 17590, + "▁também": 17591, + "}^{-": 17592, + "▁IM": 17593, + "idal": 17594, + "мор": 17595, + "Blue": 17596, + "GROUP": 17597, + "▁terminal": 17598, + "▁complexity": 17599, + "▁locally": 17600, + "DOWN": 17601, + "▁Near": 17602, + "Depth": 17603, + "▁pole": 17604, + "▁equality": 17605, + "Site": 17606, + "▁isinstance": 17607, + "Speed": 17608, + "ippi": 17609, + ",&": 17610, + "▁Enc": 17611, + "щен": 17612, + "▁mater": 17613, + "▁slaves": 17614, + "ACTION": 17615, + "usalem": 17616, + "▁haz": 17617, + "▁Beat": 17618, + "▁wrest": 17619, + "▁llam": 17620, + "Ins": 17621, + "мина": 17622, + "▁був": 17623, + "▁Frame": 17624, + "ushes": 17625, + "▁virtually": 17626, + "▁Perm": 17627, + "▁weights": 17628, + "▁llvm": 17629, + "▁cave": 17630, + "states": 17631, + "DMA": 17632, + "ellt": 17633, + "ifact": 17634, + "vendor": 17635, + "▁Emma": 17636, + "Locale": 17637, + "▁SET": 17638, + "▁geometry": 17639, + "Styles": 17640, + "▁Referee": 17641, + "▁weit": 17642, + "fica": 17643, + "▁ads": 17644, + "gray": 17645, + "▁Burg": 17646, + "iona": 17647, + "dagger": 17648, + "▁Januar": 17649, + "дей": 17650, + "isterschaft": 17651, + "ppo": 17652, + "oids": 17653, + "▁départ": 17654, + "Shader": 17655, + "▁constraint": 17656, + "Secret": 17657, + "▁Peters": 17658, + "▁eyeb": 17659, + "▁mesh": 17660, + "▁cookie": 17661, + "▁Pick": 17662, + "▁nick": 17663, + "bye": 17664, + "▁savings": 17665, + "Try": 17666, + "python": 17667, + "▁patri": 17668, + "▁multip": 17669, + "▁kinda": 17670, + "▁'_": 17671, + "▁Franz": 17672, + "▁cloth": 17673, + "зульта": 17674, + "▁fleet": 17675, + "▁humanity": 17676, + "resa": 17677, + "blob": 17678, + "▁TX": 17679, + "▁Buch": 17680, + "▁Lond": 17681, + "▁valley": 17682, + "▁murm": 17683, + "▁Trade": 17684, + "linewidth": 17685, + "▁especial": 17686, + "upper": 17687, + "▁hosp": 17688, + "▁tanto": 17689, + "▁oldest": 17690, + "▁Roose": 17691, + "▁hitting": 17692, + "dog": 17693, + "ovi": 17694, + "},\r": 17695, + "▁compatible": 17696, + "▁Website": 17697, + "poch": 17698, + "▁Bag": 17699, + "▁accomplish": 17700, + "Christ": 17701, + "asset": 17702, + "▁Until": 17703, + "▁geld": 17704, + "Listen": 17705, + "SB": 17706, + "Setup": 17707, + "icia": 17708, + "▁lum": 17709, + "▁janvier": 17710, + "PAGE": 17711, + "▁Nu": 17712, + "/\"": 17713, + "▁divorce": 17714, + "Execute": 17715, + "Depend": 17716, + "▁Scottish": 17717, + "▁Ts": 17718, + "ruppe": 17719, + "▁refuse": 17720, + "▁Oktober": 17721, + "ijk": 17722, + "▁Amy": 17723, + "▁dimin": 17724, + "▁gross": 17725, + "▁trat": 17726, + "isible": 17727, + "mixer": 17728, + "▁autres": 17729, + "▁neat": 17730, + "▁otros": 17731, + "Void": 17732, + "▁schol": 17733, + "▁Walker": 17734, + "▁tube": 17735, + "ologists": 17736, + "▁груп": 17737, + "▁haben": 17738, + "uber": 17739, + "ACTIVE": 17740, + "▁Attendance": 17741, + "▁оп": 17742, + "▁blade": 17743, + "oplus": 17744, + "▁Original": 17745, + "▁manufacturer": 17746, + "asz": 17747, + "âte": 17748, + "rer": 17749, + "▁Json": 17750, + "▁succeeded": 17751, + "uffle": 17752, + "▁backed": 17753, + "esian": 17754, + "tick": 17755, + "External": 17756, + "▁XIX": 17757, + "▁hearts": 17758, + "▁После": 17759, + "olu": 17760, + "▁лет": 17761, + "VICE": 17762, + "ário": 17763, + "▁fraud": 17764, + "edu": 17765, + "Primary": 17766, + "▁gaming": 17767, + "▁plt": 17768, + "igator": 17769, + "IES": 17770, + "Compiler": 17771, + "▁monument": 17772, + "agem": 17773, + "▁Rain": 17774, + "▁moins": 17775, + "oku": 17776, + "osex": 17777, + "▁Kansas": 17778, + "▁gepublice": 17779, + "▁Joy": 17780, + "Scene": 17781, + "▁kingdom": 17782, + "rices": 17783, + "▁juin": 17784, + "▁uncomfortable": 17785, + "▁Money": 17786, + "obb": 17787, + "expl": 17788, + "strcmp": 17789, + "▁dread": 17790, + "rition": 17791, + "▁Chi": 17792, + "▁demonstrated": 17793, + "▁vertices": 17794, + "чо": 17795, + "▁Culture": 17796, + "FX": 17797, + "Dictionary": 17798, + "▁Dru": 17799, + "trm": 17800, + "▁examine": 17801, + "▁therap": 17802, + "ième": 17803, + "мини": 17804, + "▁produces": 17805, + "▁photographs": 17806, + "▁threads": 17807, + "▁MI": 17808, + "▁extraordinary": 17809, + "ским": 17810, + "▁gepubliceerd": 17811, + "▁Poland": 17812, + "▁guaranteed": 17813, + "RG": 17814, + "osc": 17815, + "али": 17816, + "▁тех": 17817, + "errno": 17818, + "science": 17819, + "iffs": 17820, + "▁Tam": 17821, + "▁Beth": 17822, + "▁Travel": 17823, + "▁translate": 17824, + "ché": 17825, + "▁ling": 17826, + "▁belongs": 17827, + "▁electrical": 17828, + "ensk": 17829, + "▁Compet": 17830, + "cg": 17831, + "VC": 17832, + "topic": 17833, + "▁presum": 17834, + "вета": 17835, + "▁approximation": 17836, + "▁grim": 17837, + "▁Из": 17838, + "_{(": 17839, + "вин": 17840, + "ution": 17841, + "owych": 17842, + "åg": 17843, + "sterreich": 17844, + "▁characteristic": 17845, + "oming": 17846, + "▁/*!": 17847, + "▁prize": 17848, + "▁Minnesota": 17849, + "ted": 17850, + "цы": 17851, + "▁Om": 17852, + "▁indices": 17853, + "▁stem": 17854, + "regon": 17855, + "ниче": 17856, + "▁Salv": 17857, + "ése": 17858, + "▁aged": 17859, + "▁Past": 17860, + "▁internation": 17861, + "▁Vic": 17862, + "▁resume": 17863, + "akespeare": 17864, + "▁estado": 17865, + "▁abilities": 17866, + "▁brow": 17867, + "▁NFL": 17868, + "▁trends": 17869, + "▁Austin": 17870, + "▁LIMIT": 17871, + "▁Kor": 17872, + "▁folk": 17873, + "▁ward": 17874, + "▁nest": 17875, + "▁Junior": 17876, + "▁maintaining": 17877, + "Pub": 17878, + "OBJECT": 17879, + "▁bloody": 17880, + "▁sj": 17881, + "▁dtype": 17882, + "Pane": 17883, + "▁bacter": 17884, + "▁gradually": 17885, + "mr": 17886, + "Team": 17887, + "▁indicating": 17888, + "▁decrease": 17889, + "tek": 17890, + "▁Represent": 17891, + "▁developers": 17892, + "Guid": 17893, + "▁Diet": 17894, + "▁retr": 17895, + "Navigation": 17896, + "esi": 17897, + "▁lazy": 17898, + "Standard": 17899, + "Er": 17900, + "AW": 17901, + "▁États": 17902, + "▁assured": 17903, + "San": 17904, + "▁Andre": 17905, + "’,": 17906, + "fang": 17907, + "ération": 17908, + "▁industries": 17909, + "▁incon": 17910, + "Emit": 17911, + "▁где": 17912, + "▁retriev": 17913, + "eni": 17914, + "▁Turkey": 17915, + "izers": 17916, + "Angle": 17917, + "▁oc": 17918, + "▁palm": 17919, + "▁stan": 17920, + "льно": 17921, + "▁CSS": 17922, + "▁frances": 17923, + "▁grin": 17924, + "▁tiempo": 17925, + "▁Prix": 17926, + "]).": 17927, + "▁deput": 17928, + "▁Pin": 17929, + "▁sixt": 17930, + "▁predicted": 17931, + "azure": 17932, + "▁Motor": 17933, + "▁ihm": 17934, + "▁manus": 17935, + "apos": 17936, + "▁instruments": 17937, + "▁counts": 17938, + "▁aimed": 17939, + "profit": 17940, + "▁dok": 17941, + "обра": 17942, + "▁estud": 17943, + "iesz": 17944, + "▁piss": 17945, + "▁inaug": 17946, + "▁voters": 17947, + "▁packages": 17948, + "▁cute": 17949, + "▁fitness": 17950, + "▁leurs": 17951, + "▁sorted": 17952, + "phant": 17953, + "OPT": 17954, + "▁zip": 17955, + "season": 17956, + "emi": 17957, + "encoding": 17958, + "won": 17959, + "elect": 17960, + "▁tooth": 17961, + "▁upcoming": 17962, + "▁Graham": 17963, + "nut": 17964, + "▁Ark": 17965, + "ält": 17966, + "▁precious": 17967, + "agle": 17968, + "née": 17969, + "ница": 17970, + "aris": 17971, + "▁pile": 17972, + "cole": 17973, + "▁WITH": 17974, + "routing": 17975, + "▁***": 17976, + "Appearance": 17977, + "llvm": 17978, + "▁Oliver": 17979, + "▁PL": 17980, + "ifndef": 17981, + "etzt": 17982, + "skiego": 17983, + "▁pon": 17984, + "ARGET": 17985, + "kö": 17986, + "alled": 17987, + "▁=\\": 17988, + "sure": 17989, + "matches": 17990, + "▁temperatures": 17991, + "SEL": 17992, + "▁clone": 17993, + "▁eller": 17994, + "erna": 17995, + "▁поло": 17996, + "Management": 17997, + "company": 17998, + "▁lun": 17999, + "▁streaming": 18000, + "▁Ni": 18001, + "▁sí": 18002, + "Contact": 18003, + "▁Credit": 18004, + "▁Oak": 18005, + "▁представ": 18006, + "radius": 18007, + "cli": 18008, + "IENT": 18009, + "▁Lucy": 18010, + "▁calculation": 18011, + "▁pixel": 18012, + "▁mul": 18013, + "▁outcomes": 18014, + "▁centers": 18015, + "▁residence": 18016, + "Constraint": 18017, + "▁preserve": 18018, + "peon": 18019, + "uffix": 18020, + "▁Roberts": 18021, + "▁promot": 18022, + "?!": 18023, + "balance": 18024, + "▁courts": 18025, + "▁disg": 18026, + "PRINT": 18027, + "▁их": 18028, + "elfare": 18029, + "▁retreat": 18030, + "▁Ав": 18031, + "Cost": 18032, + "also": 18033, + "▁Für": 18034, + "▁März": 18035, + "DIO": 18036, + "▁bez": 18037, + "AUTH": 18038, + "Den": 18039, + "▁atom": 18040, + "▁roman": 18041, + "▁Pel": 18042, + "▁Roosevelt": 18043, + "▁Plant": 18044, + "Contents": 18045, + "▁Between": 18046, + "▁coupling": 18047, + "structure": 18048, + "▁Marshall": 18049, + "▁Career": 18050, + "▁railway": 18051, + "▁Bureau": 18052, + "▁possibilities": 18053, + "▁kor": 18054, + "){\r": 18055, + "mero": 18056, + "mov": 18057, + "англ": 18058, + "AIN": 18059, + "mund": 18060, + "lette": 18061, + "▁summar": 18062, + "▁describing": 18063, + "▁NAS": 18064, + "▁Emb": 18065, + "Instruction": 18066, + "liest": 18067, + "▁Sig": 18068, + "Bill": 18069, + "▁verd": 18070, + "plant": 18071, + "▁galaxies": 18072, + "\"])": 18073, + "▁PyObject": 18074, + "▁Gy": 18075, + "▁mě": 18076, + "▁organisation": 18077, + "Her": 18078, + "Sep": 18079, + "ocom": 18080, + "▁Same": 18081, + "▁bite": 18082, + "▁Seattle": 18083, + "зыва": 18084, + "Observer": 18085, + "’.": 18086, + "▁morph": 18087, + "urches": 18088, + "alph": 18089, + "reement": 18090, + "consin": 18091, + "^-": 18092, + "▁dann": 18093, + "translate": 18094, + "вих": 18095, + "React": 18096, + "▁cats": 18097, + "▁brew": 18098, + "▁ds": 18099, + "▁circles": 18100, + "▁drift": 18101, + "agma": 18102, + "▁Valent": 18103, + "PIN": 18104, + "ARM": 18105, + "▁surviv": 18106, + "alin": 18107, + "Pref": 18108, + "friendly": 18109, + "▁uncertainty": 18110, + "▁fd": 18111, + "▁engineer": 18112, + "Ben": 18113, + "icular": 18114, + "orest": 18115, + "▁horizontal": 18116, + "UTC": 18117, + "textrm": 18118, + "Live": 18119, + "Score": 18120, + "▁Germans": 18121, + "distance": 18122, + "uti": 18123, + "▁équ": 18124, + "▁numerical": 18125, + "▁reass": 18126, + "Activ": 18127, + "▁cod": 18128, + "bullet": 18129, + "ensing": 18130, + "▁Gem": 18131, + "▁navigation": 18132, + "addClass": 18133, + "▁simultaneously": 18134, + "вий": 18135, + "▁його": 18136, + "▁Hö": 18137, + "▁harsh": 18138, + "precated": 18139, + "ССР": 18140, + "▁Equip": 18141, + "adget": 18142, + "▁TYPE": 18143, + "▁mg": 18144, + "IGH": 18145, + "▁vin": 18146, + "▁findings": 18147, + "ivan": 18148, + "▁possession": 18149, + "▁того": 18150, + "▁parsed": 18151, + "riors": 18152, + "zeichnet": 18153, + "ников": 18154, + "Worker": 18155, + "▁enables": 18156, + "▁($\\": 18157, + "▁Copy": 18158, + "▁orientation": 18159, + "стре": 18160, + "▁Indians": 18161, + "▁Gary": 18162, + "▁Insurance": 18163, + "isan": 18164, + "Chat": 18165, + "▁comun": 18166, + "▁coron": 18167, + "ография": 18168, + "updated": 18169, + "▁Ин": 18170, + "These": 18171, + "SEC": 18172, + "▁boyfriend": 18173, + "Diagnostics": 18174, + "Hint": 18175, + "mul": 18176, + "▁inode": 18177, + "xA": 18178, + "eft": 18179, + "OPTION": 18180, + "unct": 18181, + "annon": 18182, + "ENS": 18183, + "strip": 18184, + "▁enthusi": 18185, + "▁Whit": 18186, + "▁Фи": 18187, + "aude": 18188, + "▁disagree": 18189, + "▁snapped": 18190, + "Phys": 18191, + "▁Syn": 18192, + "▁sour": 18193, + "▁Lux": 18194, + "ugar": 18195, + "tile": 18196, + "▁infection": 18197, + "▁Feb": 18198, + "▁Chem": 18199, + "dataset": 18200, + "chts": 18201, + "Dynamic": 18202, + "▁сред": 18203, + "▁queen": 18204, + "worker": 18205, + "swap": 18206, + "▁timestamp": 18207, + "▁Integr": 18208, + "▁interviews": 18209, + "such": 18210, + "▁laughter": 18211, + "prof": 18212, + "▁Bird": 18213, + "(|": 18214, + "ân": 18215, + "▁gra": 18216, + "&=": 18217, + "zens": 18218, + "getMessage": 18219, + "▁Ost": 18220, + "▁gab": 18221, + "▁mortgage": 18222, + "multicol": 18223, + "LEVEL": 18224, + "partition": 18225, + "seen": 18226, + "▁declar": 18227, + "AU": 18228, + "▁ox": 18229, + "▁ligger": 18230, + "▁Carm": 18231, + "geme": 18232, + "▁Vegas": 18233, + "▁Eug": 18234, + "orus": 18235, + "▁brick": 18236, + "▁así": 18237, + "▁Magazine": 18238, + "HasColumnType": 18239, + "VR": 18240, + "licher": 18241, + "▁Future": 18242, + "▁Jug": 18243, + "attan": 18244, + "constructor": 18245, + "VP": 18246, + "▁тур": 18247, + "чина": 18248, + "Comparator": 18249, + "▁authentic": 18250, + "▁monster": 18251, + "▁transformed": 18252, + "▁firms": 18253, + "FW": 18254, + "▁catalog": 18255, + "boards": 18256, + "▁diseases": 18257, + "▁Benjamin": 18258, + "▁horizon": 18259, + "▁Available": 18260, + "Mvc": 18261, + "Stud": 18262, + "▁lord": 18263, + "general": 18264, + "пар": 18265, + "▁cabinet": 18266, + "▁Basic": 18267, + "TestCase": 18268, + "ansk": 18269, + "▁Snow": 18270, + "ierten": 18271, + "▁vocal": 18272, + "Padding": 18273, + "halt": 18274, + "▁Alexand": 18275, + "▁Colomb": 18276, + "ivamente": 18277, + "▁artificial": 18278, + "▁Atlanta": 18279, + "▁mentre": 18280, + "▁estaba": 18281, + "jekt": 18282, + "▁slept": 18283, + "▁endless": 18284, + "éro": 18285, + "attery": 18286, + "uur": 18287, + "▁weakness": 18288, + "▁attempting": 18289, + "BYTE": 18290, + "▁founder": 18291, + "▁salv": 18292, + "▁Medicine": 18293, + "tid": 18294, + "▁Schwe": 18295, + "raction": 18296, + "▁¿": 18297, + "crate": 18298, + "SERVER": 18299, + "▁compound": 18300, + "▁conve": 18301, + "▁caf": 18302, + "▁handful": 18303, + "onne": 18304, + "ública": 18305, + "▁defensive": 18306, + "Alignment": 18307, + "▁préc": 18308, + "▁significance": 18309, + "élé": 18310, + "arta": 18311, + "Dam": 18312, + "▁perpet": 18313, + "▁caller": 18314, + "icients": 18315, + "cep": 18316, + "▁Multi": 18317, + "▁stolen": 18318, + "▁focusing": 18319, + "embed": 18320, + "▁bree": 18321, + "▁AB": 18322, + "▁occasions": 18323, + "sea": 18324, + "Prov": 18325, + "чение": 18326, + "▁Category": 18327, + "▁sq": 18328, + "▁Фе": 18329, + "VA": 18330, + "Diff": 18331, + "Tri": 18332, + "issement": 18333, + "▁actress": 18334, + "▁Пе": 18335, + "▁jej": 18336, + "▁twisted": 18337, + "▁Nicol": 18338, + "▁junior": 18339, + "Sound": 18340, + "▁Brasil": 18341, + "▁juice": 18342, + "▁>>>": 18343, + "▁Alb": 18344, + "▁softly": 18345, + "▁McK": 18346, + "▁Gren": 18347, + "▁italiano": 18348, + "▁creatures": 18349, + "▁residential": 18350, + "▁Instagram": 18351, + "ucks": 18352, + "▁killer": 18353, + "▁Johnny": 18354, + "▁enterprise": 18355, + "Dto": 18356, + "chestra": 18357, + "▁Tel": 18358, + "▁Activ": 18359, + "factor": 18360, + "oust": 18361, + "▁vacuum": 18362, + "рал": 18363, + "')->": 18364, + "▁Left": 18365, + "▁defect": 18366, + "▁ninete": 18367, + "fare": 18368, + "▁regret": 18369, + "▁shar": 18370, + "ctrine": 18371, + "mesh": 18372, + "city": 18373, + "icit": 18374, + "▁Fem": 18375, + "limited": 18376, + "oka": 18377, + "!\\!\\": 18378, + "Donald": 18379, + "зно": 18380, + "▁provision": 18381, + "▁discussions": 18382, + "Drag": 18383, + "▁Incl": 18384, + "Exit": 18385, + "▁Abd": 18386, + "story": 18387, + "ieve": 18388, + "▁był": 18389, + "olving": 18390, + "wohner": 18391, + "▁guidelines": 18392, + "▁straw": 18393, + "üss": 18394, + "▁було": 18395, + "▁burden": 18396, + "▁spatial": 18397, + "▁stretched": 18398, + "▁Inf": 18399, + "▁typedef": 18400, + "▁robot": 18401, + "▁Doc": 18402, + "pliers": 18403, + "wal": 18404, + "camp": 18405, + "▁diffé": 18406, + "▁McG": 18407, + "▁tel": 18408, + "arette": 18409, + "▁subsequently": 18410, + "▁honey": 18411, + "FUNC": 18412, + "▁establishment": 18413, + "tesy": 18414, + "▁który": 18415, + "▁сель": 18416, + "▁FO": 18417, + "▁Islands": 18418, + "▁mp": 18419, + "Scalar": 18420, + "▁Yan": 18421, + "cken": 18422, + "▁variation": 18423, + "ią": 18424, + "optim": 18425, + "azor": 18426, + "tuple": 18427, + "▁gravity": 18428, + "▁conclude": 18429, + "▁collections": 18430, + "ész": 18431, + "▁Liver": 18432, + "▁ethnic": 18433, + "compile": 18434, + "▁parl": 18435, + "Surface": 18436, + "{'": 18437, + "▁paragraph": 18438, + "posite": 18439, + "ítulo": 18440, + "oba": 18441, + "binary": 18442, + "rob": 18443, + "▁Pedro": 18444, + "▁fis": 18445, + "▁Grande": 18446, + "odox": 18447, + "▁posting": 18448, + "": 26345, + "olent": 26346, + "▁этого": 26347, + "▁Generic": 26348, + "▁*/,": 26349, + "▁combinations": 26350, + "▁rejo": 26351, + "спубли": 26352, + "capacity": 26353, + "▁traces": 26354, + "▁opacity": 26355, + "▁Official": 26356, + "icion": 26357, + "▁emotionally": 26358, + "▁Joel": 26359, + "ському": 26360, + "▁legendary": 26361, + "▁pam": 26362, + "▁También": 26363, + ".<": 26364, + "iba": 26365, + "midt": 26366, + "бом": 26367, + "▁ensuite": 26368, + "Authorization": 26369, + "Pag": 26370, + "▁helmet": 26371, + "▁territo": 26372, + "secondary": 26373, + "▁segunda": 26374, + "▁Wire": 26375, + "recated": 26376, + "▁invoked": 26377, + "▁ValueError": 26378, + "▁фо": 26379, + "ALIGN": 26380, + "CURRENT": 26381, + "\\+\\_\\": 26382, + "▁compilation": 26383, + "ær": 26384, + "▁Palmar": 26385, + "▁influences": 26386, + "/:": 26387, + "Mix": 26388, + "NOP": 26389, + "econom": 26390, + "▁tucked": 26391, + "▁});\r": 26392, + "ANK": 26393, + "reject": 26394, + "▁pension": 26395, + "▁generates": 26396, + "чё": 26397, + "▁incap": 26398, + "▁clicked": 26399, + "▁fus": 26400, + "ourses": 26401, + "▁Easter": 26402, + "%;": 26403, + "zin": 26404, + "▁obligations": 26405, + "▁Tips": 26406, + "};\r": 26407, + ".\"_": 26408, + "▁BSD": 26409, + "ática": 26410, + "▁expose": 26411, + "Pars": 26412, + "▁Amanda": 26413, + "куп": 26414, + "▁guessed": 26415, + "dsi": 26416, + "▁Leip": 26417, + "Broad": 26418, + "▁Hughes": 26419, + "ié": 26420, + "▁Wahl": 26421, + "▁formerly": 26422, + "Relative": 26423, + "▁Yu": 26424, + "▁Mountains": 26425, + "▁Enum": 26426, + "▁strang": 26427, + "_-": 26428, + "recht": 26429, + "viv": 26430, + "pause": 26431, + "▁Londres": 26432, + "▁elbow": 26433, + "▁Hawaii": 26434, + "▁Casino": 26435, + "Threshold": 26436, + "Units": 26437, + "Include": 26438, + "ито": 26439, + "asury": 26440, + "▁steht": 26441, + "▁damned": 26442, + "▁packets": 26443, + "▁Werk": 26444, + "▁elevator": 26445, + "iedad": 26446, + "govern": 26447, + "▁CONTRACT": 26448, + "mals": 26449, + "▁remem": 26450, + "▁entonces": 26451, + "▁vas": 26452, + "▁sympathy": 26453, + "▁befindet": 26454, + "incing": 26455, + "DataSet": 26456, + "▁additionally": 26457, + "▁musician": 26458, + "шего": 26459, + "▁listop": 26460, + ">\")": 26461, + "Printf": 26462, + "▁Felix": 26463, + "▁carved": 26464, + "▁nicely": 26465, + "гом": 26466, + "chap": 26467, + "▁Nieder": 26468, + "▁Lav": 26469, + "▁modifications": 26470, + "moment": 26471, + "▁balcon": 26472, + "▁dependency": 26473, + "CKET": 26474, + "▁vanished": 26475, + "▁fighters": 26476, + "▁zunächst": 26477, + "ioctl": 26478, + "▁defens": 26479, + "▁Nem": 26480, + "Utility": 26481, + "▁curv": 26482, + "▁DAMAGES": 26483, + "▁Rogers": 26484, + "▁gratitude": 26485, + "▁Denmark": 26486, + "рая": 26487, + "grpc": 26488, + "▁juni": 26489, + "▁октября": 26490, + "▁immense": 26491, + "▁prevented": 26492, + "▁foam": 26493, + "▁Extra": 26494, + "aimed": 26495, + "▁Criteria": 26496, + "▁Simply": 26497, + "boxes": 26498, + "▁Legend": 26499, + "▁Players": 26500, + "▁Mercedes": 26501, + "▁Branch": 26502, + "TERN": 26503, + "omena": 26504, + "▁incorporate": 26505, + "conde": 26506, + "▁Estado": 26507, + "▁wasted": 26508, + "▁complaining": 26509, + "▁warriors": 26510, + "oter": 26511, + "▁этом": 26512, + "▁conten": 26513, + "▁machinery": 26514, + "▁technological": 26515, + "▁TD": 26516, + "▁gras": 26517, + "▁minimize": 26518, + "▁Door": 26519, + "▁bzw": 26520, + "▁prac": 26521, + "TREE": 26522, + "▁Wing": 26523, + "▁Transaction": 26524, + "▁MVT": 26525, + "▁Klein": 26526, + "commons": 26527, + "▁}{": 26528, + "▁Heritage": 26529, + "▁fade": 26530, + "рок": 26531, + "setValue": 26532, + "▁Wallace": 26533, + "MX": 26534, + "▁ACT": 26535, + "▁footage": 26536, + "▁entstand": 26537, + "arga": 26538, + "▁nails": 26539, + "▁capitalism": 26540, + "▁Garc": 26541, + "▁suspension": 26542, + "ilis": 26543, + "▁Mov": 26544, + "uffled": 26545, + "Arc": 26546, + "▁Beautiful": 26547, + "WAY": 26548, + "Parallel": 26549, + "XXXX": 26550, + "diag": 26551, + "▁DT": 26552, + "mq": 26553, + "TextView": 26554, + "MLE": 26555, + "ennen": 26556, + "▁infected": 26557, + "▁therapist": 26558, + "INGS": 26559, + "▁cidade": 26560, + "ън": 26561, + "▁pdf": 26562, + "▁bump": 26563, + "CTX": 26564, + "▁INCLUDING": 26565, + "▁Gef": 26566, + "ENTIAL": 26567, + "▁handy": 26568, + "▁temporal": 26569, + "AtA": 26570, + "ISH": 26571, + "▁Pattern": 26572, + "▁lan": 26573, + "ependant": 26574, + "▁shining": 26575, + "idy": 26576, + "▁NT": 26577, + "▁Fran": 26578, + "▁nurses": 26579, + "▁betray": 26580, + "▁sensible": 26581, + "▁апреля": 26582, + "▁'[": 26583, + "▁thirteen": 26584, + ")}_{": 26585, + "▁Noah": 26586, + "INSERT": 26587, + "istically": 26588, + "▁Appendix": 26589, + "▁recher": 26590, + "Receiver": 26591, + "▁dernier": 26592, + "лла": 26593, + "лиза": 26594, + "▁Partido": 26595, + "▁maximal": 26596, + "snap": 26597, + "▁часть": 26598, + "STOP": 26599, + "▁ultra": 26600, + "▁développ": 26601, + "▁tegen": 26602, + "▁Чи": 26603, + "LIB": 26604, + "▁baseline": 26605, + "reload": 26606, + "▁Arbitro": 26607, + "▁kall": 26608, + "capture": 26609, + "Arm": 26610, + "quin": 26611, + "impse": 26612, + "zas": 26613, + "▁Cand": 26614, + "▁brains": 26615, + "▁hostile": 26616, + "▁marble": 26617, + "oons": 26618, + "▁Loss": 26619, + "MetaData": 26620, + "▁República": 26621, + "▁andra": 26622, + "oden": 26623, + "▁documented": 26624, + "▁Moses": 26625, + "odd": 26626, + "▁wax": 26627, + "usch": 26628, + "▁diagnosed": 26629, + "inkle": 26630, + "▁Xbox": 26631, + "▁seventy": 26632, + "cias": 26633, + "▁noviembre": 26634, + "Compute": 26635, + "});\r": 26636, + "▁Philippe": 26637, + "▁För": 26638, + "Leave": 26639, + "▁sage": 26640, + "▁unpre": 26641, + "▁Fortunately": 26642, + "▁apost": 26643, + "entities": 26644, + "▁ellos": 26645, + "authorized": 26646, + "GBT": 26647, + "▁insist": 26648, + "▁inspire": 26649, + "Mass": 26650, + "▁rôle": 26651, + "fee": 26652, + "ipart": 26653, + "цер": 26654, + "unate": 26655, + "▁CNN": 26656, + ":}": 26657, + "▁unhappy": 26658, + "▁imported": 26659, + "HIGH": 26660, + "rings": 26661, + "▁Instance": 26662, + "Bay": 26663, + "agles": 26664, + "mee": 26665, + "bery": 26666, + "▁Stories": 26667, + "▁Chase": 26668, + "▁carriage": 26669, + "▁misunder": 26670, + "▁imagin": 26671, + "pw": 26672, + "▁Meter": 26673, + "▁crowds": 26674, + "▁Fame": 26675, + "skill": 26676, + "▁comed": 26677, + "▁ranch": 26678, + "▁lacking": 26679, + "▁submar": 26680, + "iante": 26681, + "▁lanz": 26682, + "▁служ": 26683, + "-----------": 26684, + "▁obten": 26685, + "▁downstairs": 26686, + "YN": 26687, + "rotation": 26688, + "▁Jesse": 26689, + "$(\"#": 26690, + "▁puls": 26691, + "irling": 26692, + "▁Schaus": 26693, + "▁deployed": 26694, + "▁{}\",": 26695, + "▁Marvel": 26696, + "ENUM": 26697, + "▁Mathemat": 26698, + "▁nn": 26699, + "compet": 26700, + "ków": 26701, + "bil": 26702, + "Which": 26703, + "isine": 26704, + "▁rude": 26705, + "▁niveau": 26706, + "▁área": 26707, + "▁près": 26708, + "atis": 26709, + "▁[...]": 26710, + "fur": 26711, + "omm": 26712, + "packed": 26713, + "мене": 26714, + "scriptstyle": 26715, + "▁Ath": 26716, + "▁desp": 26717, + "eltemperaturen": 26718, + "▁talents": 26719, + "ocy": 26720, + "▁raises": 26721, + "LIMIT": 26722, + "▁editorial": 26723, + "▁Animal": 26724, + "drive": 26725, + "▁работа": 26726, + "bss": 26727, + "▁Sev": 26728, + "epoch": 26729, + "▁RC": 26730, + "UNUSED": 26731, + "▁mandatory": 26732, + "(?:": 26733, + "▁Bin": 26734, + "▁synthetic": 26735, + "▁gown": 26736, + "▁Dob": 26737, + "kap": 26738, + "▁harmon": 26739, + "▁liberty": 26740, + "▁Rice": 26741, + "▁prayers": 26742, + "▁mise": 26743, + "▁confusing": 26744, + "▁leap": 26745, + "▁arrives": 26746, + "kamp": 26747, + "▁thats": 26748, + "ACC": 26749, + "▁Parameters": 26750, + "▁одно": 26751, + "▁Bio": 26752, + "density": 26753, + "▁glimpse": 26754, + "FORE": 26755, + "▁Listen": 26756, + "Prev": 26757, + "}\\,\\": 26758, + "куль": 26759, + "▁SEC": 26760, + "▁explored": 26761, + "▁meantime": 26762, + "AIL": 26763, + "▁WP": 26764, + "▁raison": 26765, + "▁existe": 26766, + "▁lesser": 26767, + "▁Validate": 26768, + "▁caution": 26769, + "usta": 26770, + "heading": 26771, + "EFF": 26772, + ".'\"": 26773, + "▁Gilbert": 26774, + "▁limitation": 26775, + "▁retour": 26776, + "▁Commonwealth": 26777, + "▁gewann": 26778, + "▁miserable": 26779, + "▁networking": 26780, + "▁ottobre": 26781, + "▁Dise": 26782, + "edges": 26783, + "▁sede": 26784, + "вича": 26785, + "uniform": 26786, + "▁деятель": 26787, + "iros": 26788, + "▁desen": 26789, + "▁parc": 26790, + "▁Rico": 26791, + "Ns": 26792, + "guid": 26793, + "orio": 26794, + "avelength": 26795, + "▁Gle": 26796, + "inceton": 26797, + "Amaz": 26798, + "Construct": 26799, + "▁mx": 26800, + "▁Vern": 26801, + "▁Generation": 26802, + "Jack": 26803, + "romag": 26804, + "▁viagra": 26805, + "▁Peg": 26806, + "▁Updated": 26807, + "▁overlap": 26808, + "EventArgs": 26809, + "кро": 26810, + "▁*«": 26811, + "▁questioned": 26812, + "South": 26813, + "notice": 26814, + "▁permanently": 26815, + "lst": 26816, + "ficie": 26817, + "▁quella": 26818, + "▁colleges": 26819, + "▁disappointment": 26820, + "▁Luft": 26821, + "imgur": 26822, + "▁transitions": 26823, + "▁seller": 26824, + "▁июня": 26825, + "▁Og": 26826, + "▁ADD": 26827, + "▁Pays": 26828, + "COMMAND": 26829, + "grades": 26830, + "▁febbra": 26831, + "▁Cyr": 26832, + "▁febbraio": 26833, + "eti": 26834, + "▁arom": 26835, + "▁Claude": 26836, + "▁UEFA": 26837, + "▁живе": 26838, + "▁Victorian": 26839, + "keeping": 26840, + "ên": 26841, + "▁FIXME": 26842, + "itime": 26843, + "chestr": 26844, + "▁Samsung": 26845, + "▁doctrine": 26846, + "▁pear": 26847, + "▁Mediterranean": 26848, + "▁Ya": 26849, + "▁vault": 26850, + "▁Historic": 26851, + "▁sedan": 26852, + "▁heated": 26853, + "▁política": 26854, + "Proof": 26855, + ":{": 26856, + "fem": 26857, + "▁Frankfurt": 26858, + "pectives": 26859, + "MG": 26860, + "▁Eye": 26861, + "dai": 26862, + "▁reserves": 26863, + "NER": 26864, + "▁tobacco": 26865, + "▁fragments": 26866, + "icc": 26867, + "▁booth": 26868, + "▁cruise": 26869, + "▁Testament": 26870, + "cola": 26871, + "▁Leop": 26872, + "▁noon": 26873, + "▁terrified": 26874, + "vb": 26875, + "intel": 26876, + "alie": 26877, + "▁verification": 26878, + "yster": 26879, + "ADER": 26880, + "chied": 26881, + "▁datasets": 26882, + "▁зі": 26883, + "▁miem": 26884, + "ulates": 26885, + "▁uuid": 26886, + "▁Pictures": 26887, + "▁Brend": 26888, + "Billboard": 26889, + "▁stern": 26890, + "▁denom": 26891, + "▁accidents": 26892, + "сня": 26893, + "▁packing": 26894, + "ција": 26895, + "iblical": 26896, + "▁Так": 26897, + "▁whisk": 26898, + "▁luego": 26899, + "▁rectangle": 26900, + "▁hooks": 26901, + "▁neglect": 26902, + "▁sober": 26903, + "proposition": 26904, + "Multiple": 26905, + ":\",": 26906, + "▁bapt": 26907, + "Parts": 26908, + "▁Selection": 26909, + "▁Alpha": 26910, + "weights": 26911, + "hall": 26912, + "соб": 26913, + "▁lur": 26914, + "▁época": 26915, + "▁rested": 26916, + "ambigu": 26917, + "▁tastes": 26918, + "amazonaws": 26919, + "▁confess": 26920, + "▁diciembre": 26921, + "implement": 26922, + "▁absorption": 26923, + "Hal": 26924, + "LEAN": 26925, + "▁Zach": 26926, + "▁freeze": 26927, + "LBL": 26928, + "STM": 26929, + "▁calc": 26930, + "={()": 26931, + "=*/": 26932, + "▁bt": 26933, + "Reb": 26934, + "▁Wien": 26935, + "anska": 26936, + "▁surn": 26937, + "iative": 26938, + "▁invån": 26939, + "CY": 26940, + "▁là": 26941, + "amba": 26942, + "leen": 26943, + "wahl": 26944, + "▁functioning": 26945, + "ția": 26946, + "getContext": 26947, + "gart": 26948, + "▁обе": 26949, + "Pen": 26950, + "vik": 26951, + "Slider": 26952, + "▁Accept": 26953, + "Gap": 26954, + "▁Jorge": 26955, + "SIG": 26956, + "▁вос": 26957, + "▁голо": 26958, + "▁periodo": 26959, + "шта": 26960, + "▁patches": 26961, + "кої": 26962, + "äre": 26963, + "engono": 26964, + "lista": 26965, + "horn": 26966, + "▁Complex": 26967, + "Sent": 26968, + "trfs": 26969, + "▁convex": 26970, + "Generation": 26971, + "▁місце": 26972, + "compress": 26973, + "▁Sax": 26974, + "▁uid": 26975, + "▁Lebens": 26976, + "Completion": 26977, + "\\|_{": 26978, + "insky": 26979, + "▁schon": 26980, + "▁masters": 26981, + "independ": 26982, + "neys": 26983, + "▁lied": 26984, + "▁aspir": 26985, + "чні": 26986, + "▁breakdown": 26987, + "▁Harm": 26988, + "▁designing": 26989, + "hf": 26990, + "▁Angela": 26991, + "▁confer": 26992, + "▁partido": 26993, + "▁interference": 26994, + "mao": 26995, + "▁absorbed": 26996, + "▁Vall": 26997, + "ErrorCode": 26998, + "▁Publishing": 26999, + "vano": 27000, + "BITS": 27001, + "▁deer": 27002, + "▁Campaign": 27003, + "▁graz": 27004, + "CHANGE": 27005, + "▁feder": 27006, + "iffe": 27007, + "handed": 27008, + "cq": 27009, + "umbing": 27010, + "▁unre": 27011, + "▁siendo": 27012, + "▁simpler": 27013, + "why": 27014, + "arettes": 27015, + "anst": 27016, + "▁hass": 27017, + "▁Enterprise": 27018, + "▁mois": 27019, + "▁Fo": 27020, + "▁участ": 27021, + "ffen": 27022, + "▁MODULE": 27023, + "▁activated": 27024, + "▁internacional": 27025, + "▁Mittel": 27026, + "degree": 27027, + "▁откры": 27028, + "▁&(": 27029, + "getProperty": 27030, + "isz": 27031, + "cedure": 27032, + "▁enters": 27033, + "▁Sally": 27034, + "▁Train": 27035, + "▁logged": 27036, + "▁Rav": 27037, + "▁Avoid": 27038, + "▁Kaiser": 27039, + "▁expend": 27040, + "aphor": 27041, + "▁brass": 27042, + "▁melod": 27043, + "▁attitudes": 27044, + "*\"": 27045, + "Wall": 27046, + "▁owe": 27047, + "▁bamb": 27048, + "shader": 27049, + "cester": 27050, + "▁PP": 27051, + "▁migrations": 27052, + "entric": 27053, + "▁Setup": 27054, + "▁Artist": 27055, + "hre": 27056, + "▁polite": 27057, + "ahan": 27058, + "▁luglio": 27059, + "▁predecess": 27060, + "▁SIG": 27061, + "тів": 27062, + "▁RF": 27063, + "▁Dry": 27064, + "▁maker": 27065, + "шим": 27066, + "▁Sounds": 27067, + "▁implementing": 27068, + "▁ah": 27069, + "▁gev": 27070, + "▁duplicate": 27071, + "▁Logan": 27072, + "▁Grade": 27073, + "DUCT": 27074, + "íses": 27075, + "ért": 27076, + "▁nonsense": 27077, + "backup": 27078, + "Attachment": 27079, + "▁ecc": 27080, + "▁Squadron": 27081, + "learn": 27082, + "deprecated": 27083, + "▁Aub": 27084, + "▁Gol": 27085, + "▁overl": 27086, + "SERVICE": 27087, + "▁beautifully": 27088, + "REL": 27089, + "▁Gian": 27090, + "▁Papa": 27091, + "respond": 27092, + "▁Caribbean": 27093, + "rn": 27094, + "▁худож": 27095, + "Cfg": 27096, + "rai": 27097, + "▁sniff": 27098, + "tto": 27099, + "ологи": 27100, + "▁rb": 27101, + "▁incidents": 27102, + "▁duck": 27103, + "▁PROVIDED": 27104, + "Sources": 27105, + "▁Chelsea": 27106, + "▁tek": 27107, + "▁налази": 27108, + "▁pilots": 27109, + "тки": 27110, + "▁traded": 27111, + "▁Beijing": 27112, + "▁Gregory": 27113, + "scalar": 27114, + "▁inclined": 27115, + "▁Kamp": 27116, + "▁Marian": 27117, + "▁fierce": 27118, + "▁theft": 27119, + "ющих": 27120, + "▁Into": 27121, + "constraint": 27122, + "parentNode": 27123, + "idental": 27124, + "▁gouvernement": 27125, + "▁SND": 27126, + "▁Ruby": 27127, + "▁monaster": 27128, + "Records": 27129, + "▁Kab": 27130, + "▁Universe": 27131, + "▁approximate": 27132, + "Water": 27133, + "▁Physical": 27134, + "appers": 27135, + "oubtedly": 27136, + "ложен": 27137, + "▁towel": 27138, + "▁siblings": 27139, + "eph": 27140, + "icios": 27141, + "рами": 27142, + "▁outrage": 27143, + "▁també": 27144, + "SRC": 27145, + "телем": 27146, + "Vi": 27147, + ".');": 27148, + "LM": 27149, + "▁mitt": 27150, + "▁weed": 27151, + "▁crops": 27152, + "iman": 27153, + "Claim": 27154, + "insula": 27155, + "▁(“": 27156, + "▁Changes": 27157, + "▁invånare": 27158, + "again": 27159, + "▁cnt": 27160, + "▁Gaz": 27161, + "▁austral": 27162, + "overlay": 27163, + "▁Mechan": 27164, + "▁slammed": 27165, + "▁trailing": 27166, + "▁Biography": 27167, + "▁appealing": 27168, + "IVER": 27169, + "▁Ave": 27170, + "▁Plot": 27171, + "voj": 27172, + "▁sung": 27173, + "▁unos": 27174, + "Effects": 27175, + "vv": 27176, + "cook": 27177, + "Buttons": 27178, + "▁transm": 27179, + "ierto": 27180, + "CONTEXT": 27181, + "▁dignity": 27182, + "aired": 27183, + "javax": 27184, + "▁Alberto": 27185, + "▁Recently": 27186, + "▁facial": 27187, + "mathop": 27188, + "ało": 27189, + "вид": 27190, + "cott": 27191, + "Variables": 27192, + "▁Ran": 27193, + "▁bunk": 27194, + "amiliar": 27195, + "CAST": 27196, + "▁frü": 27197, + "VED": 27198, + "▁NOTICE": 27199, + "▁turno": 27200, + "validator": 27201, + "▁Portuguese": 27202, + "▁questioning": 27203, + "}})": 27204, + "▁lear": 27205, + "Xamarin": 27206, + "▁disadv": 27207, + "encoded": 27208, + "▁Kot": 27209, + "rated": 27210, + "▁Theory": 27211, + "cius": 27212, + "▁Darwin": 27213, + "ђе": 27214, + "▁décl": 27215, + "▁область": 27216, + "рович": 27217, + "▁mobility": 27218, + "VF": 27219, + "▁хи": 27220, + "until": 27221, + "▁barriers": 27222, + "gif": 27223, + "▁Roh": 27224, + "▁aging": 27225, + "▁Widget": 27226, + "olk": 27227, + "▁farms": 27228, + "Checker": 27229, + "Introduction": 27230, + "смо": 27231, + "▁Russians": 27232, + "naments": 27233, + "▁Insert": 27234, + "▁Whenever": 27235, + "erset": 27236, + "itori": 27237, + "▁Dort": 27238, + "▁costume": 27239, + "▁mathematical": 27240, + "▁Bast": 27241, + "▁nominated": 27242, + "▁restoration": 27243, + "posal": 27244, + "▁unfortunate": 27245, + "Ps": 27246, + "LIN": 27247, + "▁intact": 27248, + "▁provoc": 27249, + "▁située": 27250, + "▁ноября": 27251, + "ermo": 27252, + "▁fisher": 27253, + "гля": 27254, + "▁conting": 27255, + "▁Doug": 27256, + "\"?": 27257, + "▁Eva": 27258, + "▁tops": 27259, + "▁Remote": 27260, + "▁artwork": 27261, + "▁artillery": 27262, + "quick": 27263, + "▁Arabia": 27264, + "▁SDValue": 27265, + "▁Dakota": 27266, + "iated": 27267, + "▁Optim": 27268, + "buttons": 27269, + "▁cottage": 27270, + "▁wherein": 27271, + "▁tutorial": 27272, + "▁Scre": 27273, + "▁sweep": 27274, + "▁Coffee": 27275, + "})}": 27276, + "▁музы": 27277, + "hostname": 27278, + "▁Temp": 27279, + "▁Fut": 27280, + "respect": 27281, + "ocz": 27282, + "▁predomin": 27283, + "Indicator": 27284, + "encial": 27285, + "UMENT": 27286, + "▁SHALL": 27287, + "▁commanded": 27288, + "▁withdrawal": 27289, + "iour": 27290, + "REGION": 27291, + "sprintf": 27292, + "▁вме": 27293, + "▁Payment": 27294, + "▁Anim": 27295, + "publish": 27296, + "▁seeks": 27297, + "ouw": 27298, + "▁GM": 27299, + "rugu": 27300, + "ustain": 27301, + "▁))": 27302, + "▁consulting": 27303, + "▁Dialog": 27304, + "▁Lars": 27305, + "▁critique": 27306, + "▁circulation": 27307, + "▁landsc": 27308, + "managed": 27309, + "▁Craft": 27310, + "▁herman": 27311, + "afi": 27312, + "amy": 27313, + "▁discour": 27314, + "<>(": 27315, + "▁Steph": 27316, + "▁tolerance": 27317, + "typename": 27318, + "ventions": 27319, + "ział": 27320, + "стов": 27321, + "▁sticking": 27322, + "ASC": 27323, + "ISO": 27324, + "▁Spencer": 27325, + "▁Didn": 27326, + "gomery": 27327, + "imiter": 27328, + "dru": 27329, + "Clause": 27330, + "▁slides": 27331, + "###": 27332, + "▁Sugar": 27333, + "HY": 27334, + "▁эти": 27335, + "▁Edwards": 27336, + "▁cents": 27337, + "oya": 27338, + "serts": 27339, + "▁Hass": 27340, + "▁ingen": 27341, + "стри": 27342, + "▁saddle": 27343, + "solid": 27344, + "▁champions": 27345, + "-)": 27346, + "▁Slov": 27347, + "▁shiny": 27348, + "▁*)&": 27349, + "▁Define": 27350, + "če": 27351, + "▁scrut": 27352, + "onden": 27353, + "'\",": 27354, + "uffs": 27355, + "▁olymp": 27356, + "idential": 27357, + "wand": 27358, + "▁annually": 27359, + "▁Arkansas": 27360, + "▁saint": 27361, + "▁gleich": 27362, + "▁perfection": 27363, + ")>": 27364, + "▁shorts": 27365, + "▁justified": 27366, + "peated": 27367, + "packages": 27368, + "driven": 27369, + "▁Liberty": 27370, + "▁stripped": 27371, + "шение": 27372, + "▁fünf": 27373, + "▁ecosystem": 27374, + "ixa": 27375, + "▁Fresh": 27376, + "vart": 27377, + "▁treats": 27378, + "▁stance": 27379, + "чёт": 27380, + "▁pity": 27381, + "adém": 27382, + "▁окон": 27383, + "▁Chand": 27384, + "rab": 27385, + "вший": 27386, + "inski": 27387, + "▁continually": 27388, + "▁Daddy": 27389, + "▁nightmare": 27390, + "icional": 27391, + "▁efect": 27392, + "ueblo": 27393, + "▁lanç": 27394, + "▁Collections": 27395, + "due": 27396, + "ampton": 27397, + "▁memcpy": 27398, + "▁**(": 27399, + "issent": 27400, + "▁Insp": 27401, + "▁Glasgow": 27402, + "▁furono": 27403, + "▁kindness": 27404, + "Bi": 27405, + "▁competed": 27406, + "▁oak": 27407, + "Large": 27408, + "▁disgu": 27409, + "▁kings": 27410, + "тами": 27411, + "▁stuffed": 27412, + "▁hilar": 27413, + "published": 27414, + "▁stressed": 27415, + "▁Peak": 27416, + "▁loader": 27417, + "Keyboard": 27418, + "▁reconstruction": 27419, + "▁vod": 27420, + "▁dun": 27421, + "▁understands": 27422, + "tenant": 27423, + "▁chaque": 27424, + "▁prejud": 27425, + "utat": 27426, + "▁uso": 27427, + "▁Heavy": 27428, + "▁cuatro": 27429, + "▁sidewalk": 27430, + "▁Bug": 27431, + "▁månaden": 27432, + "geo": 27433, + "▁united": 27434, + "▁Files": 27435, + "▁Аль": 27436, + "▁rugby": 27437, + "▁financing": 27438, + "▁comply": 27439, + "&#": 27440, + "▁rushing": 27441, + "▁fen": 27442, + "mong": 27443, + "▁spé": 27444, + "▁presenting": 27445, + "INCLUDING": 27446, + "ěl": 27447, + "zeichnung": 27448, + "Backup": 27449, + "▁petit": 27450, + "▁allerg": 27451, + "нут": 27452, + "▁worrying": 27453, + "▁mamm": 27454, + "▁operand": 27455, + ":%.*]]": 27456, + "▁realise": 27457, + "Commands": 27458, + "▁Bew": 27459, + "▁assumes": 27460, + "▁Covid": 27461, + "▁quand": 27462, + "tyard": 27463, + "▁Mono": 27464, + "linked": 27465, + "MARK": 27466, + "Esp": 27467, + "▁blessing": 27468, + "▁eyebrows": 27469, + "▁NV": 27470, + "▁стру": 27471, + "▁modeling": 27472, + "▁greeted": 27473, + "Workspace": 27474, + "▁pedest": 27475, + "▁неза": 27476, + "lemagne": 27477, + "Statistics": 27478, + "▁aument": 27479, + "▁speeds": 27480, + "▁syndrome": 27481, + "CONNECT": 27482, + "zahl": 27483, + "verso": 27484, + "ército": 27485, + "▁astronom": 27486, + "▁aprile": 27487, + "žen": 27488, + "веро": 27489, + "draft": 27490, + "▁gioc": 27491, + "▁comport": 27492, + "▁variance": 27493, + "▁realizing": 27494, + "EDIT": 27495, + "олові": 27496, + "▁estar": 27497, + "▁sost": 27498, + "NORMAL": 27499, + "▁ó": 27500, + "▁Andr": 27501, + "ATTRIB": 27502, + "▁rede": 27503, + "▁toes": 27504, + "▁advances": 27505, + "▁Against": 27506, + "TOM": 27507, + "rss": 27508, + "MMMM": 27509, + "▁newest": 27510, + "▁VER": 27511, + "▁phrases": 27512, + "anter": 27513, + "Launch": 27514, + "▁chr": 27515, + "▁manufactured": 27516, + "$),": 27517, + "rollment": 27518, + "eston": 27519, + "▁peint": 27520, + "”)": 27521, + "endet": 27522, + "▁Hair": 27523, + "ivalent": 27524, + "▁upright": 27525, + "gren": 27526, + "anked": 27527, + "wright": 27528, + "▁mast": 27529, + "▁onChange": 27530, + "▁debris": 27531, + "▁grap": 27532, + "etry": 27533, + "▁(__": 27534, + "▁Commerce": 27535, + "BOX": 27536, + "Tax": 27537, + "▁отри": 27538, + "▁prevention": 27539, + "▁Feel": 27540, + "▁exotic": 27541, + "▁Bark": 27542, + "▁Steam": 27543, + "fon": 27544, + "olin": 27545, + "▁eliminated": 27546, + "▁bc": 27547, + "▁Cycl": 27548, + "▁$(\"#": 27549, + "▁Parl": 27550, + "manuel": 27551, + "ospher": 27552, + "WF": 27553, + "Analy": 27554, + "▁navig": 27555, + "▁renown": 27556, + "Rx": 27557, + "▁Walt": 27558, + "uffed": 27559, + "▁foster": 27560, + "$:": 27561, + "shore": 27562, + "Connector": 27563, + "фика": 27564, + "▁realization": 27565, + "Li": 27566, + "ctxt": 27567, + "ahoo": 27568, + "▁miracle": 27569, + "▁ET": 27570, + "▁GPS": 27571, + "▁Observable": 27572, + "▁hf": 27573, + "▁magnificent": 27574, + "него": 27575, + "BIN": 27576, + "▁Dorf": 27577, + "ieck": 27578, + "vee": 27579, + "▁Craw": 27580, + "/#": 27581, + "▁pci": 27582, + "ippet": 27583, + "▁Hillary": 27584, + "▁gir": 27585, + "▁rand": 27586, + "▁laying": 27587, + "▁Different": 27588, + "boys": 27589, + "virt": 27590, + "▁encryption": 27591, + "ász": 27592, + "пор": 27593, + "▁smelled": 27594, + "▁suscept": 27595, + "cluded": 27596, + "▁Carn": 27597, + "igten": 27598, + "▁Chuck": 27599, + "▁Provinc": 27600, + "▁perí": 27601, + "▁Marshal": 27602, + "мож": 27603, + "gfx": 27604, + "oshi": 27605, + "▁WHE": 27606, + "▁relaxation": 27607, + ",.": 27608, + "were": 27609, + "▁varieties": 27610, + "▁Won": 27611, + "▁gaps": 27612, + "▁stole": 27613, + "igua": 27614, + "ющие": 27615, + "▁Hampshire": 27616, + "phrase": 27617, + "▁película": 27618, + "Processing": 27619, + "▁initialization": 27620, + "oustic": 27621, + "▁Josef": 27622, + "icating": 27623, + "▁goodness": 27624, + "TES": 27625, + "▁cope": 27626, + "▁ignorance": 27627, + "▁Brist": 27628, + "▁paras": 27629, + "▁accidentally": 27630, + "▁tand": 27631, + "ittest": 27632, + "▁ули": 27633, + "▁shipped": 27634, + "▁ост": 27635, + "elseif": 27636, + "▁usize": 27637, + "horizontal": 27638, + "▁Carr": 27639, + "▁precip": 27640, + "roz": 27641, + "pathetic": 27642, + "rived": 27643, + "rok": 27644, + "▁digging": 27645, + "мом": 27646, + "▁Mull": 27647, + "▁XIII": 27648, + "▁peas": 27649, + "▁foul": 27650, + "▁travels": 27651, + "▁Ng": 27652, + "▁составе": 27653, + "Mont": 27654, + "arde": 27655, + "▁Stefan": 27656, + "^^^^": 27657, + "▁Kiss": 27658, + "▁Ek": 27659, + "▁oktober": 27660, + "▁memorable": 27661, + "')).": 27662, + "▁Vision": 27663, + "▁Nina": 27664, + "▁Solar": 27665, + "▁highlighted": 27666, + "▁memo": 27667, + "meisterschaft": 27668, + "sidebar": 27669, + "SEE": 27670, + "▁Nevada": 27671, + "Da": 27672, + "▁drawer": 27673, + "astically": 27674, + "elde": 27675, + "scribed": 27676, + "▁priests": 27677, + "▁hommes": 27678, + "▁instructor": 27679, + "клад": 27680, + "▁spett": 27681, + "\\-": 27682, + "▁мира": 27683, + "▁Looks": 27684, + "▁sleeve": 27685, + "▁strongest": 27686, + "▁tête": 27687, + "▁Nicole": 27688, + "imper": 27689, + "нача": 27690, + "ipper": 27691, + "▁inwon": 27692, + "ilers": 27693, + "▁Deputy": 27694, + "oge": 27695, + "▁depressed": 27696, + "▁arte": 27697, + "▁combining": 27698, + "LAST": 27699, + "inted": 27700, + "▁Average": 27701, + "▁pollution": 27702, + "▁Phillips": 27703, + "▁WM": 27704, + "}}}\\": 27705, + "Added": 27706, + "▁peripher": 27707, + "Creation": 27708, + "▁italien": 27709, + "▁Choice": 27710, + "▁EXPRESS": 27711, + "▁Struct": 27712, + "ysz": 27713, + "Resize": 27714, + "ARGS": 27715, + "▁repo": 27716, + "▁чтобы": 27717, + "▁pref": 27718, + "▁earthqu": 27719, + "▁Мекси": 27720, + "▁Finale": 27721, + "▁hecho": 27722, + "requests": 27723, + "Cut": 27724, + "▁deserved": 27725, + "гово": 27726, + "▁Recent": 27727, + "▁дивизи": 27728, + "▁supportive": 27729, + "прави": 27730, + "▁irrelevant": 27731, + "'\r": 27732, + "▁ctrl": 27733, + "▁Deal": 27734, + "izada": 27735, + "uo": 27736, + "▁nort": 27737, + "geometry": 27738, + "▁Individual": 27739, + "ereg": 27740, + "▁приня": 27741, + "cref": 27742, + "══": 27743, + "▁comerc": 27744, + "=_": 27745, + "bund": 27746, + "тах": 27747, + "ilen": 27748, + "чита": 27749, + "▁corporation": 27750, + "esz": 27751, + "▁==>": 27752, + "ablish": 27753, + "Apr": 27754, + "▁ripped": 27755, + "Vars": 27756, + "stret": 27757, + "▁Francesco": 27758, + "NaN": 27759, + "▁anytime": 27760, + "▁automated": 27761, + "ostream": 27762, + "▁drawings": 27763, + "▁enhancement": 27764, + "okrat": 27765, + "▁Issue": 27766, + "вра": 27767, + "Currency": 27768, + "▁wyn": 27769, + "izarre": 27770, + "ético": 27771, + "multiple": 27772, + "▁Rate": 27773, + "▁Ich": 27774, + "▁Auss": 27775, + "▁Former": 27776, + "Curve": 27777, + "▁marvel": 27778, + "attro": 27779, + "▁сп": 27780, + "BOOL": 27781, + "сия": 27782, + "gold": 27783, + "▁Nintendo": 27784, + "▁Salvador": 27785, + "▁Solution": 27786, + "ADC": 27787, + "бора": 27788, + "▁Bennett": 27789, + "▁FR": 27790, + "▁pueden": 27791, + "patient": 27792, + "▁PG": 27793, + "▁Jin": 27794, + "▁crashed": 27795, + "▁denen": 27796, + "▁Sample": 27797, + "▁Quebec": 27798, + "itories": 27799, + "▁blinked": 27800, + "▁lion": 27801, + "▁voce": 27802, + "▁Impact": 27803, + "▁Mau": 27804, + "▁Nie": 27805, + "▁lob": 27806, + "▁две": 27807, + "orneys": 27808, + "▁coastal": 27809, + "▁sensors": 27810, + "▁XII": 27811, + "▁illusion": 27812, + "oji": 27813, + "▁INC": 27814, + "▁Duncan": 27815, + "yk": 27816, + "▁affecting": 27817, + "pul": 27818, + "▁Napoleon": 27819, + "▁акаде": 27820, + "▁compt": 27821, + "▁profitable": 27822, + "loe": 27823, + "▁deuxième": 27824, + "▁WC": 27825, + "▁viable": 27826, + "▁Drug": 27827, + "TextBox": 27828, + "▁luminos": 27829, + "auté": 27830, + "yc": 27831, + "ště": 27832, + "▁affiliates": 27833, + "ilda": 27834, + "conduct": 27835, + "▁ebenfalls": 27836, + "▁AMD": 27837, + "▁Monitor": 27838, + "▁Companies": 27839, + "▁corrected": 27840, + "äck": 27841, + "SYSTEM": 27842, + "otherapy": 27843, + "▁перед": 27844, + "▁blues": 27845, + "atisf": 27846, + "although": 27847, + "rost": 27848, + "SCAN": 27849, + "▁RAM": 27850, + "ціональ": 27851, + "▁vendors": 27852, + "▁customs": 27853, + "▁activate": 27854, + "▁blogs": 27855, + "▁brace": 27856, + "▁strat": 27857, + "anje": 27858, + "щё": 27859, + "▁tide": 27860, + "▁Brigade": 27861, + "getOperand": 27862, + "▁aliment": 27863, + "▁achievements": 27864, + "▁suspicion": 27865, + "▁touchdown": 27866, + "broad": 27867, + "iore": 27868, + "Comparison": 27869, + "▁mum": 27870, + "English": 27871, + "▁Picture": 27872, + "▁Mouse": 27873, + "amd": 27874, + "▁[`": 27875, + "▁denomin": 27876, + "▁Aleks": 27877, + "▁prevents": 27878, + "ób": 27879, + "fed": 27880, + "▁Pray": 27881, + "▁shine": 27882, + "▁clutch": 27883, + "mux": 27884, + "Appro": 27885, + "▁notably": 27886, + "chio": 27887, + "nage": 27888, + "HAS": 27889, + "▁')": 27890, + "▁Miche": 27891, + "tg": 27892, + "::~": 27893, + "▁amely": 27894, + "▁rodz": 27895, + "zs": 27896, + "trait": 27897, + "▁klass": 27898, + "fö": 27899, + "▁destac": 27900, + "▁Clara": 27901, + "frequency": 27902, + "▁Git": 27903, + "▁поль": 27904, + "▁frequencies": 27905, + "▁febrero": 27906, + "▁stumbled": 27907, + "кою": 27908, + "▁Names": 27909, + "▁Flight": 27910, + "▁prey": 27911, + "▁medio": 27912, + "▁VAR": 27913, + "▁Float": 27914, + "▁Ernest": 27915, + "▁Marcatori": 27916, + "oport": 27917, + "▁cancellation": 27918, + "▁Bryan": 27919, + "————": 27920, + "Luc": 27921, + "▁libre": 27922, + "▁título": 27923, + "*>": 27924, + "▁Sandy": 27925, + "▁Marina": 27926, + "Been": 27927, + "▁wal": 27928, + "▁Kultur": 27929, + "▁explode": 27930, + "▁limiting": 27931, + "▁presumably": 27932, + "▁pb": 27933, + "▁Merc": 27934, + "▁реки": 27935, + "learning": 27936, + "Catalog": 27937, + "▁Census": 27938, + "lte": 27939, + "▁NET": 27940, + "raising": 27941, + "ське": 27942, + "staff": 27943, + "▁Quinn": 27944, + "▁memorial": 27945, + "пня": 27946, + "▁cuenta": 27947, + "▁XI": 27948, + "lbl": 27949, + "▁varies": 27950, + "▁fluctuations": 27951, + "▁долж": 27952, + "▁особи": 27953, + "▁warehouse": 27954, + "However": 27955, + "▁corrections": 27956, + "dhd": 27957, + "▁fals": 27958, + "▁controversy": 27959, + "▁curse": 27960, + "▁télé": 27961, + "řed": 27962, + "▁AU": 27963, + "▁тор": 27964, + "▁crít": 27965, + "idan": 27966, + "iliary": 27967, + "▁Panel": 27968, + "cule": 27969, + "▁Poor": 27970, + "▁BA": 27971, + "▁ignorant": 27972, + "èmes": 27973, + "▁aesthetic": 27974, + "Linked": 27975, + "getInt": 27976, + "Unicode": 27977, + "[@": 27978, + "▁Zent": 27979, + "Manifest": 27980, + "▁vars": 27981, + "PB": 27982, + "▁ву": 27983, + "▁Describe": 27984, + "▁Anything": 27985, + "oirs": 27986, + "▁socks": 27987, + "▁imped": 27988, + "▁neue": 27989, + "▁dispers": 27990, + "Collect": 27991, + "filer": 27992, + "▁Frau": 27993, + "▁Hockey": 27994, + "▁teens": 27995, + "▁Roberto": 27996, + "lauf": 27997, + "вать": 27998, + "▁ско": 27999, + "isArray": 28000, + "▁teenager": 28001, + "Built": 28002, + "▁loudly": 28003, + "Capacity": 28004, + "▁adventures": 28005, + "▁Molly": 28006, + "recogn": 28007, + "bars": 28008, + "▁Lor": 28009, + "▁può": 28010, + "▁mong": 28011, + "inement": 28012, + "Assignment": 28013, + "▁diz": 28014, + "lessness": 28015, + "▁Halloween": 28016, + "▁bitmap": 28017, + "Rom": 28018, + "нар": 28019, + "▁rebel": 28020, + "▁radial": 28021, + "measure": 28022, + "nit": 28023, + "▁Assume": 28024, + "▁assignments": 28025, + "▁Isn": 28026, + "▁altre": 28027, + "ßer": 28028, + "наль": 28029, + "▁flies": 28030, + "▁droit": 28031, + "▁thickness": 28032, + "▁enjo": 28033, + "▁dwell": 28034, + "▁homosexual": 28035, + "▁eval": 28036, + "$_{": 28037, + "asia": 28038, + "▁philos": 28039, + "getCurrent": 28040, + "▁veterans": 28041, + "▁Berkeley": 28042, + "▁wildlife": 28043, + "Cop": 28044, + "vern": 28045, + "▁Ú": 28046, + "tos": 28047, + "▁Led": 28048, + "▁keywords": 28049, + "▁medications": 28050, + "neum": 28051, + "▁jamais": 28052, + "▁Buc": 28053, + "▁PD": 28054, + "▁Statement": 28055, + "▁PI": 28056, + "▁Jackie": 28057, + "▁ordin": 28058, + "▁kör": 28059, + "enze": 28060, + "▁utilized": 28061, + "áct": 28062, + "azed": 28063, + "▁severely": 28064, + "▁även": 28065, + "▁libro": 28066, + "▁Eu": 28067, + "äst": 28068, + "PART": 28069, + "▁Butler": 28070, + "▁puzzle": 28071, + "Fall": 28072, + "Country": 28073, + "pfn": 28074, + "▁україн": 28075, + "▁Orchestra": 28076, + "▁alto": 28077, + "▁ancora": 28078, + "▁decomposition": 28079, + "▁م": 28080, + "▁appetite": 28081, + "adu": 28082, + "▁THAT": 28083, + "▁comenz": 28084, + "mina": 28085, + "▁initiated": 28086, + "▁Tat": 28087, + "▁sometime": 28088, + "rek": 28089, + "bread": 28090, + "▁Statistics": 28091, + "▁Cob": 28092, + "Follow": 28093, + "▁geometric": 28094, + "шла": 28095, + "▁proceedings": 28096, + "Dlg": 28097, + "seven": 28098, + "▁[-": 28099, + "▁Buffalo": 28100, + "▁blacks": 28101, + "▁sov": 28102, + "▁custody": 28103, + "▁ras": 28104, + "▁tattoo": 28105, + "öffentlicht": 28106, + "Blo": 28107, + "Austral": 28108, + "▁recuper": 28109, + "лев": 28110, + "▁bem": 28111, + "▁thou": 28112, + "oriented": 28113, + "vir": 28114, + "▁colony": 28115, + "▁Stanford": 28116, + "Absolute": 28117, + "adrat": 28118, + "▁Situ": 28119, + "▁souvent": 28120, + "EXEC": 28121, + "▁mű": 28122, + "▁apartments": 28123, + "▁случа": 28124, + "▁ano": 28125, + "WINDO": 28126, + "acci": 28127, + "▁Lau": 28128, + "court": 28129, + "▁manifold": 28130, + "▁coalition": 28131, + "▁XIV": 28132, + "Attrib": 28133, + "ascade": 28134, + "▁wheat": 28135, + "▁strengths": 28136, + "FREE": 28137, + "EMPTY": 28138, + "▁hey": 28139, + "ascular": 28140, + "▁plasma": 28141, + "▁bob": 28142, + "Separator": 28143, + "=\"${": 28144, + "▁Zag": 28145, + "▁projet": 28146, + "▁smoothly": 28147, + "SEQU": 28148, + "analy": 28149, + "attachment": 28150, + "▁ES": 28151, + "▁popped": 28152, + "ős": 28153, + "tom": 28154, + "▁són": 28155, + "▁rott": 28156, + "Utilities": 28157, + "hadoop": 28158, + "▁sotto": 28159, + "autor": 28160, + "▁Georges": 28161, + "▁který": 28162, + "▁gruppo": 28163, + "▁когда": 28164, + "▁меда": 28165, + "▁instrumental": 28166, + "▁Writer": 28167, + "▁setTimeout": 28168, + "ikk": 28169, + "▁Dopo": 28170, + "]);\r": 28171, + "▁practicing": 28172, + "▁Ronald": 28173, + "▁уби": 28174, + "▁agrees": 28175, + "▁denoted": 28176, + "ismiss": 28177, + "▁interviewed": 28178, + "templates": 28179, + "ři": 28180, + "administr": 28181, + "▁Butter": 28182, + "▁XVII": 28183, + "▁positioned": 28184, + "▁Fourth": 28185, + "▁overwhelmed": 28186, + "▁Regular": 28187, + "▁reprezent": 28188, + "кономи": 28189, + "▁expects": 28190, + "Indices": 28191, + "▁marijuana": 28192, + "▁zaj": 28193, + "▁Bren": 28194, + "▁begg": 28195, + "▁nahm": 28196, + "▁interrog": 28197, + "тие": 28198, + "▁Bun": 28199, + "▁серед": 28200, + "▁shelves": 28201, + "▁которых": 28202, + "▁Frauen": 28203, + "▁Sergeant": 28204, + "▁успе": 28205, + "matched": 28206, + "▁donne": 28207, + "▁touches": 28208, + "abort": 28209, + "▁vale": 28210, + "▁institutional": 28211, + "▁Mons": 28212, + "▁ambitious": 28213, + "▁nonetheless": 28214, + "jd": 28215, + "пей": 28216, + "▁backpack": 28217, + "dao": 28218, + "вия": 28219, + "▁surroundings": 28220, + "|_{": 28221, + "▁gegründ": 28222, + "disp": 28223, + "▁moisture": 28224, + "▁wyd": 28225, + "▁traders": 28226, + "▁Erst": 28227, + "▁Galaxy": 28228, + "▁воло": 28229, + "▁Peru": 28230, + "▁priorities": 28231, + "▁pronounced": 28232, + "▁CBS": 28233, + "▁Palm": 28234, + "▁expans": 28235, + "▁energet": 28236, + "▁Condition": 28237, + "▁Sver": 28238, + "nested": 28239, + "▁февраля": 28240, + "hero": 28241, + "▁коло": 28242, + "▁Films": 28243, + "Bon": 28244, + "éal": 28245, + "ployed": 28246, + "trained": 28247, + "▁első": 28248, + "▁lust": 28249, + "atinum": 28250, + "oyle": 28251, + "▁Jet": 28252, + "ждения": 28253, + "▁surveys": 28254, + "bee": 28255, + "workers": 28256, + "records": 28257, + "calendar": 28258, + "bbing": 28259, + "regation": 28260, + "dashboard": 28261, + "King": 28262, + "▁vista": 28263, + "▁depicted": 28264, + "▁occurring": 28265, + "▁офи": 28266, + "▁sandwich": 28267, + "rcu": 28268, + "kern": 28269, + "▁minut": 28270, + "▁смер": 28271, + "▁td": 28272, + "solete": 28273, + "Complex": 28274, + "▁tunn": 28275, + "▁scarc": 28276, + "stead": 28277, + "▁Fail": 28278, + "▁Rs": 28279, + "▁trails": 28280, + "kem": 28281, + "▁Romans": 28282, + "ativity": 28283, + "Previous": 28284, + "▁depress": 28285, + "▁resigned": 28286, + "getDefault": 28287, + "▁Tibet": 28288, + "▁Franco": 28289, + "\")));": 28290, + "▁injection": 28291, + "removed": 28292, + "▁praised": 28293, + "▁Asc": 28294, + "erase": 28295, + "▁commissioned": 28296, + "MAIL": 28297, + "▁Boh": 28298, + "Poly": 28299, + "▁cinq": 28300, + "▁Above": 28301, + "▁Joshua": 28302, + "ZERO": 28303, + "▁summit": 28304, + "▁Urs": 28305, + "▁curl": 28306, + "▁visa": 28307, + "▁resur": 28308, + "={'": 28309, + "feat": 28310, + "▁absorb": 28311, + "▁planets": 28312, + "▁princess": 28313, + "▁Jahrhunderts": 28314, + "xp": 28315, + "▁NBC": 28316, + "▁коми": 28317, + "▁FUN": 28318, + "▁neuen": 28319, + "▁déjà": 28320, + "▁Oz": 28321, + "bben": 28322, + "VIDEO": 28323, + "▁ejempl": 28324, + "▁considers": 28325, + "atri": 28326, + "▁arrog": 28327, + "ioso": 28328, + "▁hace": 28329, + "▁contacted": 28330, + "▁unple": 28331, + "▁sponsored": 28332, + "▁trainer": 28333, + "sbi": 28334, + "▁занима": 28335, + "Criterion": 28336, + "ното": 28337, + "scheme": 28338, + "ennial": 28339, + "perform": 28340, + "▁fixing": 28341, + "▁постро": 28342, + "arb": 28343, + "EXIT": 28344, + "▁café": 28345, + "ituted": 28346, + "riages": 28347, + "Tur": 28348, + "▁haber": 28349, + "elasticsearch": 28350, + "▁ал": 28351, + "rh": 28352, + "▁voll": 28353, + "CLU": 28354, + "Mil": 28355, + "▁membres": 28356, + "▁remarked": 28357, + "вана": 28358, + "=\"_": 28359, + "Less": 28360, + "(\"\");": 28361, + "▁Yale": 28362, + "berries": 28363, + "▁releasing": 28364, + "▁imports": 28365, + "idea": 28366, + "▁(+": 28367, + "▁arqu": 28368, + "ificación": 28369, + "▁пара": 28370, + "▁Rangers": 28371, + "Mic": 28372, + "▁nederbörd": 28373, + "▁imaginary": 28374, + "▁specialists": 28375, + "▁hoof": 28376, + "Modules": 28377, + "▁sadly": 28378, + "ceil": 28379, + "TabIndex": 28380, + "ationale": 28381, + "▁Partner": 28382, + "tbody": 28383, + "▁leverage": 28384, + "DN": 28385, + "▁Prec": 28386, + "▁Sé": 28387, + "▁Mam": 28388, + "▁afin": 28389, + "isValid": 28390, + "Pse": 28391, + "▁сторо": 28392, + "▁chopped": 28393, + "▁Minor": 28394, + "▁dabei": 28395, + "David": 28396, + "ussia": 28397, + "▁деревня": 28398, + "▁Identity": 28399, + "▁LGBT": 28400, + "ције": 28401, + "▁Orts": 28402, + "▁parti": 28403, + "▁Bachelor": 28404, + "uga": 28405, + "▁OPT": 28406, + "▁Seth": 28407, + "▁LIABLE": 28408, + "▁inaugur": 28409, + "▁Shanghai": 28410, + "▁relaxing": 28411, + "циона": 28412, + "\"%": 28413, + "▁obey": 28414, + "▁Airlines": 28415, + "Links": 28416, + "▁Celt": 28417, + "▁Admin": 28418, + "agation": 28419, + "▁worries": 28420, + "INTE": 28421, + "arith": 28422, + "Fatalf": 28423, + "]])": 28424, + "colm": 28425, + "▁archae": 28426, + "▁brushed": 28427, + "▁tät": 28428, + "▁structured": 28429, + "тии": 28430, + "▁homem": 28431, + "[:,": 28432, + "▁navy": 28433, + "getKey": 28434, + "powered": 28435, + "▁sucked": 28436, + "▁zomb": 28437, + "issant": 28438, + "▁Might": 28439, + "▁Pull": 28440, + "rir": 28441, + "▁пі": 28442, + "▁seas": 28443, + "▁Wrest": 28444, + "▁tense": 28445, + "▁atm": 28446, + "▁havet": 28447, + "▁pierws": 28448, + "▁tragic": 28449, + "▁Diff": 28450, + "▁confidential": 28451, + "successful": 28452, + "ęż": 28453, + "▁Chain": 28454, + "▁Kenya": 28455, + "Choice": 28456, + "ocur": 28457, + "aniu": 28458, + "▁consultant": 28459, + "▁Advis": 28460, + "Lif": 28461, + "▁Lors": 28462, + "avorite": 28463, + "▁utilizing": 28464, + "▁vintage": 28465, + "Matcher": 28466, + "▁membre": 28467, + "▁Expect": 28468, + "▁tracing": 28469, + "nog": 28470, + "▁dej": 28471, + "▁уче": 28472, + "▁loops": 28473, + "▁onclick": 28474, + "▁GPU": 28475, + "▁Albums": 28476, + "▁Archives": 28477, + "вата": 28478, + "▁stove": 28479, + "шли": 28480, + "ancies": 28481, + "▁gemeente": 28482, + "mob": 28483, + "PDF": 28484, + "eso": 28485, + "▁vég": 28486, + "Resolve": 28487, + "▁teaches": 28488, + "ложе": 28489, + "▁ство": 28490, + "▁Одна": 28491, + "▁fid": 28492, + "Something": 28493, + "▁nebo": 28494, + "▁Valentine": 28495, + "rowning": 28496, + "▁але": 28497, + "awi": 28498, + "ishi": 28499, + "▁SPI": 28500, + "▁spel": 28501, + "▁біль": 28502, + "▁participant": 28503, + "▁Ned": 28504, + "▁Gast": 28505, + "▁blond": 28506, + "▁saves": 28507, + "colored": 28508, + "▁ACTION": 28509, + "▁Politiker": 28510, + "}$)": 28511, + "▁Dum": 28512, + "dentry": 28513, + "Student": 28514, + "▁~=": 28515, + "loads": 28516, + "▁Foster": 28517, + "一个": 28518, + "▁PK": 28519, + "▁SB": 28520, + "▁Hern": 28521, + "▁Exhib": 28522, + "Listeners": 28523, + "Sun": 28524, + "plac": 28525, + "▁Bever": 28526, + "▁incluy": 28527, + "▁dc": 28528, + "argc": 28529, + "▁ged": 28530, + "спа": 28531, + "▁Formula": 28532, + "▁сем": 28533, + "▁empt": 28534, + "unregister": 28535, + "▁Queensland": 28536, + "ández": 28537, + "otive": 28538, + "▁alley": 28539, + "▁Democrat": 28540, + "▁travail": 28541, + "▁$,": 28542, + "RP": 28543, + "рое": 28544, + "personal": 28545, + "▁période": 28546, + "HOME": 28547, + "omes": 28548, + "▁recognised": 28549, + "heng": 28550, + "▁Jung": 28551, + "▁Roland": 28552, + "▁convicted": 28553, + "Locked": 28554, + "▁mari": 28555, + "▁Luxem": 28556, + "referto": 28557, + "Deleted": 28558, + "intent": 28559, + "▁Staats": 28560, + "▁області": 28561, + "ит": 28562, + "▁саве": 28563, + "▁Protocol": 28564, + "ając": 28565, + "chk": 28566, + "TypeInfo": 28567, + "▁pkt": 28568, + "▁scandal": 28569, + "▁individually": 28570, + "FMT": 28571, + "▁nj": 28572, + "abile": 28573, + "▁Rivers": 28574, + "PROPERTY": 28575, + "VB": 28576, + "wort": 28577, + "▁splitting": 28578, + "achten": 28579, + "▁ARISING": 28580, + "▁sip": 28581, + "▁fres": 28582, + "▁groom": 28583, + "Hol": 28584, + "▁canon": 28585, + "▁abruptly": 28586, + "▁afterward": 28587, + "▁Running": 28588, + "▁ji": 28589, + "▁%,": 28590, + "▁Palestinian": 28591, + "RW": 28592, + "pgfscope": 28593, + "▁countryside": 28594, + "▁fortunate": 28595, + "▁cél": 28596, + "▁Pointer": 28597, + "ensors": 28598, + "rating": 28599, + "▁buffers": 28600, + "▁remot": 28601, + "▁PropTypes": 28602, + "▁Nah": 28603, + "altern": 28604, + "▁easiest": 28605, + "▁invas": 28606, + "▁clk": 28607, + "copyright": 28608, + "▁blanc": 28609, + "SAMP": 28610, + "▁Cohen": 28611, + "▁Shell": 28612, + "▁destroying": 28613, + "▁Zel": 28614, + "dater": 28615, + "čen": 28616, + "▁filing": 28617, + "▁integrate": 28618, + "xit": 28619, + "▁RET": 28620, + "lene": 28621, + "calls": 28622, + "▁slaughter": 28623, + "initialized": 28624, + "unches": 28625, + "▁Trace": 28626, + "efficient": 28627, + "▁Woods": 28628, + "▁longitud": 28629, + "GN": 28630, + "▁Kont": 28631, + "▁chunks": 28632, + "ách": 28633, + "▁unemployment": 28634, + "acom": 28635, + "▁slowed": 28636, + "▁outlined": 28637, + "xffff": 28638, + "▁ikke": 28639, + "▁workspace": 28640, + "Mc": 28641, + "▁kicking": 28642, + "▁embedding": 28643, + "chnitt": 28644, + "erten": 28645, + "▁Interior": 28646, + "▁Songs": 28647, + "mmc": 28648, + "▁analyzed": 28649, + "▁Coupe": 28650, + "▁favorites": 28651, + "▁tt": 28652, + "▁той": 28653, + "Routing": 28654, + "▁Silva": 28655, + "▁anderem": 28656, + "▁honom": 28657, + "▁использова": 28658, + ".\"]": 28659, + "▁Wu": 28660, + "legt": 28661, + "▁spoon": 28662, + "▁jap": 28663, + "▁Extension": 28664, + "erne": 28665, + "▁vagy": 28666, + "▁села": 28667, + "▁функ": 28668, + "▁analytics": 28669, + "▁sug": 28670, + "▁Async": 28671, + "▁peaks": 28672, + "▁Gym": 28673, + "▁lawsuit": 28674, + "<>": 28675, + "ialis": 28676, + "etric": 28677, + "faced": 28678, + "▁disrupt": 28679, + "▁få": 28680, + "Inputs": 28681, + "`);": 28682, + "▁Mend": 28683, + "gon": 28684, + "▁\",\"": 28685, + "▁nerves": 28686, + "▁doubts": 28687, + "sap": 28688, + "▁sow": 28689, + ",\\,\\": 28690, + "▁BS": 28691, + "▁Glad": 28692, + "▁aster": 28693, + "œuvre": 28694, + "▁Bangl": 28695, + "▁iPad": 28696, + "useppe": 28697, + "▁conducting": 28698, + "▁({\\": 28699, + "▁Harbor": 28700, + "psz": 28701, + "▁FIFA": 28702, + "_**": 28703, + "emor": 28704, + "▁": 28705, + "e": 28706, + "t": 28707, + "a": 28708, + "o": 28709, + "i": 28710, + "n": 28711, + "r": 28712, + "s": 28713, + "l": 28714, + "d": 28715, + "h": 28716, + "c": 28717, + "u": 28718, + "m": 28719, + "p": 28720, + "g": 28721, + "f": 28722, + ".": 28723, + "y": 28724, + ",": 28725, + "b": 28726, + "w": 28727, + "v": 28728, + "k": 28729, + "_": 28730, + ")": 28731, + "(": 28732, + "-": 28733, + "0": 28734, + "S": 28735, + "*": 28736, + "I": 28737, + "T": 28738, + "\"": 28739, + "1": 28740, + "A": 28741, + "'": 28742, + "C": 28743, + "x": 28744, + ";": 28745, + "=": 28746, + ":": 28747, + "/": 28748, + "E": 28749, + "2": 28750, + "{": 28751, + "}": 28752, + "P": 28753, + "R": 28754, + "M": 28755, + "\\": 28756, + "D": 28757, + "L": 28758, + "N": 28759, + "B": 28760, + "о": 28761, + "O": 28762, + "а": 28763, + "z": 28764, + "F": 28765, + "|": 28766, + ">": 28767, + "j": 28768, + "H": 28769, + "3": 28770, + "#": 28771, + "и": 28772, + "е": 28773, + "9": 28774, + "q": 28775, + "$": 28776, + "G": 28777, + "н": 28778, + "U": 28779, + "W": 28780, + "4": 28781, + "5": 28782, + "8": 28783, + "6": 28784, + "р": 28785, + "т": 28786, + "7": 28787, + "с": 28788, + "<": 28789, + "V": 28790, + "в": 28791, + "[": 28792, + "]": 28793, + "л": 28794, + "к": 28795, + "K": 28796, + "é": 28797, + "J": 28798, + "д": 28799, + "&": 28800, + "\r": 28801, + "Y": 28802, + "м": 28803, + "?": 28804, + "у": 28805, + "+": 28806, + "п": 28807, + "!": 28808, + "’": 28809, + "г": 28810, + "я": 28811, + "з": 28812, + "і": 28813, + "X": 28814, + "^": 28815, + "–": 28816, + "б": 28817, + "@": 28818, + "й": 28819, + "á": 28820, + "—": 28821, + "ь": 28822, + "%": 28823, + "Q": 28824, + "ó": 28825, + "ч": 28826, + "í": 28827, + "Z": 28828, + "ы": 28829, + "ä": 28830, + "х": 28831, + "`": 28832, + "ц": 28833, + "ö": 28834, + "“": 28835, + "ж": 28836, + "ü": 28837, + "”": 28838, + "à": 28839, + "è": 28840, + "ш": 28841, + "ю": 28842, + "ł": 28843, + "С": 28844, + "~": 28845, + "ф": 28846, + "П": 28847, + "»": 28848, + "В": 28849, + "«": 28850, + "å": 28851, + "К": 28852, + "щ": 28853, + "·": 28854, + "ј": 28855, + "М": 28856, + "ç": 28857, + "А": 28858, + "Н": 28859, + "Р": 28860, + "Б": 28861, + "č": 28862, + "ú": 28863, + "ę": 28864, + "ã": 28865, + "ą": 28866, + "ă": 28867, + "Д": 28868, + "ї": 28869, + "ъ": 28870, + "ě": 28871, + "Г": 28872, + "š": 28873, + "О": 28874, + "Т": 28875, + "ê": 28876, + "ñ": 28877, + "…": 28878, + "ž": 28879, + "ß": 28880, + "ё": 28881, + "ż": 28882, + "ř": 28883, + "ś": 28884, + "Л": 28885, + "ő": 28886, + "„": 28887, + "э": 28888, + "ý": 28889, + "У": 28890, + "â": 28891, + "И": 28892, + "є": 28893, + "‘": 28894, + "î": 28895, + "З": 28896, + "Ф": 28897, + "ò": 28898, + "•": 28899, + "ć": 28900, + "É": 28901, + "°": 28902, + "ș": 28903, + "Х": 28904, + "ț": 28905, + "ô": 28906, + "Е": 28907, + "ń": 28908, + "Ч": 28909, + "Ш": 28910, + "ø": 28911, + "ù": 28912, + "ů": 28913, + "的": 28914, + "ا": 28915, + "æ": 28916, + "њ": 28917, + "љ": 28918, + "ë": 28919, + "ï": 28920, + "Э": 28921, + "£": 28922, + "−": 28923, + "，": 28924, + "õ": 28925, + "ћ": 28926, + "­": 28927, + "Ц": 28928, + "І": 28929, + "ā": 28930, + "ű": 28931, + "†": 28932, + "ل": 28933, + "ō": 28934, + "​": 28935, + "º": 28936, + "Я": 28937, + "′": 28938, + "Á": 28939, + "Ö": 28940, + "²": 28941, + "Ж": 28942, + "ì": 28943, + "。": 28944, + "数": 28945, + "×": 28946, + "ر": 28947, + "α": 28948, + "́": 28949, + "Ю": 28950, + "û": 28951, + "œ": 28952, + "ı": 28953, + "م": 28954, + "ن": 28955, + "ª": 28956, + "ź": 28957, + "ο": 28958, + "″": 28959, + "€": 28960, + "Ü": 28961, + "و": 28962, + "用": 28963, + "À": 28964, + "Č": 28965, + "Š": 28966, + "ت": 28967, + "د": 28968, + "一": 28969, + "¿": 28970, + "是": 28971, + "ي": 28972, + "ђ": 28973, + "®": 28974, + "ی": 28975, + "ν": 28976, + "đ": 28977, + "τ": 28978, + "─": 28979, + "ι": 28980, + "ε": 28981, + "→": 28982, + "ب": 28983, + "Å": 28984, + "ū": 28985, + "№": 28986, + "ş": 28987, + "不": 28988, + "џ": 28989, + "ー": 28990, + "中": 28991, + "Î": 28992, + "の": 28993, + "：": 28994, + "个": 28995, + "Й": 28996, + "ρ": 28997, + "有": 28998, + "Ä": 28999, + " ": 29000, + "ī": 29001, + "©": 29002, + "为": 29003, + "ه": 29004, + "י": 29005, + "ו": 29006, + "时": 29007, + "س": 29008, + "Ś": 29009, + "在": 29010, + "件": 29011, + "取": 29012, + "ς": 29013, + "™": 29014, + "이": 29015, + "σ": 29016, + "μ": 29017, + "定": 29018, + "文": 29019, + "据": 29020, + "置": 29021, + "Ž": 29022, + "±": 29023, + "表": 29024, + "成": 29025, + "ň": 29026, + "λ": 29027, + "¡": 29028, + "È": 29029, + "π": 29030, + "字": 29031, + "│": 29032, + "Ј": 29033, + "回": 29034, + "Є": 29035, + "到": 29036, + "行": 29037, + "§": 29038, + "½": 29039, + "ع": 29040, + "、": 29041, + "Ł": 29042, + "다": 29043, + "ン": 29044, + "κ": 29045, + "名": 29046, + "ה": 29047, + "入": 29048, + "η": 29049, + "大": 29050, + "对": 29051, + "可": 29052, + "Â": 29053, + "上": 29054, + "█": 29055, + "新": 29056, + "ف": 29057, + "加": 29058, + "要": 29059, + "Ż": 29060, + "下": 29061, + "分": 29062, + "值": 29063, + "ת": 29064, + "出": 29065, + "类": 29066, + "请": 29067, + "’": 29068, + "息": 29069, + "Ú": 29070, + "υ": 29071, + "获": 29072, + "示": 29073, + "以": 29074, + "ר": 29075, + "接": 29076, + "ל": 29077, + "を": 29078, + "存": 29079, + "信": 29080, + "设": 29081, + "方": 29082, + "ش": 29083, + "能": 29084, + "点": 29085, + "人": 29086, + "前": 29087, + "ğ": 29088, + "作": 29089, + "═": 29090, + "↘": 29091, + "ð": 29092, + "理": 29093, + "■": 29094, + "法": 29095, + "️": 29096, + "ˈ": 29097, + "果": 29098, + "发": 29099, + "ح": 29100, + "γ": 29101, + "ɵ": 29102, + "า": 29103, + "َ": 29104, + "了": 29105, + "户": 29106, + "Í": 29107, + "ə": 29108, + "ス": 29109, + "查": 29110, + "し": 29111, + "מ": 29112, + "单": 29113, + "ť": 29114, + "ق": 29115, + "る": 29116, + "间": 29117, + "如": 29118, + "本": 29119, + "后": 29120, + "ί": 29121, + "式": 29122, + "ト": 29123, + "Щ": 29124, + "Ó": 29125, + "す": 29126, + "א": 29127, + "生": 29128, + "动": 29129, + "ک": 29130, + "和": 29131, + "い": 29132, + "€": 29133, + "ა": 29134, + "가": 29135, + "하": 29136, + "�": 29137, + "小": 29138, + "返": 29139, + "否": 29140, + "ة": 29141, + "日": 29142, + "로": 29143, + "标": 29144, + "码": 29145, + "地": 29146, + "位": 29147, + "에": 29148, + " ": 29149, + "列": 29150, + "수": 29151, + "β": 29152, + "除": 29153, + "使": 29154, + "ש": 29155, + "ج": 29156, + "イ": 29157, + "δ": 29158, + "自": 29159, + "于": 29160, + "지": 29161, + "当": 29162, + "所": 29163, + "기": 29164, + "ი": 29165, + "ב": 29166, + "ร": 29167, + "★": 29168, + "子": 29169, + "号": 29170, + "ك": 29171, + "参": 29172, + "型": 29173, + "に": 29174, + "는": 29175, + "这": 29176, + "开": 29177, + "น": 29178, + "会": 29179, + "器": 29180, + "面": 29181, + "ル": 29182, + "图": 29183, + "度": 29184, + "）": 29185, + "（": 29186, + "의": 29187, + "内": 29188, + "을": 29189, + "最": 29190, + "”": 29191, + "化": 29192, + "建": 29193, + "니": 29194, + "量": 29195, + "😂": 29196, + "始": 29197, + "ē": 29198, + "خ": 29199, + "를": 29200, + "ά": 29201, + "过": 29202, + "³": 29203, + "´": 29204, + "组": 29205, + "功": 29206, + "‎": 29207, + "Ÿ": 29208, + "区": 29209, + "ز": 29210, + "ґ": 29211, + "ό": 29212, + "ッ": 29213, + "ω": 29214, + "Ç": 29215, + "选": 29216, + "通": 29217, + "结": 29218, + "录": 29219, + "改": 29220, + "ク": 29221, + "目": 29222, + "指": 29223, + "务": 29224, + "๐": 29225, + "输": 29226, + "た": 29227, + "อ": 29228, + "关": 29229, + "で": 29230, + "调": 29231, + "ा": 29232, + "정": 29233, + "合": 29234, + "已": 29235, + "시": 29236, + "部": 29237, + "页": 29238, + "━": 29239, + "ː": 29240, + "ま": 29241, + "我": 29242, + "求": 29243, + "市": 29244, + "次": 29245, + "נ": 29246, + "实": 29247, + "将": 29248, + "重": 29249, + "更": 29250, + "制": 29251, + "符": 29252, + "配": 29253, + "象": 29254, + "θ": 29255, + "ก": 29256, + "て": 29257, + "进": 29258, + "需": 29259, + "Đ": 29260, + "性": 29261, + "认": 29262, + "来": 29263, + "题": 29264, + "程": 29265, + "模": 29266, + "！": 29267, + "失": 29268, + "口": 29269, + "な": 29270, + "έ": 29271, + "": 29272, + "空": 29273, + "‍": 29274, + "期": 29275, + "者": 29276, + "は": 29277, + "Ђ": 29278, + "提": 29279, + "ή": 29280, + "ラ": 29281, + "한": 29282, + "态": 29283, + "复": 29284, + "ง": 29285, + "ე": 29286, + "Ø": 29287, + "리": 29288, + "修": 29289, + "‚": 29290, + "得": 29291, + "多": 29292, + "格": 29293, + "자": 29294, + "ע": 29295, + "่": 29296, + "函": 29297, + "应": 29298, + "↗": 29299, + "्": 29300, + "เ": 29301, + "正": 29302, + "注": 29303, + "스": 29304, + "서": 29305, + "リ": 29306, + "φ": 29307, + "ص": 29308, + "が": 29309, + "则": 29310, + "消": 29311, + "节": 29312, + "序": 29313, + "代": 29314, + "사": 29315, + "と": 29316, + "ד": 29317, + "้": 29318, + "र": 29319, + "此": 29320, + "保": 29321, + "ア": 29322, + "ư": 29323, + "인": 29324, + "ė": 29325, + "处": 29326, + "删": 29327, + "ɛ": 29328, + "容": 29329, + "ط": 29330, + "“": 29331, + "之": 29332, + "包": 29333, + "状": 29334, + "ド": 29335, + "İ": 29336, + "体": 29337, + "同": 29338, + "事": 29339, + "🙂": 29340, + "タ": 29341, + "χ": 29342, + "ʿ": 29343, + "Ș": 29344, + "主": 29345, + "品": 29346, + "ק": 29347, + "询": 29348, + "创": 29349, + "该": 29350, + "　": 29351, + "元": 29352, + "第": 29353, + "天": 29354, + "或": 29355, + "年": 29356, + "转": 29357, + "ח": 29358, + "传": 29359, + "ţ": 29360, + "路": 29361, + "例": 29362, + "机": 29363, + "Ã": 29364, + "ď": 29365, + "高": 29366, + "相": 29367, + "โ": 29368, + "片": 29369, + "―": 29370, + "操": 29371, + "ա": 29372, + "ม": 29373, + "全": 29374, + "无": 29375, + "月": 29376, + "称": 29377, + "ั": 29378, + "就": 29379, + "™": 29380, + "明": 29381, + "计": 29382, + "你": 29383, + "败": 29384, + "密": 29385, + "解": 29386, + "れ": 29387, + "أ": 29388, + "变": 29389, + "段": 29390, + "条": 29391, + "默": 29392, + "●": 29393, + "ล": 29394, + "色": 29395, + "断": 29396, + "商": 29397, + "ם": 29398, + "か": 29399, + "里": 29400, + "系": 29401, + "编": 29402, + "错": 29403, + "트": 29404, + "只": 29405, + "县": 29406, + "ს": 29407, + "常": 29408, + "初": 29409, + "ɔ": 29410, + "Α": 29411, + "フ": 29412, + "►": 29413, + "等": 29414, + "일": 29415, + "・": 29416, + "Ō": 29417, + "情": 29418, + "现": 29419, + "Ř": 29420, + "ِ": 29421, + "さ": 29422, + "ạ": 29423, + "용": 29424, + "证": 29425, + "해": 29426, + "手": 29427, + "支": 29428, + "입": 29429, + "服": 29430, + "்": 29431, + "道": 29432, + "어": 29433, + "送": 29434, + "载": 29435, + "限": 29436, + "线": 29437, + "属": 29438, + "—": 29439, + "他": 29440, + "放": 29441, + "记": 29442, + "公": 29443, + "没": 29444, + "添": 29445, + "显": 29446, + "บ": 29447, + "ย": 29448, + "რ": 29449, + "其": 29450, + "集": 29451, + "金": 29452, + "国": 29453, + "任": 29454, + "ە": 29455, + "话": 29456, + "并": 29457, + "被": 29458, + "ύ": 29459, + "都": 29460, + "گ": 29461, + "意": 29462, + "כ": 29463, + "经": 29464, + "성": 29465, + "看": 29466, + "פ": 29467, + "址": 29468, + "ס": 29469, + "드": 29470, + "交": 29471, + "¼": 29472, + "Џ": 29473, + "完": 29474, + "Δ": 29475, + "义": 29476, + "보": 29477, + "向": 29478, + "换": 29479, + "山": 29480, + "算": 29481, + "二": 29482, + "پ": 29483, + "⁄": 29484, + "判": 29485, + "级": 29486, + "工": 29487, + "ด": 29488, + "⠀": 29489, + "家": 29490, + "レ": 29491, + "三": 29492, + "原": 29493, + "】": 29494, + "长": 29495, + "া": 29496, + "管": 29497, + "ѝ": 29498, + "क": 29499, + "学": 29500, + "ロ": 29501, + "验": 29502, + "写": 29503, + "Œ": 29504, + "从": 29505, + "【": 29506, + "收": 29507, + "ả": 29508, + "未": 29509, + "登": 29510, + "고": 29511, + "源": 29512, + "每": 29513, + "µ": 29514, + "误": 29515, + "り": 29516, + "요": 29517, + "按": 29518, + "ว": 29519, + "权": 29520, + "根": 29521, + "プ": 29522, + "串": 29523, + "ส": 29524, + "›": 29525, + "제": 29526, + "シ": 29527, + "Ş": 29528, + "确": 29529, + "好": 29530, + "统": 29531, + "效": 29532, + "网": 29533, + "\u0001": 29534, + "物": 29535, + "아": 29536, + "也": 29537, + "은": 29538, + "ệ": 29539, + "न": 29540, + "项": 29541, + "资": 29542, + "こ": 29543, + "引": 29544, + "ジ": 29545, + "ค": 29546, + "版": 29547, + "ท": 29548, + "平": 29549, + "们": 29550, + "与": 29551, + "き": 29552, + "移": 29553, + "ि": 29554, + "素": 29555, + "执": 29556, + "주": 29557, + "‐": 29558, + "Ґ": 29559, + "ี": 29560, + "板": 29561, + "问": 29562, + "Ε": 29563, + "安": 29564, + "면": 29565, + "소": 29566, + "ต": 29567, + "ิ": 29568, + "持": 29569, + "습": 29570, + "Σ": 29571, + "ら": 29572, + "コ": 29573, + "心": 29574, + "Π": 29575, + "打": 29576, + "」": 29577, + "상": 29578, + "「": 29579, + "检": 29580, + "库": 29581, + "÷": 29582, + "으": 29583, + "测": 29584, + "ん": 29585, + "े": 29586, + "ُ": 29587, + "力": 29588, + "直": 29589, + "由": 29590, + "ى": 29591, + "试": 29592, + "必": 29593, + "端": 29594, + "ʻ": 29595, + "先": 29596, + "↑": 29597, + "命": 29598, + "도": 29599, + "전": 29600, + "ห": 29601, + "员": 29602, + "ɪ": 29603, + "있": 29604, + "比": 29605, + "ṣ": 29606, + "時": 29607, + "择": 29608, + "ذ": 29609, + "テ": 29610, + "‌": 29611, + "构": 29612, + "备": 29613, + "그": 29614, + "链": 29615, + "说": 29616, + "ლ": 29617, + "ן": 29618, + "签": 29619, + "う": 29620, + "غ": 29621, + "ế": 29622, + "ض": 29623, + "ḥ": 29624, + "启": 29625, + "력": 29626, + "ო": 29627, + "付": 29628, + "მ": 29629, + "索": 29630, + "特": 29631, + "ג": 29632, + "西": 29633, + "대": 29634, + "├": 29635, + "–": 29636, + "Ž": 29637, + "外": 29638, + "צ": 29639, + "头": 29640, + "连": 29641, + "流": 29642, + "◄": 29643, + "デ": 29644, + "カ": 29645, + "র": 29646, + "오": 29647, + "找": 29648, + "清": 29649, + "🤣": 29650, + "去": 29651, + "₹": 29652, + "경": 29653, + "グ": 29654, + "ْ": 29655, + "¢": 29656, + "因": 29657, + "": 29658, + "Κ": 29659, + "增": 29660, + "知": 29661, + "¶": 29662, + "像": 29663, + "♥": 29664, + "터": 29665, + "く": 29666, + "ậ": 29667, + "メ": 29668, + "Æ": 29669, + "省": 29670, + "स": 29671, + "म": 29672, + "❤": 29673, + "あ": 29674, + "样": 29675, + "起": 29676, + "台": 29677, + "读": 29678, + "角": 29679, + "南": 29680, + "整": 29681, + "订": 29682, + "\f": 29683, + "ט": 29684, + "マ": 29685, + "্": 29686, + "우": 29687, + "ն": 29688, + "您": 29689, + "ئ": 29690, + "基": 29691, + "水": 29692, + "생": 29693, + "‑": 29694, + "나": 29695, + "画": 29696, + "描": 29697, + "击": 29698, + "っ": 29699, + "라": 29700, + "ნ": 29701, + "ր": 29702, + "业": 29703, + "ბ": 29704, + "别": 29705, + "♦": 29706, + "ィ": 29707, + "त": 29708, + "给": 29709, + "문": 29710, + "形": 29711, + "控": 29712, + "然": 29713, + "동": 29714, + "Њ": 29715, + "⁠": 29716, + "东": 29717, + "ป": 29718, + "州": 29719, + "排": 29720, + "세": 29721, + "装": 29722, + "할": 29723, + "Ć": 29724, + "∞": 29725, + "海": 29726, + "城": 29727, + "键": 29728, + "径": 29729, + "호": 29730, + "화": 29731, + "្": 29732, + "料": 29733, + "ơ": 29734, + "ी": 29735, + "ウ": 29736, + "具": 29737, + "ブ": 29738, + "块": 29739, + "再": 29740, + "ố": 29741, + "电": 29742, + "；": 29743, + "위": 29744, + "两": 29745, + "而": 29746, + "장": 29747, + "آ": 29748, + "Ț": 29749, + "バ": 29750, + "还": 29751, + "令": 29752, + "キ": 29753, + "ّ": 29754, + "값": 29755, + "번": 29756, + "만": 29757, + "总": 29758, + "ल": 29759, + "▲": 29760, + "异": 29761, + "光": 29762, + "客": 29763, + "非": 29764, + "ị": 29765, + "": 29766, + "þ": 29767, + "設": 29768, + "述": 29769, + "합": 29770, + "？": 29771, + "✔": 29772, + "导": 29773, + "ṇ": 29774, + "부": 29775, + "˙": 29776, + "Τ": 29777, + "も": 29778, + "구": 29779, + "镇": 29780, + "작": 29781, + "░": 29782, + "步": 29783, + "ộ": 29784, + "活": 29785, + "พ": 29786, + "←": 29787, + "ǎ": 29788, + "จ": 29789, + "束": 29790, + "ـ": 29791, + "‘": 29792, + "那": 29793, + "प": 29794, + "エ": 29795, + "志": 29796, + "么": 29797, + "运": 29798, + "北": 29799, + "超": 29800, + "་": 29801, + "布": 29802, + "ώ": 29803, + "͡": 29804, + "少": 29805, + "파": 29806, + "ʃ": 29807, + "ム": 29808, + "•": 29809, + "卡": 29810, + "ন": 29811, + "Μ": 29812, + "ɑ": 29813, + "😉": 29814, + "辑": 29815, + "원": 29816, + "美": 29817, + "产": 29818, + "利": 29819, + "모": 29820, + "联": 29821, + "界": 29822, + "체": 29823, + "种": 29824, + "王": 29825, + "ľ": 29826, + "여": 29827, + "메": 29828, + "域": 29829, + "ვ": 29830, + "立": 29831, + "록": 29832, + "게": 29833, + "إ": 29834, + "ṭ": 29835, + "神": 29836, + "ո": 29837, + "音": 29838, + "☆": 29839, + "Ñ": 29840, + "조": 29841, + "動": 29842, + "缓": 29843, + "과": 29844, + "报": 29845, + "ʼ": 29846, + "ា": 29847, + "되": 29848, + "ե": 29849, + "视": 29850, + "ช": 29851, + "详": 29852, + "แ": 29853, + "¦": 29854, + "把": 29855, + "க": 29856, + "ি": 29857, + "출": 29858, + "비": 29859, + "边": 29860, + "框": 29861, + "व": 29862, + "サ": 29863, + "Ι": 29864, + "Ο": 29865, + "オ": 29866, + "¾": 29867, + "历": 29868, + "ŏ": 29869, + "门": 29870, + "ข": 29871, + "含": 29872, + "¬": 29873, + "周": 29874, + "填": 29875, + "待": 29876, + "ะ": 29877, + "დ": 29878, + "Ї": 29879, + "额": 29880, + "음": 29881, + "四": 29882, + "だ": 29883, + "회": 29884, + "止": 29885, + "率": 29886, + "环": 29887, + "パ": 29888, + "래": 29889, + "闭": 29890, + "̀": 29891, + "语": 29892, + "개": 29893, + "身": 29894, + "藏": 29895, + "य": 29896, + "된": 29897, + "即": 29898, + "拉": 29899, + "선": 29900, + "변": 29901, + "≥": 29902, + "ุ": 29903, + "些": 29904, + "🤷": 29905, + "せ": 29906, + "左": 29907, + "ợ": 29908, + "右": 29909, + "ể": 29910, + "내": 29911, + "ּ": 29912, + "ז": 29913, + "ে": 29914, + "告": 29915, + "ấ": 29916, + "白": 29917, + "账": 29918, + "费": 29919, + "江": 29920, + "み": 29921, + "‹": 29922, + "์": 29923, + "‡": 29924, + "造": 29925, + "但": 29926, + "十": 29927, + "它": 29928, + "ं": 29929, + "ŋ": 29930, + "ў": 29931, + "セ": 29932, + "女": 29933, + "⣿": 29934, + "ի": 29935, + "京": 29936, + "触": 29937, + "함": 29938, + "들": 29939, + "Ā": 29940, + "˜": 29941, + "石": 29942, + "よ": 29943, + "田": 29944, + "易": 29945, + "规": 29946, + "展": 29947, + "¯": 29948, + "做": 29949, + "星": 29950, + "უ": 29951, + "✓": 29952, + "თ": 29953, + "供": 29954, + "명": 29955, + "ξ": 29956, + "己": 29957, + "且": 29958, + "插": 29959, + "景": 29960, + "切": 29961, + "ไ": 29962, + "없": 29963, + "ョ": 29964, + "及": 29965, + "Ν": 29966, + "미": 29967, + "ث": 29968, + "데": 29969, + "价": 29970, + "乡": 29971, + "ह": 29972, + "チ": 29973, + "真": 29974, + "太": 29975, + "ู": 29976, + "ダ": 29977, + "局": 29978, + "♂": 29979, + "退": 29980, + "ு": 29981, + "ক": 29982, + "ி": 29983, + "何": 29984, + "😭": 29985, + "¥": 29986, + "": 29987, + "≈": 29988, + "司": 29989, + "层": 29990, + "실": 29991, + "站": 29992, + "首": 29993, + "款": 29994, + "រ": 29995, + "間": 29996, + "ָ": 29997, + "저": 29998, + "监": 29999, + "ァ": 30000, + "册": 30001, + "案": 30002, + "ो": 30003, + "反": 30004, + "听": 30005, + "族": 30006, + "析": 30007, + "ื": 30008, + "秒": 30009, + "공": 30010, + "œ": 30011, + "🚀": 30012, + "거": 30013, + "재": 30014, + "‚": 30015, + "場": 30016, + "广": 30017, + "播": 30018, + "║": 30019, + "⋅": 30020, + "技": 30021, + "贴": 30022, + "想": 30023, + "ʁ": 30024, + "ớ": 30025, + "ャ": 30026, + "중": 30027, + "》": 30028, + "速": 30029, + "频": 30030, + "队": 30031, + "ำ": 30032, + "け": 30033, + "ु": 30034, + "≤": 30035, + "↓": 30036, + "须": 30037, + "菜": 30038, + "̃": 30039, + "剪": 30040, + "버": 30041, + "ェ": 30042, + "Λ": 30043, + "细": 30044, + "選": 30045, + "द": 30046, + "¹": 30047, + "许": 30048, + "ầ": 30049, + "世": 30050, + "ュ": 30051, + "ء": 30052, + "‡": 30053, + "候": 30054, + "共": 30055, + "크": 30056, + "ธ": 30057, + "설": 30058, + "快": 30059, + "友": 30060, + "ְ": 30061, + "车": 30062, + "推": 30063, + "花": 30064, + "言": 30065, + "چ": 30066, + "至": 30067, + "開": 30068, + "校": 30069, + "個": 30070, + "村": 30071, + "つ": 30072, + "▌": 30073, + "ப": 30074, + "결": 30075, + "ņ": 30076, + "优": 30077, + "ន": 30078, + "达": 30079, + "核": 30080, + "ナ": 30081, + "场": 30082, + "影": 30083, + "🏻": 30084, + "钮": 30085, + "ظ": 30086, + "Þ": 30087, + "▼": 30088, + "お": 30089, + "份": 30090, + "微": 30091, + "ờ": 30092, + "识": 30093, + "행": 30094, + "《": 30095, + "ใ": 30096, + "ọ": 30097, + "预": 30098, + "ব": 30099, + "த": 30100, + "": 30101, + "ų": 30102, + "마": 30103, + "않": 30104, + "ɡ": 30105, + "계": 30106, + "연": 30107, + "五": 30108, + "Ź": 30109, + "め": 30110, + "很": 30111, + "간": 30112, + "無": 30113, + "ប": 30114, + "社": 30115, + "Ê": 30116, + "书": 30117, + "顶": 30118, + "ტ": 30119, + "才": 30120, + "云": 30121, + "└": 30122, + "ζ": 30123, + "،": 30124, + "搜": 30125, + "신": 30126, + "유": 30127, + "‏": 30128, + "✅": 30129, + "⭐": 30130, + "照": 30131, + "短": 30132, + "川": 30133, + "後": 30134, + "范": 30135, + "民": 30136, + "治": 30137, + "章": 30138, + "ề": 30139, + "바": 30140, + "ә": 30141, + "⚭": 30142, + "河": 30143, + "论": 30144, + "え": 30145, + "Ω": 30146, + "√": 30147, + "Ă": 30148, + "Γ": 30149, + "坐": 30150, + "적": 30151, + "停": 30152, + "추": 30153, + "受": 30154, + "♀": 30155, + "ʾ": 30156, + "树": 30157, + "林": 30158, + "치": 30159, + "fi": 30160, + "▒": 30161, + "张": 30162, + "着": 30163, + "访": 30164, + "考": 30165, + "教": 30166, + "ग": 30167, + "准": 30168, + "印": 30169, + "精": 30170, + "窗": 30171, + "宝": 30172, + "ち": 30173, + "围": 30174, + "ַ": 30175, + "致": 30176, + "モ": 30177, + "때": 30178, + "随": 30179, + "储": 30180, + "况": 30181, + "邮": 30182, + "武": 30183, + "⛔": 30184, + "维": 30185, + "ү": 30186, + "跳": 30187, + "ब": 30188, + "投": 30189, + "ủ": 30190, + "표": 30191, + "반": 30192, + "英": 30193, + "ʰ": 30194, + "👍": 30195, + "ज": 30196, + "带": 30197, + "為": 30198, + "续": 30199, + "ɨ": 30200, + "처": 30201, + "₂": 30202, + "클": 30203, + "群": 30204, + "현": 30205, + "风": 30206, + "购": 30207, + "ក": 30208, + "老": 30209, + "留": 30210, + "球": 30211, + "프": 30212, + "▄": 30213, + "史": 30214, + "Љ": 30215, + "⟩": 30216, + "분": 30217, + "გ": 30218, + "店": 30219, + "审": 30220, + "료": 30221, + "목": 30222, + "略": 30223, + "관": 30224, + "ִ": 30225, + "科": 30226, + "货": 30227, + "ம": 30228, + "络": 30229, + "阳": 30230, + "Ḥ": 30231, + "資": 30232, + "若": 30233, + "স": 30234, + "ہ": 30235, + "宽": 30236, + "见": 30237, + "ズ": 30238, + "游": 30239, + "방": 30240, + "ồ": 30241, + "ɾ": 30242, + "열": 30243, + "러": 30244, + "ך": 30245, + "\u001b": 30246, + "်": 30247, + "余": 30248, + "响": 30249, + "缩": 30250, + "ட": 30251, + "评": 30252, + "允": 30253, + "离": 30254, + "🤔": 30255, + "Ё": 30256, + "ʊ": 30257, + "黑": 30258, + "马": 30259, + "⟨": 30260, + "値": 30261, + "箱": 30262, + "야": 30263, + "ម": 30264, + "Ő": 30265, + "感": 30266, + "ツ": 30267, + "ụ": 30268, + "ポ": 30269, + "확": 30270, + "声": 30271, + "战": 30272, + "ѕ": 30273, + "変": 30274, + "와": 30275, + "父": 30276, + "ベ": 30277, + "助": 30278, + "업": 30279, + "ʲ": 30280, + "ÿ": 30281, + "充": 30282, + "强": 30283, + "博": 30284, + "ミ": 30285, + "销": 30286, + "당": 30287, + "記": 30288, + "什": 30289, + "匹": 30290, + "ւ": 30291, + "そ": 30292, + "코": 30293, + "ল": 30294, + "ŭ": 30295, + "午": 30296, + "ニ": 30297, + "\u0012": 30298, + "ʒ": 30299, + "შ": 30300, + "某": 30301, + "ォ": 30302, + "足": 30303, + "타": 30304, + "Ð": 30305, + "ხ": 30306, + "름": 30307, + "木": 30308, + "楼": 30309, + "최": 30310, + "红": 30311, + "¨": 30312, + "古": 30313, + "\u0006": 30314, + "단": 30315, + "今": 30316, + "ʔ": 30317, + "ट": 30318, + "ম": 30319, + "斯": 30320, + "語": 30321, + "Ÿ": 30322, + "🙄": 30323, + "牌": 30324, + "안": 30325, + "ស": 30326, + "颜": 30327, + "～": 30328, + "克": 30329, + "深": 30330, + "금": 30331, + "會": 30332, + "尔": 30333, + "释": 30334, + "批": 30335, + "산": 30336, + "野": 30337, + "防": 30338, + "Η": 30339, + "ө": 30340, + "ψ": 30341, + "ボ": 30342, + "š": 30343, + "各": 30344, + "진": 30345, + "追": 30346, + "句": 30347, + "警": 30348, + "Φ": 30349, + "ѣ": 30350, + "ḍ": 30351, + "词": 30352, + "男": 30353, + "글": 30354, + "식": 30355, + "隐": 30356, + "복": 30357, + "盘": 30358, + "Ì": 30359, + "申": 30360, + "议": 30361, + "ザ": 30362, + "近": 30363, + "능": 30364, + "য": 30365, + "東": 30366, + "這": 30367, + "ர": 30368, + "距": 30369, + "院": 30370, + "德": 30371, + "ǐ": 30372, + "针": 30373, + "▀": 30374, + "↔": 30375, + "房": 30376, + "青": 30377, + "政": 30378, + "😅": 30379, + "递": 30380, + "প": 30381, + "波": 30382, + "ソ": 30383, + "绑": 30384, + "ビ": 30385, + "ễ": 30386, + "포": 30387, + "\u0010": 30388, + "ử": 30389, + "등": 30390, + "환": 30391, + "士": 30392, + "ত": 30393, + "Θ": 30394, + "초": 30395, + "境": 30396, + "差": 30397, + "采": 30398, + "디": 30399, + "ĩ": 30400, + "升": 30401, + "背": 30402, + "배": 30403, + "龙": 30404, + "街": 30405, + "್": 30406, + "ṛ": 30407, + "ু": 30408, + "弹": 30409, + "魔": 30410, + "객": 30411, + "‰": 30412, + "⌁": 30413, + "ἐ": 30414, + "禁": 30415, + "ผ": 30416, + "қ": 30417, + "島": 30418, + "ா": 30419, + "♭": 30420, + "百": 30421, + "ứ": 30422, + "ネ": 30423, + "专": 30424, + "來": 30425, + "刷": 30426, + "필": 30427, + "յ": 30428, + "ắ": 30429, + "华": 30430, + "Β": 30431, + "श": 30432, + "¸": 30433, + "屏": 30434, + "死": 30435, + "遍": 30436, + "검": 30437, + "Χ": 30438, + "것": 30439, + "八": 30440, + "览": 30441, + "택": 30442, + "唯": 30443, + "∙": 30444, + "¤": 30445, + "페": 30446, + "让": 30447, + "锁": 30448, + "무": 30449, + "思": 30450, + "隔": 30451, + "Ô": 30452, + "\u0013": 30453, + "ṃ": 30454, + "ワ": 30455, + "低": 30456, + "션": 30457, + "半": 30458, + "较": 30459, + "ត": 30460, + "享": 30461, + "积": 30462, + "ˆ": 30463, + "😊": 30464, + "典": 30465, + "ǔ": 30466, + "六": 30467, + "便": 30468, + "ɐ": 30469, + "简": 30470, + "继": 30471, + "仅": 30472, + "尾": 30473, + "‹": 30474, + "வ": 30475, + "կ": 30476, + "ƒ": 30477, + "영": 30478, + "火": 30479, + "湖": 30480, + "書": 30481, + "발": 30482, + "ハ": 30483, + "循": 30484, + "术": 30485, + "結": 30486, + "ļ": 30487, + "乐": 30488, + "滤": 30489, + "종": 30490, + "ถ": 30491, + "ὶ": 30492, + "满": 30493, + "╝": 30494, + "わ": 30495, + "ど": 30496, + "็": 30497, + "형": 30498, + "國": 30499, + "ự": 30500, + "線": 30501, + "블": 30502, + "封": 30503, + "確": 30504, + "依": 30505, + "ս": 30506, + "永": 30507, + "색": 30508, + "歌": 30509, + "數": 30510, + "福": 30511, + "삭": 30512, + "実": 30513, + "레": 30514, + "ſ": 30515, + "千": 30516, + "\u000e": 30517, + "母": 30518, + "더": 30519, + "임": 30520, + "տ": 30521, + "ے": 30522, + "几": 30523, + "双": 30524, + "노": 30525, + "ณ": 30526, + "掉": 30527, + "Ρ": 30528, + "ἀ": 30529, + "標": 30530, + "長": 30531, + "档": 30532, + "태": 30533, + "ペ": 30534, + "본": 30535, + "Œ": 30536, + "底": 30537, + "终": 30538, + "請": 30539, + "კ": 30540, + "̯": 30541, + "예": 30542, + "▬": 30543, + "報": 30544, + "ピ": 30545, + "๏": 30546, + "暂": 30547, + "李": 30548, + "Υ": 30549, + "\u0005": 30550, + "\u0002": 30551, + "替": 30552, + "운": 30553, + "射": 30554, + "\u0018": 30555, + "매": 30556, + "\u0011": 30557, + "🏼": 30558, + "票": 30559, + "附": 30560, + "ノ": 30561, + "ũ": 30562, + "压": 30563, + "阿": 30564, + "Ò": 30565, + "테": 30566, + "∼": 30567, + "万": 30568, + "մ": 30569, + "후": 30570, + "普": 30571, + "截": 30572, + "속": 30573, + "括": 30574, + "😀": 30575, + "ை": 30576, + "▶": 30577, + "까": 30578, + "ট": 30579, + "曲": 30580, + "师": 30581, + "钱": 30582, + "栏": 30583, + "Ы": 30584, + "走": 30585, + "ữ": 30586, + "‬": 30587, + "归": 30588, + "점": 30589, + "🔥": 30590, + "었": 30591, + "連": 30592, + "私": 30593, + "청": 30594, + "刘": 30595, + "免": 30596, + "": 30597, + "奖": 30598, + "見": 30599, + "ֹ": 30600, + "☺": 30601, + "ケ": 30602, + "역": 30603, + "际": 30604, + "받": 30605, + "望": 30606, + "帝": 30607, + "减": 30608, + "두": 30609, + "领": 30610, + "„": 30611, + "钟": 30612, + "ガ": 30613, + "架": 30614, + "든": 30615, + "ல": 30616, + "松": 30617, + "□": 30618, + "越": 30619, + "答": 30620, + "ɕ": 30621, + "ῦ": 30622, + "染": 30623, + "": 30624, + "质": 30625, + "顺": 30626, + "气": 30627, + "╗": 30628, + "計": 30629, + "ქ": 30630, + "亮": 30631, + "🤦": 30632, + "̂": 30633, + "ٹ": 30634, + "座": 30635, + "ˌ": 30636, + "均": 30637, + "\u000b": 30638, + "官": 30639, + "适": 30640, + "护": 30641, + "久": 30642, + "春": 30643, + "曹": 30644, + "皇": 30645, + "脚": 30646, + "池": 30647, + "延": 30648, + "키": 30649, + "품": 30650, + "現": 30651, + "檔": 30652, + "ば": 30653, + "ⴰ": 30654, + "希": 30655, + "玩": 30656, + "固": 30657, + "黄": 30658, + "": 30659, + "☽": 30660, + "银": 30661, + "\u0003": 30662, + "┃": 30663, + "👏": 30664, + "불": 30665, + "攻": 30666, + "へ": 30667, + "决": 30668, + "⊙": 30669, + "宁": 30670, + "च": 30671, + "機": 30672, + "義": 30673, + "ɲ": 30674, + "\u0015": 30675, + "했": 30676, + "ẩ": 30677, + "愛": 30678, + "矩": 30679, + "패": 30680, + "ặ": 30681, + "郎": 30682, + "Ь": 30683, + "绘": 30684, + "负": 30685, + "ổ": 30686, + "ய": 30687, + "汉": 30688, + "編": 30689, + "ێ": 30690, + "്": 30691, + "じ": 30692, + "카": 30693, + "似": 30694, + "ں": 30695, + "や": 30696, + "認": 30697, + "\u000f": 30698, + "過": 30699, + "통": 30700, + "▪": 30701, + "约": 30702, + "香": 30703, + "买": 30704, + "住": 30705, + "╚": 30706, + "😁": 30707, + "扩": 30708, + "静": 30709, + "려": 30710, + "학": 30711, + "钥": 30712, + "증": 30713, + "ỉ": 30714, + "她": 30715, + "食": 30716, + "往": 30717, + "點": 30718, + "偏": 30719, + "康": 30720, + "\u0014": 30721, + "į": 30722, + "준": 30723, + "\u0004": 30724, + "ฟ": 30725, + "♣": 30726, + "戏": 30727, + "ʂ": 30728, + "井": 30729, + "军": 30730, + "爱": 30731, + "ٱ": 30732, + "七": 30733, + "차": 30734, + "币": 30735, + "♠": 30736, + "哈": 30737, + "阅": 30738, + "介": 30739, + "观": 30740, + "區": 30741, + "˜": 30742, + "ً": 30743, + "又": 30744, + "冲": 30745, + "朝": 30746, + "姓": 30747, + "课": 30748, + "龍": 30749, + "각": 30750, + "∈": 30751, + "米": 30752, + "ƒ": 30753, + "喜": 30754, + "夜": 30755, + "团": 30756, + "⇒": 30757, + "远": 30758, + "\u001a": 30759, + "ὐ": 30760, + "承": 30761, + "ಿ": 30762, + "室": 30763, + "ʀ": 30764, + "ង": 30765, + "अ": 30766, + "罗": 30767, + "🙏": 30768, + "软": 30769, + "🟡": 30770, + "건": 30771, + "؟": 30772, + "း": 30773, + "ᴇ": 30774, + "ユ": 30775, + "토": 30776, + "策": 30777, + "̄": 30778, + "국": 30779, + "ֶ": 30780, + "协": 30781, + "营": 30782, + "関": 30783, + "吉": 30784, + "💀": 30785, + "奇": 30786, + "滚": 30787, + "轴": 30788, + "処": 30789, + "土": 30790, + "划": 30791, + "ड": 30792, + "临": 30793, + "ֵ": 30794, + "航": 30795, + "浏": 30796, + "ゴ": 30797, + "別": 30798, + "寺": 30799, + "於": 30800, + "進": 30801, + "ὸ": 30802, + "風": 30803, + "ன": 30804, + "班": 30805, + "◼": 30806, + "九": 30807, + "̥": 30808, + "號": 30809, + "류": 30810, + "础": 30811, + "般": 30812, + "︙": 30813, + "̈": 30814, + "番": 30815, + "✨": 30816, + "😎": 30817, + "ো": 30818, + "😍": 30819, + "單": 30820, + "帧": 30821, + "授": 30822, + "赋": 30823, + "巴": 30824, + "占": 30825, + "假": 30826, + "ṅ": 30827, + "透": 30828, + "項": 30829, + "ħ": 30830, + "馬": 30831, + "🟢": 30832, + "Ľ": 30833, + "լ": 30834, + "券": 30835, + "같": 30836, + "類": 30837, + "對": 30838, + "월": 30839, + "激": 30840, + "\u0017": 30841, + "戦": 30842, + "独": 30843, + "訊": 30844, + "ិ": 30845, + "套": 30846, + "ʷ": 30847, + "跟": 30848, + "ở": 30849, + "渲": 30850, + "顯": 30851, + "降": 30852, + "ာ": 30853, + "尼": 30854, + "血": 30855, + "언": 30856, + "牛": 30857, + "將": 30858, + "ศ": 30859, + "拍": 30860, + "刻": 30861, + "ზ": 30862, + "╔": 30863, + "藤": 30864, + "్": 30865, + "ῶ": 30866, + "🟠": 30867, + "良": 30868, + "김": 30869, + "দ": 30870, + "Ṣ": 30871, + "録": 30872, + "伊": 30873, + "落": 30874, + "雄": 30875, + "雪": 30876, + "映": 30877, + "著": 30878, + "른": 30879, + "ფ": 30880, + "対": 30881, + "智": 30882, + "译": 30883, + "┬": 30884, + "抽": 30885, + "ῖ": 30886, + "酒": 30887, + "Ћ": 30888, + "股": 30889, + "់": 30890, + "순": 30891, + "직": 30892, + "भ": 30893, + "谷": 30894, + "물": 30895, + "ǒ": 30896, + "⠄": 30897, + "热": 30898, + "終": 30899, + "夹": 30900, + "干": 30901, + "彩": 30902, + "敗": 30903, + "ќ": 30904, + "♯": 30905, + "̣": 30906, + "վ": 30907, + "轮": 30908, + "阵": 30909, + "夏": 30910, + "幕": 30911, + "吧": 30912, + "港": 30913, + "益": 30914, + "儿": 30915, + "액": 30916, + "售": 30917, + "兵": 30918, + "惠": 30919, + "欢": 30920, + "›": 30921, + "零": 30922, + "學": 30923, + "ž": 30924, + "員": 30925, + "ỗ": 30926, + "玉": 30927, + "逻": 30928, + "᥀": 30929, + "吗": 30930, + "沒": 30931, + "≠": 30932, + "너": 30933, + "ச": 30934, + "\u0016": 30935, + "夫": 30936, + "წ": 30937, + "堂": 30938, + "電": 30939, + "≡": 30940, + "陆": 30941, + "져": 30942, + "研": 30943, + "荐": 30944, + "健": 30945, + "碼": 30946, + "练": 30947, + "検": 30948, + "송": 30949, + "ै": 30950, + "哪": 30951, + "圆": 30952, + "Ա": 30953, + "↩": 30954, + "托": 30955, + "̪": 30956, + "ू": 30957, + "缀": 30958, + "네": 30959, + "沙": 30960, + "兴": 30961, + "病": 30962, + "\u0007": 30963, + "ល": 30964, + "ừ": 30965, + "Ἀ": 30966, + "강": 30967, + "항": 30968, + "\u0019": 30969, + "換": 30970, + "温": 30971, + "帖": 30972, + "ទ": 30973, + "込": 30974, + "削": 30975, + "알": 30976, + "征": 30977, + "习": 30978, + "법": 30979, + "栈": 30980, + "绝": 30981, + "": 30982, + "ڕ": 30983, + "圖": 30984, + "苏": 30985, + "発": 30986, + "ု": 30987, + "町": 30988, + "互": 30989, + "়": 30990, + "ც": 30991, + "守": 30992, + "새": 30993, + "侧": 30994, + "草": 30995, + "ས": 30996, + "扫": 30997, + "‒": 30998, + "恢": 30999, + "ң": 31000, + "ण": 31001, + "ற": 31002, + "째": 31003, + "්": 31004, + "拟": 31005, + "派": 31006, + "🏽": 31007, + "呼": 31008, + "Š": 31009, + "演": 31010, + "究": 31011, + "교": 31012, + "ɣ": 31013, + "ए": 31014, + "ី": 31015, + "ף": 31016, + "富": 31017, + "駅": 31018, + "ず": 31019, + "♪": 31020, + "😆": 31021, + "접": 31022, + "ғ": 31023, + "▓": 31024, + "존": 31025, + "ಾ": 31026, + "旋": 31027, + "ゃ": 31028, + "补": 31029, + "ץ": 31030, + "門": 31031, + "ច": 31032, + "날": 31033, + "ภ": 31034, + "ག": 31035, + "傳": 31036, + "∆": 31037, + "†": 31038, + "ׁ": 31039, + "缺": 31040, + "頭": 31041, + "怪": 31042, + "組": 31043, + "별": 31044, + "Ъ": 31045, + "發": 31046, + "雷": 31047, + "ರ": 31048, + "ซ": 31049, + "び": 31050, + "翻": 31051, + "ھ": 31052, + "პ": 31053, + "題": 31054, + "居": 31055, + "집": 31056, + "🌍": 31057, + "˚": 31058, + "避": 31059, + "줄": 31060, + "ុ": 31061, + "滑": 31062, + "故": 31063, + "ญ": 31064, + "〜": 31065, + "ನ": 31066, + "양": 31067, + "완": 31068, + "ள": 31069, + "倍": 31070, + "宗": 31071, + "択": 31072, + "브": 31073, + "ɴ": 31074, + "効": 31075, + "尺": 31076, + "視": 31077, + "ẽ": 31078, + "覆": 31079, + "ध": 31080, + "骨": 31081, + "달": 31082, + "ᴛ": 31083, + "蓝": 31084, + "關": 31085, + "額": 31086, + "Õ": 31087, + "∗": 31088, + "卷": 31089, + "갑": 31090, + "르": 31091, + "众": 31092, + "ᴀ": 31093, + "態": 31094, + "ٰ": 31095, + "暗": 31096, + "君": 31097, + "錯": 31098, + "ɒ": 31099, + "យ": 31100, + "ḫ": 31101, + "ῆ": 31102, + "亚": 31103, + "♡": 31104, + "割": 31105, + "鼠": 31106, + "̶": 31107, + "Ë": 31108, + "読": 31109, + "격": 31110, + "ゲ": 31111, + "眼": 31112, + "Ý": 31113, + "ژ": 31114, + "雨": 31115, + "宮": 31116, + "쪽": 31117, + "ष": 31118, + "複": 31119, + "剩": 31120, + "早": 31121, + "杂": 31122, + "焦": 31123, + "贝": 31124, + "突": 31125, + "워": 31126, + "另": 31127, + "摄": 31128, + "\b": 31129, + "‭": 31130, + "府": 31131, + "외": 31132, + "盖": 31133, + "\u001c": 31134, + "ษ": 31135, + "佛": 31136, + "概": 31137, + "與": 31138, + "經": 31139, + "－": 31140, + "һ": 31141, + "問": 31142, + "ು": 31143, + "ἰ": 31144, + "話": 31145, + "倒": 31146, + "葛": 31147, + "べ": 31148, + "ろ": 31149, + "\u001e": 31150, + "।": 31151, + "ေ": 31152, + "ᴏ": 31153, + "训": 31154, + "體": 31155, + "👌": 31156, + "內": 31157, + "က": 31158, + "企": 31159, + "약": 31160, + "찾": 31161, + "ོ": 31162, + "破": 31163, + "輸": 31164, + "림": 31165, + "塔": 31166, + "턴": 31167, + "杀": 31168, + "』": 31169, + "味": 31170, + "浮": 31171, + "┆": 31172, + "ġ": 31173, + "郡": 31174, + "┐": 31175, + "『": 31176, + "阶": 31177, + "雅": 31178, + "┈": 31179, + "园": 31180, + "．": 31181, + "吃": 31182, + "남": 31183, + " ": 31184, + "ར": 31185, + "帮": 31186, + "毛": 31187, + "耗": 31188, + "举": 31189, + "ర": 31190, + "拿": 31191, + "밀": 31192, + "ご": 31193, + "够": 31194, + "礼": 31195, + "ព": 31196, + "ね": 31197, + "‰": 31198, + "兰": 31199, + "❌": 31200, + "折": 31201, + "십": 31202, + "💎": 31203, + "業": 31204, + "诸": 31205, + "孙": 31206, + "བ": 31207, + "😳": 31208, + "種": 31209, + "Ï": 31210, + "ึ": 31211, + "⁣": 31212, + "医": 31213, + "拼": 31214, + "↵": 31215, + "⅓": 31216, + "\u001f": 31217, + "မ": 31218, + "叫": 31219, + "জ": 31220, + "予": 31221, + "寸": 31222, + "梅": 31223, + "醒": 31224, + "津": 31225, + "န": 31226, + "ి": 31227, + "厂": 31228, + "屋": 31229, + "ख": 31230, + "師": 31231, + "👀": 31232, + "ỏ": 31233, + "ヤ": 31234, + "ὰ": 31235, + "\u001d": 31236, + "◆": 31237, + "ដ": 31238, + "材": 31239, + "ホ": 31240, + "張": 31241, + "洞": 31242, + "餐": 31243, + "천": 31244, + "হ": 31245, + "達": 31246, + "們": 31247, + "斗": 31248, + "横": 31249, + "백": 31250, + "ំ": 31251, + "ۆ": 31252, + "말": 31253, + "গ": 31254, + "佳": 31255, + "랜": 31256, + "仁": 31257, + "陈": 31258, + "飞": 31259, + "极": 31260, + "": 31261, + "및": 31262, + "仓": 31263, + "⬛": 31264, + "昌": 31265, + "錢": 31266, + "殊": 31267, + "┴": 31268, + "○": 31269, + "길": 31270, + "泉": 31271, + "甲": 31272, + "활": 31273, + "ひ": 31274, + "শ": 31275, + "ን": 31276, + "Ť": 31277, + "ღ": 31278, + "皮": 31279, + "強": 31280, + "赛": 31281, + "ా": 31282, + "預": 31283, + "င": 31284, + "튼": 31285, + "플": 31286, + "ყ": 31287, + "⋆": 31288, + "ք": 31289, + "ા": 31290, + "尚": 31291, + "또": 31292, + "բ": 31293, + "┌": 31294, + "節": 31295, + "森": 31296, + "आ": 31297, + "办": 31298, + "園": 31299, + "牙": 31300, + "庆": 31301, + "隆": 31302, + "😔": 31303, + "叉": 31304, + "գ": 31305, + "피": 31306, + "ギ": 31307, + "啊": 31308, + "続": 31309, + "灵": 31310, + "ヒ": 31311, + "忽": 31312, + "ʌ": 31313, + "량": 31314, + "油": 31315, + "讯": 31316, + "ⵉ": 31317, + "릭": 31318, + "刚": 31319, + "氏": 31320, + "ိ": 31321, + "Ī": 31322, + "誤": 31323, + "齐": 31324, + "末": 31325, + "🙌": 31326, + "̞": 31327, + "圈": 31328, + "念": 31329, + "숫": 31330, + "毫": 31331, + "當": 31332, + "規": 31333, + "판": 31334, + "ు": 31335, + "旧": 31336, + "卖": 31337, + "ฉ": 31338, + "幸": 31339, + "署": 31340, + "근": 31341, + "ই": 31342, + "岛": 31343, + "դ": 31344, + "觉": 31345, + "害": 31346, + "毕": 31347, + "ฐ": 31348, + "威": 31349, + "育": 31350, + "呢": 31351, + "峰": 31352, + "职": 31353, + "陽": 31354, + "ි": 31355, + "亞": 31356, + "ұ": 31357, + "₃": 31358, + "따": 31359, + "施": 31360, + "泰": 31361, + "載": 31362, + "…": 31363, + "笑": 31364, + "華": 31365, + "迎": 31366, + "됩": 31367, + "豆": 31368, + "嘉": 31369, + "🤡": 31370, + "ĕ": 31371, + "庄": 31372, + "級": 31373, + "Ψ": 31374, + "ི": 31375, + "気": 31376, + "责": 31377, + "հ": 31378, + "អ": 31379, + "乱": 31380, + "休": 31381, + "約": 31382, + "ฆ": 31383, + "∑": 31384, + "察": 31385, + "온": 31386, + "😬": 31387, + "ড": 31388, + "乘": 31389, + "람": 31390, + "इ": 31391, + "Ά": 31392, + "ந": 31393, + "ើ": 31394, + "亲": 31395, + "េ": 31396, + "委": 31397, + "赤": 31398, + "됨": 31399, + "勝": 31400, + "怎": 31401, + "감": 31402, + "宋": 31403, + "調": 31404, + "짜": 31405, + "ী": 31406, + "难": 31407, + "못": 31408, + "티": 31409, + "備": 31410, + "塞": 31411, + "វ": 31412, + "险": 31413, + "旅": 31414, + "虚": 31415, + "↳": 31416, + "笔": 31417, + "馆": 31418, + "Қ": 31419, + "⚡": 31420, + "ೆ": 31421, + "※": 31422, + "唐": 31423, + "律": 31424, + "稍": 31425, + "散": 31426, + "ર": 31427, + "ヴ": 31428, + "副": 31429, + "尽": 31430, + "挂": 31431, + "県": 31432, + "⚠": 31433, + "洋": 31434, + "鬼": 31435, + "암": 31436, + "孩": 31437, + "℃": 31438, + "並": 31439, + "ց": 31440, + "ូ": 31441, + "ℓ": 31442, + "ⵏ": 31443, + "扣": 31444, + "铁": 31445, + "闻": 31446, + "ˆ": 31447, + "戳": 31448, + "む": 31449, + "秀": 31450, + "細": 31451, + "ပ": 31452, + "御": 31453, + "拖": 31454, + "좌": 31455, + "ؤ": 31456, + "绍": 31457, + "ỹ": 31458, + "참": 31459, + "향": 31460, + "Ď": 31461, + "끝": 31462, + "민": 31463, + "ძ": 31464, + "贵": 31465, + "纪": 31466, + "秋": 31467, + "ಕ": 31468, + "ӏ": 31469, + "網": 31470, + "铺": 31471, + "恋": 31472, + "fl": 31473, + "兼": 31474, + "羽": 31475, + "창": 31476, + "啟": 31477, + "弟": 31478, + "년": 31479, + "慢": 31480, + "효": 31481, + "許": 31482, + "硬": 31483, + "잘": 31484, + "템": 31485, + "્": 31486, + "න": 31487, + "術": 31488, + "ڈ": 31489, + "溪": 31490, + "": 31491, + "暴": 31492, + "混": 31493, + "夢": 31494, + "랑": 31495, + "আ": 31496, + "還": 31497, + "探": 31498, + "祖": 31499, + "织": 31500, + "軍": 31501, + "թ": 31502, + "務": 31503, + "艺": 31504, + "ད": 31505, + "ት": 31506, + "ṁ": 31507, + "應": 31508, + "擇": 31509, + "🥰": 31510, + "ķ": 31511, + "渡": 31512, + "葉": 31513, + "령": 31514, + "決": 31515, + "刀": 31516, + "從": 31517, + "變": 31518, + "올": 31519, + "💪": 31520, + "灣": 31521, + "ር": 31522, + "평": 31523, + "衣": 31524, + "😄": 31525, + "ി": 31526, + "ჩ": 31527, + "ὁ": 31528, + "ほ": 31529, + "Û": 31530, + "চ": 31531, + "ර": 31532, + "製": 31533, + "隊": 31534, + "₱": 31535, + "纳": 31536, + "赖": 31537, + "农": 31538, + "桥": 31539, + "ỳ": 31540, + "🏾": 31541, + "阻": 31542, + "ជ": 31543, + "秘": 31544, + "박": 31545, + "伤": 31546, + "稿": 31547, + "ం": 31548, + "拦": 31549, + "넣": 31550, + "💕": 31551, + "₁": 31552, + "宿": 31553, + "錄": 31554, + "镜": 31555, + "채": 31556, + "Ə": 31557, + "ང": 31558, + "⇔": 31559, + "☼": 31560, + "ུ": 31561, + "党": 31562, + "급": 31563, + "洲": 31564, + "ղ": 31565, + "說": 31566, + "ĭ": 31567, + "尝": 31568, + "담": 31569, + "फ": 31570, + "哥": 31571, + "圣": 31572, + "萨": 31573, + "😏": 31574, + "ʏ": 31575, + "ெ": 31576, + "丁": 31577, + "虎": 31578, + "권": 31579, + "善": 31580, + "岩": 31581, + "커": 31582, + "◦": 31583, + "抛": 31584, + "석": 31585, + "Έ": 31586, + "宣": 31587, + "拳": 31588, + "팅": 31589, + "枚": 31590, + "洛": 31591, + "証": 31592, + "陵": 31593, + "佐": 31594, + "館": 31595, + "누": 31596, + "돌": 31597, + "₄": 31598, + "稱": 31599, + "聊": 31600, + "車": 31601, + "루": 31602, + "״": 31603, + "ಠ": 31604, + "庫": 31605, + "མ": 31606, + "統": 31607, + "련": 31608, + "़": 31609, + "ṯ": 31610, + "ക": 31611, + "旗": 31612, + "励": 31613, + "紀": 31614, + "忠": 31615, + "າ": 31616, + "杨": 31617, + "丹": 31618, + "Ù": 31619, + "ฝ": 31620, + "却": 31621, + "舞": 31622, + "轉": 31623, + "တ": 31624, + "丽": 31625, + "借": 31626, + "ා": 31627, + "ょ": 31628, + "옵": 31629, + "편": 31630, + "蒙": 31631, + "衡": 31632, + "ʋ": 31633, + "叶": 31634, + "̇": 31635, + "⬜": 31636, + "🇺": 31637, + "Հ": 31638, + "谢": 31639, + "Ą": 31640, + "ே": 31641, + "ằ": 31642, + "既": 31643, + "济": 31644, + "≯": 31645, + "準": 31646, + "답": 31647, + "ಲ": 31648, + "残": 31649, + "虑": 31650, + "̆": 31651, + "┘": 31652, + "急": 31653, + "招": 31654, + "막": 31655, + "≮": 31656, + "產": 31657, + "Ṭ": 31658, + "😢": 31659, + "垂": 31660, + "親": 31661, + "ģ": 31662, + "־": 31663, + "猫": 31664, + "ʟ": 31665, + "☃": 31666, + "✪": 31667, + "刪": 31668, + "胡": 31669, + "☉": 31670, + "晚": 31671, + "군": 31672, + "승": 31673, + "న": 31674, + "ὴ": 31675, + "曾": 31676, + "論": 31677, + "ɯ": 31678, + "త": 31679, + "戰": 31680, + "鱼": 31681, + "ǧ": 31682, + "寶": 31683, + "특": 31684, + "💯": 31685, + "崎": 31686, + "甘": 31687, + "該": 31688, + "링": 31689, + "😡": 31690, + "उ": 31691, + "ែ": 31692, + "頁": 31693, + "큰": 31694, + "➤": 31695, + "총": 31696, + "💰": 31697, + "∂": 31698, + "毁": 31699, + "聖": 31700, + "麻": 31701, + "ʐ": 31702, + "敏": 31703, + "運": 31704, + "될": 31705, + "쓰": 31706, + "ಸ": 31707, + "စ": 31708, + "✦": 31709, + "젝": 31710, + "復": 31711, + "寻": 31712, + "茶": 31713, + "ਾ": 31714, + "竹": 31715, + "遇": 31716, + "順": 31717, + "며": 31718, + "累": 31719, + "ĝ": 31720, + "ˇ": 31721, + "覧": 31722, + "এ": 31723, + "株": 31724, + "취": 31725, + "ስ": 31726, + "争": 31727, + "势": 31728, + "宇": 31729, + "橋": 31730, + "Ӏ": 31731, + "堆": 31732, + "ⵙ": 31733, + "丶": 31734, + "棋": 31735, + "肉": 31736, + "የ": 31737, + "": 31738, + "❶": 31739, + "季": 31740, + "ል": 31741, + "殿": 31742, + "優": 31743, + "試": 31744, + "첫": 31745, + "Ό": 31746, + "戶": 31747, + "ண": 31748, + "羅": 31749, + "桃": 31750, + "립": 31751, + "浪": 31752, + "脑": 31753, + "😛": 31754, + "弃": 31755, + "炮": 31756, + "轻": 31757, + "울": 31758, + "": 31759, + "ヘ": 31760, + "奥": 31761, + "💜": 31762, + "忘": 31763, + "遠": 31764, + "飛": 31765, + "魏": 31766, + "Ē": 31767, + "汇": 31768, + "央": 31769, + "逆": 31770, + "露": 31771, + "須": 31772, + "ѐ": 31773, + "ḷ": 31774, + "ದ": 31775, + "✭": 31776, + "寄": 31777, + "盟": 31778, + "财": 31779, + "際": 31780, + "ἔ": 31781, + "ǫ": 31782, + "थ": 31783, + "ാ": 31784, + "宫": 31785, + "巨": 31786, + "途": 31787, + "ʹ": 31788, + "ಗ": 31789, + "帐": 31790, + "‪": 31791, + "拒": 31792, + "药": 31793, + "🙃": 31794, + "ŕ": 31795, + "亡": 31796, + "壁": 31797, + "ም": 31798, + "參": 31799, + "😩": 31800, + "շ": 31801, + "ವ": 31802, + "ណ": 31803, + "丰": 31804, + "獲": 31805, + "莉": 31806, + "좋": 31807, + "ရ": 31808, + "₦": 31809, + "겠": 31810, + "👉": 31811, + "吴": 31812, + "岡": 31813, + "诉": 31814, + "읽": 31815, + "🥺": 31816, + "爆": 31817, + "🇸": 31818, + "ভ": 31819, + "迭": 31820, + "엔": 31821, + "ἄ": 31822, + "捷": 31823, + "納": 31824, + "邀": 31825, + "ಯ": 31826, + "爾": 31827, + "船": 31828, + "赞": 31829, + "胜": 31830, + "므": 31831, + "သ": 31832, + "構": 31833, + "磁": 31834, + "冰": 31835, + "딩": 31836, + "ે": 31837, + "媒": 31838, + "繁": 31839, + "☠": 31840, + "❒": 31841, + "仪": 31842, + "렬": 31843, + "昭": 31844, + "珠": 31845, + "離": 31846, + "ན": 31847, + "ల": 31848, + "ತ": 31849, + "拷": 31850, + "粉": 31851, + "벤": 31852, + "⇽": 31853, + "乌": 31854, + "拥": 31855, + "ҳ": 31856, + "ය": 31857, + "ེ": 31858, + "仙": 31859, + "塊": 31860, + "幅": 31861, + "🎉": 31862, + "Մ": 31863, + "跨": 31864, + "ٔ": 31865, + "恩": 31866, + "损": 31867, + "养": 31868, + "奈": 31869, + "ǀ": 31870, + "严": 31871, + "卫": 31872, + "迟": 31873, + "様": 31874, + "裡": 31875, + "난": 31876, + "았": 31877, + "͜": 31878, + "Ζ": 31879, + "ਰ": 31880, + "պ": 31881, + "ং": 31882, + "丢": 31883, + "伝": 31884, + "컨": 31885, + "ව": 31886, + "ြ": 31887, + "冷": 31888, + "遗": 31889, + "銀": 31890, + "̌": 31891, + "ᴜ": 31892, + "瑞": 31893, + "ฌ": 31894, + "❍": 31895, + "ふ": 31896, + "聚": 31897, + "碎": 31898, + "衛": 31899, + "অ": 31900, + "ញ": 31901, + "퍼": 31902, + "Ս": 31903, + "ນ": 31904, + "ẓ": 31905, + "✌": 31906, + "孝": 31907, + "陳": 31908, + "히": 31909, + "ක": 31910, + "黒": 31911, + "💖": 31912, + "ḩ": 31913, + "応": 31914, + "饰": 31915, + "∪": 31916, + "宜": 31917, + "樂": 31918, + "則": 31919, + "勇": 31920, + "徐": 31921, + "ⵓ": 31922, + "權": 31923, + "鲁": 31924, + "‟": 31925, + "庭": 31926, + "苗": 31927, + "🔴": 31928, + "闲": 31929, + "독": 31930, + "ɹ": 31931, + "ҽ": 31932, + "ថ": 31933, + "宏": 31934, + "尊": 31935, + "總": 31936, + "裝": 31937, + "ම": 31938, + "▸": 31939, + "測": 31940, + "ಮ": 31941, + "አ": 31942, + "轩": 31943, + "兄": 31944, + "剑": 31945, + "ન": 31946, + "朱": 31947, + "ǝ": 31948, + "Ḩ": 31949, + "担": 31950, + "灰": 31951, + "讲": 31952, + "롤": 31953, + "︎": 31954, + "😤": 31955, + "ោ": 31956, + "애": 31957, + "였": 31958, + "질": 31959, + "振": 31960, + "灯": 31961, + "ĉ": 31962, + "ස": 31963, + "閉": 31964, + "램": 31965, + "ಂ": 31966, + "げ": 31967, + "̧": 31968, + "狂": 31969, + "融": 31970, + "仍": 31971, + "實": 31972, + "楽": 31973, + "範": 31974, + "ٌ": 31975, + "వ": 31976, + "嵌": 31977, + "摩": 31978, + "袁": 31979, + "ষ": 31980, + "乎": 31981, + "규": 31982, + "岗": 31983, + "糊": 31984, + "క": 31985, + "雲": 31986, + "심": 31987, + "ई": 31988, + "འ": 31989, + "ἡ": 31990, + "丝": 31991, + "Ħ": 31992, + "ٍ": 31993, + "ٓ": 31994, + "အ": 31995, + "執": 31996, + "벨": 31997, + "ゼ": 31998, + "梦": 31999 + }, + "merges": [ + "▁ t", + "i n", + "e r", + "▁ a", + "h e", + "o n", + "r e", + "▁ s", + "e n", + "a t", + "o r", + "▁t he", + "▁th e", + "▁ the", + "e s", + "▁ w", + "a n", + "▁ c", + "i s", + "i t", + "o u", + "▁ d", + "a l", + "a r", + "▁ p", + "▁ f", + "e d", + "▁ b", + "in g", + "i ng", + "▁ o", + "▁ m", + "l e", + "n d", + "a s", + "i c", + "▁ h", + "io n", + "i on", + "▁i n", + "▁ in", + "▁t o", + "▁ to", + "e t", + "o m", + "e l", + "▁o f", + "▁ of", + "s t", + "▁a nd", + "▁an d", + "▁ and", + "▁ l", + "▁t h", + "▁ th", + "▁ n", + "en t", + "e nt", + "i l", + "c t", + "r o", + "▁r e", + "▁ re", + "i d", + "a m", + "▁ I", + "a d", + "▁ e", + "▁ S", + "▁ g", + "▁ T", + "i m", + "o t", + "a c", + "u r", + "▁ (", + "i g", + "▁ =", + "o l", + "u t", + "▁ A", + "s e", + "▁ u", + "v e", + "▁ C", + "i f", + "o w", + "▁ y", + "c h", + "a y", + "▁d e", + "▁ de", + "▁s t", + "▁ st", + "▁ |", + "ve r", + "v er", + ") ;", + "▁ \"", + "l y", + "▁b e", + "▁ be", + "* *", + "▁i s", + "▁ is", + "o d", + "▁ M", + "at ion", + "ati on", + "atio n", + "u l", + "▁f or", + "▁fo r", + "▁ for", + "▁o n", + "▁ on", + "a g", + "c e", + "te r", + "t er", + "i r", + "t h", + "▁ v", + "q u", + "▁ B", + "e m", + "▁ P", + "▁y ou", + "▁yo u", + "▁ you", + "▁t hat", + "▁th at", + "▁ that", + "u n", + "▁ {", + "it h", + "i th", + "r i", + "es t", + "e st", + "a b", + "- -", + "a p", + "▁i t", + "▁ it", + "▁c on", + "▁co n", + "▁ con", + "at e", + "a te", + "u s", + "▁ H", + "u m", + "▁ D", + "o s", + "p e", + "▁ -", + "▁w h", + "▁ wh", + "▁a l", + "▁ al", + "▁a s", + "▁ as", + "an d", + "a nd", + "is t", + "i st", + "▁ L", + "▁ W", + "▁w ith", + "▁ with", + "▁a n", + "▁ an", + "er e", + "e re", + "▁ *", + "▁ R", + "▁h e", + "▁ he", + "▁ F", + "o c", + "▁w as", + "▁wa s", + "▁ was", + "er s", + "e rs", + "k e", + "ou t", + "o ut", + "h t", + "▁ r", + "es s", + "e ss", + "o p", + "re s", + "r es", + "i e", + "▁ E", + "▁ \\", + "▁T he", + "▁Th e", + "▁ The", + "en d", + "e nd", + "l d", + "▁ N", + "or t", + "o rt", + "▁ G", + "/ /", + "▁ #", + "ou r", + "o ur", + "t e", + "il l", + "i ll", + "ai n", + "a in", + "▁s e", + "▁ se", + "▁ $", + "▁p ro", + "▁pr o", + "▁ pro", + "or e", + "o re", + "▁c om", + "▁co m", + "▁ com", + "am e", + "a me", + "t r", + "▁n e", + "▁ ne", + "ro m", + "r om", + "u b", + "▁a t", + "▁ at", + "▁e x", + "▁ ex", + "an t", + "a nt", + "u e", + "▁o r", + "▁ or", + "▁ }", + "ar t", + "a rt", + "ct ion", + "▁ k", + "p t", + "n t", + "i v", + "d e", + "▁ O", + "p l", + "ur n", + "u rn", + "ig ht", + "igh t", + "i ght", + "al l", + "a ll", + "▁t his", + "▁th is", + "▁ this", + "se r", + "s er", + "av e", + "a ve", + "▁n ot", + "▁no t", + "▁ not", + "▁a re", + "▁ar e", + "▁ are", + "▁ j", + "▁l e", + "▁ le", + "i z", + "▁ '", + "ag e", + "a ge", + "me nt", + "men t", + "m ent", + "▁t r", + "▁ tr", + "ac k", + "a ck", + "us t", + "u st", + "( )", + "- >", + "it y", + "i ty", + "in e", + "i ne", + "ou ld", + "oul d", + "o uld", + "▁ J", + "o g", + "▁f rom", + "▁fr om", + "▁fro m", + "▁ from", + "▁w e", + "▁ we", + "el l", + "e ll", + "▁s h", + "▁ sh", + "▁e n", + "▁ en", + "ur e", + "u re", + "por t", + "po rt", + "p ort", + "▁c h", + "▁ ch", + "n e", + "▁b y", + "▁ by", + "pe r", + "p er", + "ar d", + "a rd", + "as s", + "a ss", + "g e", + "a k", + "ar e", + "a re", + "o k", + "a v", + "iv e", + "i ve", + "f f", + "ie s", + "i es", + "at h", + "a th", + "tu rn", + "t urn", + "▁ U", + "in t", + "i nt", + "-- --", + "--- -", + "- ---", + "▁i m", + "▁ im", + "os t", + "o st", + "ia l", + "i al", + "▁h ave", + "▁ha ve", + "▁hav e", + "▁ have", + "in d", + "i nd", + "i p", + "an s", + "a ns", + "x t", + "▁d o", + "▁ do", + "c l", + "▁i f", + "▁ if", + "co n", + "c on", + "i a", + "▁h is", + "▁hi s", + "▁ his", + "ul t", + "u lt", + "ro u", + "r ou", + "▁s u", + "▁ su", + "r a", + "▁u n", + "▁ un", + "ab le", + "abl e", + "a ble", + "▁ <", + "▁ K", + "om e", + "o me", + "▁q u", + "▁ qu", + "ge t", + "g et", + "▁m e", + "▁ me", + "as t", + "a st", + "ec t", + "e ct", + "▁# #", + "▁ ##", + "t o", + "▁c l", + "▁ cl", + "▁a b", + "▁ ab", + "ic e", + "i ce", + "ir e", + "i re", + "be r", + "b er", + "on e", + "o ne", + "ic h", + "i ch", + "he n", + "h en", + "▁c an", + "▁ca n", + "▁ can", + "▁T h", + "▁ Th", + "▁l a", + "▁ la", + "▁a ll", + "▁al l", + "▁ all", + "im e", + "i me", + "il e", + "i le", + "id e", + "i de", + "\" ,", + "▁p l", + "▁ pl", + "▁ V", + "r u", + "or m", + "o rm", + "▁h ad", + "▁ha d", + "▁ had", + "u d", + "as e", + "a se", + "or d", + "o rd", + ") ,", + "▁h er", + "▁he r", + "▁ her", + "▁I n", + "▁ In", + "ac e", + "a ce", + "▁b ut", + "▁bu t", + "▁ but", + "at a", + "a ta", + ": :", + "** **", + "*** *", + "* ***", + "on g", + "o ng", + "▁ &", + ". .", + "it e", + "i te", + "yp e", + "y pe", + "ac t", + "a ct", + "od e", + "o de", + "▁y our", + "▁you r", + "▁yo ur", + "▁ your", + "▁o ut", + "▁ou t", + "▁ out", + "▁g o", + "▁ go", + "li c", + "l ic", + "al ly", + "all y", + "▁s o", + "▁ so", + "or k", + "a u", + "▁u p", + "▁ up", + "▁ _", + "l l", + "= =", + "▁m y", + "▁ my", + "p p", + "c c", + "▁/ /", + "▁ //", + "▁the y", + "▁th ey", + "▁ they", + "g h", + "▁u s", + "▁ us", + "i b", + "ion s", + "io ns", + "i ons", + "ac h", + "a ch", + "en s", + "e ns", + "▁a r", + "▁ ar", + "o b", + "el f", + "oo k", + "o ok", + "at ed", + "ate d", + "a ted", + "an g", + "a ng", + "ig n", + "i gn", + "▁re turn", + "▁r eturn", + "▁ret urn", + "▁ return", + "▁re s", + "▁r es", + "▁ res", + "c k", + "ou s", + "o us", + "с т", + ") .", + "▁ п", + ". \"", + "н а", + "▁ i", + "ai l", + "a il", + "e p", + "▁a d", + "▁ ad", + "an ce", + "anc e", + "( \"", + "▁* *", + "▁ **", + "th er", + "the r", + "t her", + "ak e", + "a ke", + "▁w ill", + "▁ will", + "▁c omp", + "▁com p", + "▁co mp", + "▁ comp", + "▁o ne", + "▁on e", + "▁ one", + "▁g et", + "▁ge t", + "▁ get", + "o v", + "▁ Y", + "ar y", + "a ry", + "oc k", + "o ck", + "▁s he", + "▁sh e", + "▁ she", + "ch e", + "c he", + "f t", + "▁n ew", + "▁ne w", + "▁ new", + "▁d es", + "▁de s", + "▁ des", + "▁l i", + "▁ li", + "en ce", + "enc e", + "▁s a", + "▁ sa", + "re ss", + "res s", + "r ess", + "▁e l", + "▁ el", + "▁u nd", + "▁un d", + "▁ und", + "e g", + "fe r", + "f er", + "r y", + "ea r", + "e ar", + "os e", + "o se", + "ve ry", + "ver y", + "v ery", + "' ,", + "▁ +", + "▁ в", + "▁H e", + "▁ He", + "ub lic", + "ubl ic", + "u blic", + "▁the ir", + "iz e", + "i ze", + "▁w ere", + "▁we re", + "▁wer e", + "▁ were", + "in k", + "ow n", + "o wn", + "I n", + "{ \\", + "▁h as", + "▁ha s", + "▁ has", + "▁p er", + "▁pe r", + "▁ per", + "▁I t", + "▁ It", + "▁S t", + "▁ St", + "he r", + "h er", + "je ct", + "j ect", + "р а", + "il d", + "i ld", + "s o", + "▁s p", + "▁ sp", + "н и", + "d u", + "ro w", + "r ow", + "al ue", + "alu e", + "se t", + "s et", + "fo rm", + "for m", + "f orm", + "co m", + "c om", + "▁m an", + "▁ma n", + "▁ man", + "on t", + "o nt", + "ul l", + "u ll", + "▁c ont", + "▁con t", + "▁co nt", + "▁ cont", + "▁m ore", + "▁mor e", + "▁mo re", + "▁ more", + "ic k", + "i ck", + "▁w ould", + "▁wo uld", + "▁e v", + "▁ ev", + "▁ab out", + "▁ about", + "it ion", + "iti on", + "▁ z", + "ou nd", + "oun d", + "o und", + "re e", + "r ee", + "▁C h", + "▁ Ch", + "▁wh ich", + "▁ which", + "i o", + "() ;", + "( );", + "▁w ho", + "▁wh o", + "▁ who", + "er r", + "e rr", + "or y", + "o ry", + "ou nt", + "oun t", + "o unt", + "at ions", + "ation s", + "ati ons", + "atio ns", + "▁ с", + "ri ng", + "rin g", + "r ing", + "< /", + "▁f e", + "▁ fe", + "к о", + "н о", + "▁d is", + "▁di s", + "▁ dis", + "m a", + "▁t hem", + "▁the m", + "▁th em", + "▁a ny", + "▁an y", + "▁ any", + "▁n o", + "▁ no", + "-- ------", + "---- ----", + "--- -----", + "----- ---", + "------ --", + "------- -", + "- -------", + "▁p re", + "▁pr e", + "▁ pre", + "▁t e", + "▁ te", + "▁r o", + "▁ ro", + "▁h im", + "▁hi m", + "▁ him", + "▁ :", + "u p", + "▁in t", + "▁i nt", + "▁ int", + "▁a g", + "▁ ag", + "S t", + "ar k", + "e x", + "p h", + "ie nt", + "ien t", + "i ent", + "el y", + "e ly", + "▁p r", + "▁ pr", + "E R", + "▁im port", + "▁imp ort", + "▁ import", + "▁t ime", + "▁tim e", + "▁ti me", + "▁ time", + "р о", + "pr o", + "p ro", + "Us er", + "Use r", + "U ser", + "l o", + "▁ /", + "▁ [", + "or s", + "o rs", + "= \"", + "▁t here", + "▁the re", + "▁th ere", + "▁ther e", + "▁ there", + "▁l ike", + "▁li ke", + "▁lik e", + "▁ like", + "ol d", + "o ld", + "▁w hen", + "▁wh en", + "▁whe n", + "▁ when", + "ve rs", + "ver s", + "v ers", + "▁s ome", + "▁so me", + "▁som e", + "▁ some", + "in gs", + "ing s", + ") )", + "▁p art", + "▁par t", + "▁pa rt", + "▁ part", + "ic al", + "ica l", + "i cal", + "▁f un", + "▁fu n", + "▁ fun", + "▁k n", + "▁ kn", + "ay s", + "a ys", + "ie r", + "i er", + "▁b een", + "▁be en", + "ov e", + "o ve", + "▁s c", + "▁ sc", + "ia n", + "i an", + "▁o ver", + "▁ov er", + "▁ over", + "ie l", + "i el", + "▁p e", + "▁ pe", + "ri b", + "r ib", + "pu t", + "p ut", + "e c", + "et h", + "e th", + "ar am", + "ara m", + "a ram", + "ap p", + "a pp", + "▁ –", + "▁s tat", + "▁st at", + "▁sta t", + "▁ stat", + "po n", + "p on", + "▁w hat", + "▁wh at", + "▁ what", + "pt ion", + "w e", + "ad e", + "a de", + "▁w ork", + "▁wor k", + "▁ work", + "te xt", + "tex t", + "t ext", + "▁s aid", + "▁sa id", + "▁# ##", + "▁## #", + "▁ ###", + "I N", + "▁j ust", + "▁ju st", + "▁ just", + "ir st", + "irs t", + "▁in to", + "▁int o", + "▁ into", + "▁con st", + "▁cons t", + "▁ const", + "our ce", + "t t", + "p s", + "p r", + "er v", + "e rv", + "it t", + "i tt", + "u g", + "_ {", + "en ts", + "ent s", + "is h", + "i sh", + "en er", + "ene r", + "e ner", + "▁in ter", + "▁int er", + "▁inte r", + "▁ inter", + "pl e", + "p le", + "ol l", + "o ll", + "me r", + "m er", + "at er", + "ate r", + "a ter", + "oo l", + "o ol", + "e f", + "▁p ublic", + "▁pub lic", + "▁pu blic", + "▁publi c", + "▁ public", + "▁o ther", + "▁ot her", + "▁ other", + "р е", + "▁d ef", + "▁de f", + "▁ def", + "▁ @", + "г о", + "oin t", + "oi nt", + "o int", + "▁o ff", + "▁of f", + "▁ off", + "oi d", + "o id", + "re turn", + "ret urn", + "r eturn", + "▁s et", + "▁se t", + "▁ set", + "w o", + "ft er", + "fte r", + "f ter", + "s h", + "** ******", + "**** ****", + "****** **", + "▁o ur", + "▁ou r", + "▁ our", + "ri v", + "r iv", + "is s", + "i ss", + "▁W e", + "▁ We", + "n g", + "▁o b", + "▁ ob", + "s s", + "g r", + "▁t han", + "▁th an", + "▁ than", + "pe ct", + "pec t", + "p ect", + "ie d", + "i ed", + "s c", + "ie w", + "i ew", + "de r", + "d er", + "ys t", + "y st", + "e v", + "▁c ould", + "▁co uld", + "▁cou ld", + "▁ could", + "an n", + "a nn", + "en c", + "e nc", + "O N", + "i x", + "an c", + "a nc", + "▁al so", + "▁als o", + "▁ also", + "re at", + "rea t", + "▁a m", + "▁ am", + "▁b ec", + "▁be c", + "▁ bec", + "▁ и", + "ua l", + "u al", + "pe c", + "p ec", + "▁ .", + "▁b l", + "▁ bl", + "le ct", + "l ect", + "op le", + "opl e", + "o ple", + "y s", + "▁g r", + "▁ gr", + "ic t", + "i ct", + "i k", + "tr ing", + "tri ng", + "t ring", + "▁T his", + "▁Th is", + "▁ This", + "▁b ack", + "▁ba ck", + "▁ back", + "▁ о", + "▁f in", + "▁fi n", + "▁ fin", + "at ch", + "Co n", + "C on", + "( '", + "er m", + "e rm", + "▁= =", + "▁ ==", + "_ _", + "na me", + "nam e", + "n ame", + ", \"", + "▁d id", + "▁di d", + "▁ did", + "is e", + "i se", + "▁on ly", + "▁ only", + "ru ct", + "r uct", + "le s", + "l es", + "▁t hen", + "▁the n", + "▁th en", + "▁ then", + "au se", + "aus e", + "a use", + "в а", + "▁it s", + "▁i ts", + "▁ its", + "ri t", + "r it", + "▁k now", + "▁kn ow", + "▁ know", + "ie ld", + "iel d", + "i eld", + "▁c lass", + "▁cl ass", + "▁clas s", + "▁ class", + "▁ >", + "▁e m", + "▁ em", + "▁$ \\", + "▁ $\\", + "▁y ear", + "▁ye ar", + "▁ year", + "w n", + "} ,", + "▁d el", + "▁de l", + "▁ del", + "al e", + "a le", + "t y", + "fi g", + "f ig", + "s p", + "he d", + "h ed", + "ro und", + "rou nd", + "r ound", + "e w", + "▁d i", + "▁ di", + "▁d er", + "▁de r", + "▁ der", + "р и", + "re d", + "r ed", + "th is", + "t his", + "le t", + "l et", + "R E", + "a x", + "f r", + "ess age", + "essa ge", + "ou gh", + "o ugh", + "▁c omm", + "▁com m", + "▁co mm", + "▁ comm", + "f o", + "uc h", + "u ch", + "o y", + "▁pe ople", + "▁ people", + "yst em", + "ys tem", + "▁f irst", + "▁fir st", + "▁ first", + "▁f unction", + "▁fun ction", + "▁ function", + "an ge", + "ang e", + "▁h ow", + "▁ho w", + "▁ how", + "▁e t", + "▁ et", + "a h", + "▁l ook", + "▁lo ok", + "▁ look", + "т о", + "un d", + "u nd", + "▁u nder", + "▁un der", + "▁und er", + "▁ under", + "к а", + "▁ !", + "ra y", + "r ay", + "S T", + "if ic", + "ifi c", + "i fic", + "л и", + "re ad", + "rea d", + "r ead", + "▁b et", + "▁be t", + "▁ bet", + "io us", + "i ous", + "ar g", + "a rg", + "▁n eed", + "▁ne ed", + "▁ need", + "ma th", + "mat h", + "m ath", + "▁н а", + "▁ на", + "er t", + "e rt", + "▁o p", + "▁ op", + "▁a cc", + "▁ac c", + "▁ acc", + "Pr o", + "P ro", + "▁e st", + "▁es t", + "▁ est", + "▁U n", + "▁ Un", + "▁e nt", + "▁en t", + "▁ ent", + "▁re c", + "▁r ec", + "▁ rec", + "▁u se", + "▁us e", + "▁ use", + "е н", + "▁p ar", + "▁pa r", + "▁ par", + "a z", + "▁ д", + "▁W h", + "▁ Wh", + "sel f", + "s elf", + "▁k e", + "▁ ke", + "т а", + "▁w ant", + "▁wa nt", + "▁ want", + "▁e nd", + "▁en d", + "▁ end", + "▁d on", + "▁do n", + "▁ don", + "e k", + "re n", + "r en", + "Na me", + "N ame", + "▁= >", + "▁ =>", + "▁a pp", + "▁ap p", + "▁ app", + "▁qu e", + "▁q ue", + "▁ que", + "ig h", + "i gh", + "▁b u", + "▁ bu", + "eq u", + "e qu", + "ve l", + "v el", + "▁a ct", + "▁ac t", + "▁ act", + "cr e", + "c re", + "A T", + "▁v ar", + "▁va r", + "▁ var", + "ce ss", + "ces s", + "c ess", + "== ==", + "=== =", + "= ===", + "E x", + "▁a dd", + "▁ad d", + "▁ add", + "▁m od", + "▁mo d", + "▁ mod", + "un g", + "u ng", + "▁w here", + "▁wh ere", + "▁whe re", + "▁ where", + "ni ng", + "n ing", + "▁f l", + "▁ fl", + "al s", + "a ls", + "ter n", + "te rn", + "t ern", + "} }", + "▁A l", + "▁ Al", + "▁p os", + "▁po s", + "▁ pos", + "an k", + "▁a p", + "▁ ap", + "en g", + "e ng", + "▁ “", + "bl e", + "b le", + "▁re g", + "▁r eg", + "▁ reg", + "^ {", + "▁S he", + "▁Sh e", + "▁ She", + "▁* /", + "▁ */", + "ud e", + "u de", + "ad d", + "a dd", + "▁t wo", + "▁tw o", + "▁ two", + "▁c ol", + "▁co l", + "▁ col", + "▁s m", + "▁ sm", + "ai r", + "a ir", + "▁m ay", + "▁ma y", + "▁ may", + "fo re", + "for e", + "f ore", + "▁Y ou", + "▁ You", + "ro ugh", + "rou gh", + "r ough", + "▁c he", + "▁ch e", + "▁ che", + "▁a tt", + "▁at t", + "▁ att", + "ot h", + "o th", + "л а", + "▁c o", + "▁ co", + "at es", + "ate s", + "a tes", + "▁re m", + "▁r em", + "▁ rem", + "oo d", + "o od", + "Ty pe", + "Typ e", + "T ype", + "le d", + "l ed", + "fu l", + "f ul", + "▁s elf", + "▁sel f", + "▁ self", + "o f", + "▁A r", + "▁ Ar", + "qu e", + "q ue", + "▁e very", + "▁ev ery", + "▁ever y", + "▁ every", + "re f", + "r ef", + "Th e", + "T he", + "▁A nd", + "▁An d", + "▁ And", + "▁re l", + "▁r el", + "▁ rel", + "O R", + "I d", + "▁e ven", + "▁ev en", + "▁ even", + "E N", + "▁h and", + "▁ha nd", + "▁han d", + "▁ hand", + "ai t", + "a it", + "▁sh ould", + "▁ should", + "▁a fter", + "▁af ter", + "▁ after", + "▁d if", + "▁di f", + "gh t", + "g ht", + "if e", + "i fe", + "at or", + "ato r", + "a tor", + "as h", + "a sh", + "ri but", + "rib ut", + "ribu t", + "um ber", + "umb er", + "u mber", + "▁s ee", + "▁se e", + "▁ see", + "m s", + "▁c all", + "▁cal l", + "▁ca ll", + "▁ call", + "y n", + "d d", + "▁e s", + "▁ es", + "▁m ake", + "▁ma ke", + "▁ make", + "ot her", + "oth er", + "othe r", + "o ther", + "▁ —", + "\") ;", + "\" );", + "st r", + "s tr", + "▁l ong", + "▁lo ng", + "▁lon g", + "▁ long", + "le ment", + "lem ent", + "l ement", + "▁w or", + "▁wo r", + "▁ wor", + "it s", + "i ts", + "▁I f", + "▁ If", + "al se", + "als e", + "л ь", + "wa rd", + "war d", + "w ard", + "▁п о", + "▁ по", + "va l", + "v al", + "on s", + "o ns", + "▁ Z", + "▁n ow", + "▁no w", + "▁ now", + "da ta", + "dat a", + "d ata", + "am p", + "a mp", + "en se", + "ens e", + "▁th rough", + "▁thr ough", + "▁thro ugh", + "▁ through", + "▁d own", + "▁do wn", + "▁dow n", + "▁ down", + "at t", + "a tt", + "▁st atic", + "▁stat ic", + "▁ static", + "ic s", + "i cs", + "# #", + "po s", + "p os", + "▁v oid", + "▁vo id", + "▁ void", + "a w", + "ou n", + "o un", + "▁w ay", + "▁wa y", + "▁ way", + "ib le", + "i ble", + "ve nt", + "ven t", + "v ent", + "ow er", + "owe r", + "o wer", + "▁th ink", + "▁thin k", + "▁ think", + "t s", + "* /", + "▁a gain", + "▁ag ain", + "▁ again", + "at ing", + "ati ng", + "atin g", + "a ting", + "т е", + "ne r", + "n er", + "▁m ost", + "▁mo st", + "▁mos t", + "▁ most", + "li ne", + "lin e", + "l ine", + "y m", + "▁s ub", + "▁su b", + "▁ sub", + "er son", + "ers on", + "▁re qu", + "▁r equ", + "▁req u", + "▁ requ", + "A L", + "A R", + "ab el", + "abe l", + "a bel", + "on d", + "o nd", + ")) ;", + ") );", + "▁S e", + "▁ Se", + "▁B ut", + "▁Bu t", + "▁ But", + "al k", + "▁A n", + "▁ An", + "ne w", + "n ew", + "▁b ecause", + "▁bec ause", + "▁ because", + "ge r", + "g er", + "ul ar", + "ula r", + "u lar", + "ro up", + "rou p", + "r oup", + "t a", + ".. .", + ". ..", + "▁c ons", + "▁con s", + "▁co ns", + "▁ cons", + "▁r ight", + "▁ri ght", + "▁rig ht", + "▁ right", + "▁f r", + "▁ fr", + "b e", + "il y", + "i ly", + "к и", + "▁p h", + "▁ ph", + "ea d", + "e ad", + "? \"", + "▁g u", + "▁ gu", + "▁el se", + "▁els e", + "▁ else", + "▁s om", + "▁so m", + "▁ som", + "re nt", + "ren t", + "r ent", + "c o", + "em ent", + "eme nt", + "emen t", + "e ment", + "▁s tr", + "▁st r", + "▁ str", + "au lt", + "aul t", + "a ult", + "▁ з", + "л о", + "se rt", + "ser t", + "s ert", + "va r", + "v ar", + "ty pe", + "typ e", + "t ype", + "▁C om", + "▁Co m", + "▁ Com", + "л е", + "in s", + "i ns", + "m e", + "wa y", + "w ay", + "id ent", + "ide nt", + "iden t", + "▁p rov", + "▁pro v", + "▁pr ov", + "▁ prov", + "▁ м", + "▁tr ue", + "▁ true", + "▁P ro", + "▁Pr o", + "▁ Pro", + "f l", + "▁s l", + "▁ sl", + "▁A s", + "▁ As", + "} \\", + "I D", + "ue s", + "u es", + "▁in st", + "▁ins t", + "▁ inst", + "▁n ame", + "▁na me", + "▁nam e", + "▁ name", + "o x", + "▁ )", + "l i", + "am es", + "ame s", + "a mes", + "Re s", + "R es", + "▁s ur", + "▁su r", + "▁ sur", + "par am", + "pa ram", + "para m", + "p aram", + "▁st art", + "▁star t", + "▁sta rt", + "▁ start", + "a j", + "S E", + "as k", + "a sk", + "I T", + "St ring", + "Str ing", + "S tring", + "▁a ss", + "▁as s", + "▁ ass", + "▁p lay", + "▁pl ay", + "▁ play", + "ti ng", + "t ing", + "to n", + "t on", + "▁b efore", + "▁be fore", + "▁bef ore", + "▁ before", + "▁p ol", + "▁po l", + "▁ pol", + "ar ch", + "arc h", + "▁w ell", + "▁we ll", + "▁wel l", + "▁ well", + "Co m", + "C om", + "an y", + "a ny", + "ol og", + "olo g", + "o log", + "▁e rr", + "▁er r", + "▁ err", + "▁the se", + "▁th ese", + "ar s", + "a rs", + "e b", + "▁b r", + "▁ br", + "▁in cl", + "▁inc l", + "▁ incl", + "▁h el", + "▁he l", + "▁ hel", + "er n", + "e rn", + "od y", + "o dy", + "в о", + "▁in d", + "▁i nd", + "▁ ind", + "-- --------------", + "---- ------------", + "-------- --------", + "--- -------------", + "------------ ----", + "----- -----------", + "---------- ------", + "------ ----------", + "------------- ---", + "-------------- --", + "--------- -------", + "------- ---------", + "----------- -----", + "▁d ata", + "▁da ta", + "▁dat a", + "▁ data", + "▁g ood", + "▁go od", + "▁ good", + "L E", + "] ,", + "▁a v", + "▁ av", + "▁a c", + "▁ ac", + "id er", + "ide r", + "i der", + "н е", + "▁ Q", + "▁m in", + "▁mi n", + "▁ min", + "▁m uch", + "▁mu ch", + "c i", + "el s", + "e ls", + "▁c ur", + "▁cu r", + "▁ cur", + "▁v alue", + "▁val ue", + "▁ value", + "er y", + "e ry", + "u f", + "▁l oc", + "▁lo c", + "▁ loc", + "re ak", + "rea k", + "at ive", + "ati ve", + "ativ e", + "im es", + "ime s", + "i mes", + "C l", + "▁ ,", + "▁s er", + "▁se r", + "▁ ser", + "▁d ie", + "▁di e", + "▁ die", + "▁tr ans", + "▁tra ns", + "▁ trans", + "▁res ult", + "▁ result", + "ex t", + "e xt", + "▁a ut", + "▁au t", + "▁ aut", + "la nd", + "lan d", + "l and", + "▁& &", + "▁ &&", + "C h", + "te n", + "t en", + "} $", + "▁t ype", + "▁typ e", + "▁ty pe", + "▁ type", + "con d", + "co nd", + "c ond", + "ic es", + "ice s", + "i ces", + "▁v ery", + "▁ver y", + "▁ve ry", + "▁ very", + "▁o wn", + "▁ own", + "▁f il", + "▁fi l", + "▁ fil", + "it ies", + "iti es", + "i ties", + "▁p rodu", + "▁pro du", + "▁prod u", + "▁ produ", + "▁re ad", + "▁r ead", + "▁ read", + "▁f orm", + "▁for m", + "▁fo rm", + "▁ form", + "▁c ase", + "▁cas e", + "▁ca se", + "▁ case", + "at her", + "ath er", + "a ther", + "т и", + "д а", + "е р", + "T h", + "au t", + "a ut", + "▁s pec", + "▁sp ec", + "▁spe c", + "▁ spec", + "i j", + "b l", + "il ity", + "ili ty", + "▁ é", + "▁e r", + "▁ er", + "▁d oes", + "▁do es", + "▁ does", + "▁h ere", + "▁he re", + "▁her e", + "▁ here", + "th e", + "t he", + "ur es", + "ure s", + "u res", + "▁ %", + "mi n", + "m in", + "▁n ull", + "▁nu ll", + "▁ null", + "ra p", + "r ap", + "\" )", + "r r", + "Li st", + "L ist", + "ri ght", + "rig ht", + "r ight", + "▁U ser", + "▁Us er", + "▁Use r", + "▁ User", + "U L", + "at ional", + "ation al", + "ati onal", + "atio nal", + "▁b eing", + "▁be ing", + "▁bei ng", + "▁ being", + "A N", + "s k", + "▁c ar", + "▁ca r", + "▁ car", + "ol e", + "o le", + "▁d ist", + "▁dis t", + "▁di st", + "▁ dist", + "pl ic", + "p lic", + "ol low", + "oll ow", + "▁p res", + "▁pre s", + "▁pr es", + "▁ pres", + "▁s uch", + "▁su ch", + "▁suc h", + "▁ such", + "re am", + "rea m", + "in ce", + "inc e", + "ga n", + "g an", + "▁F or", + "▁Fo r", + "▁ For", + "\" :", + "so n", + "s on", + "riv ate", + "▁y ears", + "▁year s", + "▁ye ars", + "▁s erv", + "▁se rv", + "▁ser v", + "▁ serv", + "▁m ade", + "▁ma de", + "▁mad e", + "▁ made", + "de f", + "d ef", + "; \r", + "▁g l", + "▁ gl", + "▁b el", + "▁be l", + "▁ bel", + "▁l ist", + "▁li st", + "▁ list", + "▁c or", + "▁co r", + "▁ cor", + "▁d et", + "▁de t", + "▁ det", + "ce ption", + "cept ion", + "eg in", + "e gin", + "▁ б", + "▁c har", + "▁ch ar", + "▁cha r", + "▁ char", + "tr ans", + "tra ns", + "▁f am", + "▁fa m", + "▁! =", + "▁ !=", + "ou se", + "ous e", + "o use", + "▁d ec", + "▁de c", + "▁ dec", + "ic a", + "i ca", + "▁m any", + "▁man y", + "▁ma ny", + "▁ many", + "ak ing", + "aki ng", + "a king", + "▁ à", + "▁s im", + "▁si m", + "▁ sim", + "ag es", + "age s", + "a ges", + "uf f", + "u ff", + "as ed", + "ase d", + "a sed", + "ma n", + "m an", + "▁S h", + "▁ Sh", + "ie t", + "i et", + "ir ect", + "ire ct", + "i rect", + "▁R e", + "▁ Re", + "▁d iffer", + "▁dif fer", + "▁diff er", + "▁f ind", + "▁fin d", + "▁fi nd", + "▁ find", + "eth od", + "▁ \r", + "in es", + "ine s", + "i nes", + "▁in v", + "▁i nv", + "▁ inv", + "▁p oint", + "▁po int", + "▁poi nt", + "▁ point", + "▁The y", + "▁Th ey", + "▁ They", + "▁u sed", + "▁us ed", + "▁use d", + "▁ used", + "ct ions", + "ction s", + "▁st ill", + "i ó", + "in ed", + "ine d", + "i ned", + "▁wh ile", + "▁ while", + "I t", + "em ber", + "emb er", + "e mber", + "▁s ay", + "▁sa y", + "▁ say", + "▁he lp", + "▁hel p", + "▁ help", + "▁c re", + "▁cr e", + "▁ cre", + "▁ x", + "▁T r", + "▁ Tr", + "um ent", + "ume nt", + "umen t", + "u ment", + "▁s k", + "▁ sk", + "ou ght", + "ough t", + "ual ly", + "u ally", + "m essage", + "▁C on", + "▁Co n", + "▁ Con", + "▁m on", + "▁mo n", + "▁ mon", + "ar ed", + "are d", + "a red", + "wor k", + "w ork", + ") :", + "is ter", + "ist er", + "iste r", + "i ster", + "ar n", + "a rn", + "iz ed", + "ize d", + "i zed", + "Dat a", + "Da ta", + "D ata", + "or n", + "o rn", + "▁h ead", + "▁he ad", + "▁ head", + "D E", + "▁L e", + "▁ Le", + "▁p erson", + "▁per son", + "▁pers on", + "▁ person", + "ment s", + "men ts", + "m ents", + "eng th", + "e ngth", + "▁f alse", + "▁fal se", + "▁fals e", + "▁ false", + "▁m ed", + "▁me d", + "▁ med", + "▁D e", + "▁ De", + "ac he", + "ach e", + "a che", + "it ed", + "ite d", + "i ted", + "▁l et", + "▁le t", + "▁ let", + "▁s how", + "▁sh ow", + "▁ show", + "▁s ame", + "▁sa me", + "▁sam e", + "▁ same", + "us s", + "u ss", + "▁g ener", + "▁gen er", + "▁ge ner", + "▁gene r", + "▁ gener", + "▁ у", + "cu r", + "c ur", + "▁re al", + "▁ real", + "ce d", + "c ed", + "\" >", + "st ruct", + "str uct", + "stru ct", + "be gin", + "b egin", + "ce pt", + "cep t", + "▁b o", + "▁ bo", + "ir ed", + "ire d", + "i red", + "▁F r", + "▁ Fr", + "▁st ud", + "▁ stud", + "de v", + "d ev", + "A r", + "( \\", + "▁C l", + "▁ Cl", + "we en", + "w een", + "▁t oo", + "▁to o", + "▁ too", + "▁t est", + "▁te st", + "▁ test", + "▁d ay", + "▁da y", + "▁ day", + "o h", + "▁f ollow", + "▁fol low", + "▁ follow", + "at ure", + "atur e", + "atu re", + "z e", + "ie n", + "i en", + "re g", + "r eg", + "ce s", + "c es", + "ur ing", + "uri ng", + "u ring", + "am b", + "a mb", + "in a", + "i na", + "cr i", + "c ri", + "▁e d", + "▁ ed", + "S S", + "uc k", + "u ck", + "▁/ *", + "▁ /*", + "C T", + "▁T here", + "▁The re", + "▁Th ere", + "▁Ther e", + "▁ There", + "▁t ake", + "▁tak e", + "▁ta ke", + "▁ take", + "pa r", + "p ar", + "ul e", + "u le", + "ca l", + "c al", + "fo r", + "f or", + "** **************", + "**** ************", + "******** ********", + "************ ****", + "************** **", + "s ource", + "▁th ose", + "co l", + "c ol", + "▁e ff", + "▁ eff", + "mo d", + "m od", + "con t", + "co nt", + "c ont", + "} {", + "▁a round", + "▁ar ound", + "▁ around", + "pr ess", + "pre ss", + "pres s", + "p ress", + "b y", + "▁go ing", + "▁ going", + "pon se", + "pons e", + "▁ С", + "▁l ine", + "▁li ne", + "▁lin e", + "▁ line", + "da te", + "dat e", + "d ate", + "co de", + "cod e", + "c ode", + "[ '", + "▁l ife", + "▁li fe", + "▁lif e", + "▁ life", + "as on", + "a son", + "▁u sing", + "▁us ing", + "▁ using", + "▁v al", + "▁va l", + "▁ val", + "▁d u", + "▁ du", + "y p", + "▁O n", + "▁ On", + "▁f ound", + "▁fo und", + "▁fou nd", + "▁ found", + "ol ut", + "olu t", + "' ]", + "ar ent", + "are nt", + "aren t", + "a rent", + "▁s tring", + "▁st ring", + "▁str ing", + "▁stri ng", + "▁ string", + "▁m et", + "▁me t", + "▁ met", + "▁w r", + "▁ wr", + "us h", + "u sh", + "st ring", + "str ing", + "stri ng", + "s tring", + "si ze", + "s ize", + "▁v er", + "▁ve r", + "▁ ver", + "▁e ach", + "▁ each", + "val ue", + "v alue", + "▁l ast", + "▁la st", + "▁las t", + "▁ last", + "▁g ot", + "▁go t", + "▁ got", + "ve n", + "v en", + "ba ck", + "b ack", + "Se t", + "S et", + "e y", + "ro l", + "r ol", + "▁c r", + "▁ cr", + "th ing", + "t hing", + "re t", + "r et", + "é s", + "is m", + "i sm", + "▁bet ween", + "▁ between", + "O b", + "et hing", + "eth ing", + "e thing", + "m p", + "▁l o", + "▁ lo", + "at s", + "a ts", + "▁N ew", + "▁Ne w", + "▁ New", + "в и", + "ad o", + "a do", + "de x", + "d ex", + "д и", + "▁p ass", + "▁pas s", + "▁pa ss", + "▁ pass", + "w h", + "▁d en", + "▁de n", + "▁ den", + "Ge t", + "G et", + "ap t", + "a pt", + "▁a sk", + "▁as k", + "▁ ask", + "▁s up", + "▁su p", + "▁ sup", + "Val ue", + "V alue", + "н ы", + "▁t ry", + "▁tr y", + "▁ try", + "lat ion", + "l ation", + "da y", + "d ay", + "ne ss", + "nes s", + "n ess", + "et s", + "e ts", + "▁ex per", + "▁exp er", + "▁ exper", + "T r", + "▁M ar", + "▁Ma r", + "▁ Mar", + "se rv", + "ser v", + "s erv", + "b r", + "▁n umber", + "▁num ber", + "▁nu mber", + "▁ number", + "in al", + "ina l", + "i nal", + "ce nt", + "cen t", + "c ent", + "/ *", + "no t", + "n ot", + "ion al", + "io nal", + "iona l", + "i onal", + "▁f inal", + "▁fin al", + "▁fi nal", + "▁ final", + "' )", + "▁r un", + "▁ru n", + "▁ run", + "ov er", + "ove r", + "o ver", + "▁n ever", + "▁ne ver", + "▁ never", + "u c", + "▁h igh", + "▁hig h", + "▁hi gh", + "▁ high", + "yl e", + "y le", + "▁in s", + "▁i ns", + "▁ ins", + "▁b est", + "▁be st", + "▁bes t", + "▁ best", + "it tle", + "itt le", + "ri c", + "r ic", + "▁s ign", + "▁si gn", + "▁sig n", + "▁ sign", + "▁d em", + "▁de m", + "▁ dem", + "in ess", + "ine ss", + "ines s", + "i ness", + "g y", + "▁w ar", + "▁wa r", + "▁ war", + "is hed", + "ish ed", + "▁g iv", + "▁gi v", + "ke y", + "k ey", + "▁ X", + "( $", + "▁ch ild", + "▁chi ld", + "▁ child", + "le ss", + "les s", + "l ess", + "way s", + "wa ys", + "w ays", + "in cl", + "inc l", + "ro p", + "r op", + "ra w", + "r aw", + ": //", + "▁ «", + "n o", + "ind ow", + "indo w", + "f e", + "ri end", + "rie nd", + "rien d", + "▁l es", + "▁le s", + "▁ les", + "▁l os", + "▁lo s", + "▁ los", + "fil e", + "fi le", + "f ile", + "form ation", + "format ion", + "cc ess", + "c cess", + "▁ В", + "n a", + "▁i l", + "▁ il", + "is ion", + "isi on", + "le r", + "l er", + "▁a rt", + "▁ar t", + "▁ art", + "Con t", + "Co nt", + "C ont", + "▁w orld", + "▁wor ld", + "▁ world", + "▁t urn", + "▁tu rn", + "▁tur n", + "▁ turn", + "▁re ally", + "▁real ly", + "▁E x", + "▁ Ex", + "м а", + "▁ П", + "ter s", + "te rs", + "t ers", + "ar get", + "arg et", + "arge t", + "Er r", + "E rr", + "▁h app", + "▁ha pp", + "ti me", + "tim e", + "t ime", + "▁S o", + "▁ So", + "di v", + "d iv", + "▁did n", + "▁di dn", + "ad a", + "a da", + "oo t", + "o ot", + "} )", + "▁s ch", + "▁sc h", + "▁ sch", + "▁c le", + "▁cl e", + "▁ cle", + "▁some thing", + "▁som ething", + "▁somet hing", + "▁ something", + "() .", + "( ).", + "▁c our", + "▁co ur", + "▁cou r", + "ev er", + "eve r", + "e ver", + "an ts", + "ant s", + "▁ ?", + "T o", + "▁ `", + "tr y", + "t ry", + "u x", + "ai s", + "a is", + "ro ss", + "ros s", + "r oss", + "hi p", + "h ip", + "▁re p", + "▁r ep", + "▁ rep", + "la bel", + "lab el", + "l abel", + "▁b oth", + "▁bo th", + "▁bot h", + "▁ both", + "* ,", + "ot t", + "o tt", + "м и", + "an e", + "a ne", + "▁o pen", + "▁op en", + "▁ open", + "w w", + "▁c ome", + "▁com e", + "▁co me", + "▁ come", + "▁e xt", + "▁ex t", + "▁ ext", + "re m", + "r em", + "_{ \\", + "_ {\\", + "▁o ld", + "▁ol d", + "▁ old", + "ch ed", + "che d", + "c hed", + ". _", + "M E", + "if y", + "i fy", + "g g", + "Co l", + "C ol", + "vi ew", + "v iew", + "▁b us", + "▁bu s", + "▁ bus", + "▁m ust", + "▁mus t", + "▁mu st", + "▁ must", + "▁d ifferent", + "▁differ ent", + "lo g", + "l og", + "is ts", + "ist s", + "i sts", + "ro ll", + "rol l", + "r oll", + "a i", + "▁з а", + "▁ за", + "▁s ystem", + "▁sys tem", + "▁syst em", + "▁ system", + "iv ers", + "ive rs", + "iver s", + "i vers", + "at us", + "atu s", + "ot e", + "o te", + "me d", + "m ed", + "] .", + "ak es", + "ake s", + "a kes", + "R O", + "▁c ent", + "▁ce nt", + "▁ cent", + "gr am", + "gra m", + "g ram", + "▁p rivate", + "▁priv ate", + "▁ private", + "▁g reat", + "▁gre at", + "\" ;", + "op y", + "o py", + "▁fe el", + "▁fee l", + "▁H ow", + "▁Ho w", + "▁ How", + "// //", + "/// /", + "/ ///", + "I C", + "▁d r", + "▁ dr", + "ain s", + "ai ns", + "a ins", + "lo ck", + "loc k", + "l ock", + "E n", + "▁S ch", + "▁Sc h", + "▁ Sch", + "▁m at", + "▁ma t", + "▁ mat", + "▁h ome", + "▁hom e", + "▁ho me", + "▁ home", + "per ty", + "pert y", + "te st", + "tes t", + "t est", + "lo c", + "l oc", + "▁w om", + "▁wo m", + "s w", + "ar ly", + "arl y", + "▁E n", + "▁ En", + "▁к о", + "▁ ко", + "de n", + "d en", + "ст а", + "с та", + "▁ а", + "et er", + "ete r", + "e ter", + "▁incl ud", + "▁inclu d", + "UL L", + "U LL", + "▁m em", + "▁me m", + "▁ mem", + "▁p o", + "▁ po", + "▁l ittle", + "▁lit tle", + "▁litt le", + "▁a rg", + "▁ar g", + "▁ arg", + "▁} ,", + "▁ },", + "in clude", + "incl ude", + "et a", + "e ta", + "▁p lace", + "▁pl ace", + "▁plac e", + "▁ place", + "id th", + "us tom", + "ust om", + "▁| |", + "▁ ||", + "▁t em", + "▁te m", + "▁ tem", + "ri ed", + "rie d", + "r ied", + "▁f act", + "▁fac t", + "▁fa ct", + "▁ fact", + "ien ce", + "i ence", + "▁P l", + "▁ Pl", + "op t", + "o pt", + "el e", + "e le", + "g o", + "A C", + "in ter", + "int er", + "inte r", + "==== ====", + "() ,", + "( ),", + "ot s", + "o ts", + "ra l", + "r al", + "iqu e", + "iq ue", + "i que", + "av ing", + "avi ng", + "a ving", + "m l", + "▁th ought", + "▁though t", + "▁thou ght", + "fr ac", + "f rac", + "▁c are", + "▁car e", + "▁ca re", + "▁ care", + "() );", + "()) ;", + "( ));", + "▁p ut", + "▁pu t", + "▁ put", + "▁m ight", + "▁mi ght", + "▁mig ht", + "▁A mer", + "▁Am er", + "▁ Amer", + "▁( !", + "▁ (!", + "am ple", + "amp le", + "al th", + "alt h", + "▁f ew", + "▁fe w", + "▁st ate", + "▁stat e", + "▁sta te", + "▁ state", + "su b", + "s ub", + "▁O r", + "▁ Or", + "] ;", + "▁s ize", + "▁si ze", + "▁ size", + "▁S p", + "▁ Sp", + "▁with out", + "▁ without", + "▁p oss", + "▁pos s", + "▁po ss", + "▁ poss", + "e q", + "pl ay", + "p lay", + "▁ex pect", + "▁exp ect", + "▁ expect", + "▁se cond", + "▁sec ond", + "▁ second", + "▁S tring", + "▁St ring", + "▁Str ing", + "▁ String", + "ui ld", + "u ild", + "▁n ext", + "▁ne xt", + "▁ next", + "+ +", + "re qu", + "req u", + "r equ", + "▁A ll", + "▁Al l", + "▁ All", + "▁m en", + "▁me n", + "▁ men", + "▁W hen", + "▁Wh en", + "▁Whe n", + "▁ When", + "it er", + "ite r", + "i ter", + "am ent", + "ame nt", + "amen t", + "a ment", + "ne t", + "n et", + "▁ К", + "ro n", + "r on", + "ain t", + "ai nt", + "a int", + "▁I s", + "▁ Is", + "в е", + "pe nd", + "pen d", + "p end", + "trans lation", + "transl ation", + "▁г о", + "▁ го", + "ч е", + "▁v an", + "▁va n", + "▁ van", + "▁an other", + "▁ano ther", + "▁re t", + "▁r et", + "▁ ret", + "▁L a", + "▁ La", + "Mo d", + "M od", + "IO N", + "I ON", + "li st", + "l ist", + "▁p ost", + "▁pos t", + "▁po st", + "▁ post", + "d a", + "wa re", + "war e", + "w are", + "▁w ord", + "▁wor d", + "▁wo rd", + "▁ word", + "Err or", + "Er ror", + "▁se em", + "▁see m", + "▁cont in", + "▁ contin", + "at ic", + "ati c", + "▁th ree", + "▁thr ee", + "▁ three", + "Ob ject", + "Obj ect", + "▁part ic", + "▁parti c", + "$ .", + "▁m ark", + "▁mar k", + "▁ mark", + "▁v is", + "▁vi s", + "▁ vis", + "r c", + "▁s w", + "▁ sw", + "pt ions", + "ption s", + "▁b reak", + "▁bre ak", + "▁ break", + "▁th ings", + "▁thing s", + "▁thin gs", + "ut e", + "u te", + "u i", + "▁T hat", + "▁Th at", + "▁ That", + "ur s", + "u rs", + "g l", + "р у", + "▁f ile", + "▁fil e", + "▁fi le", + "▁ file", + "us e", + "u se", + "ig ned", + "ign ed", + "igne d", + "par t", + "pa rt", + "p art", + "U n", + "▁e qu", + "▁eq u", + "▁ equ", + "( &", + "▁l ead", + "▁le ad", + "r m", + "ain ed", + "ai ned", + "aine d", + "a ined", + "▁B e", + "▁ Be", + "pat h", + "pa th", + "p ath", + "▁sm all", + "▁ small", + "ag er", + "age r", + "a ger", + "▁al ways", + "▁ always", + "▁E l", + "▁ El", + "▁or der", + "▁ord er", + "▁ order", + "▁e y", + "▁ ey", + "▁w on", + "▁wo n", + "▁ won", + "ap e", + "a pe", + "▁l eft", + "▁le ft", + "▁ left", + "av a", + "a va", + "it em", + "ite m", + "i tem", + "ho r", + "h or", + "▁a way", + "▁aw ay", + "▁ away", + "b b", + "fu n", + "f un", + "▁I nd", + "▁In d", + "▁ Ind", + "m b", + "▁st ruct", + "▁str uct", + "▁stru ct", + "▁ struct", + "▁pro cess", + "▁proc ess", + "▁proces s", + "▁ process", + "▁s upport", + "▁sup port", + "▁supp ort", + "▁ support", + "); \r", + ") ;\r", + "ió n", + "i ón", + "L O", + "▁o per", + "▁op er", + "▁ oper", + "U T", + "▁ ·", + "P E", + "lo ad", + "l oad", + "of f", + "o ff", + "▁N o", + "▁ No", + "iv es", + "ive s", + "i ves", + "ic an", + "ica n", + "i can", + "▁v e", + "▁ ve", + "act ion", + "a ction", + "' ;", + "▁v o", + "▁ vo", + "$ ,", + "▁G r", + "▁ Gr", + "pr e", + "p re", + "n y", + "ain ing", + "ai ning", + "a ining", + "io r", + "i or", + "in it", + "ini t", + "i nit", + "le ction", + "lect ion", + "l ection", + "ar m", + "a rm", + "um n", + "u mn", + "ag s", + "a gs", + "ц и", + "ск о", + "с ко", + "vers ion", + "v ersion", + "▁T o", + "▁ To", + "▁re f", + "▁r ef", + "▁ ref", + "st and", + "sta nd", + "stan d", + "▁A t", + "▁ At", + "if t", + "i ft", + "▁e in", + "fa ce", + "fac e", + "f ace", + "b o", + "if ied", + "ifi ed", + "ve d", + "v ed", + "su m", + "s um", + "un e", + "u ne", + "it al", + "ita l", + "i tal", + "um p", + "u mp", + "com m", + "co mm", + "c omm", + "▁m ov", + "▁mo v", + "▁ mov", + "el t", + "e lt", + "▁v on", + "▁vo n", + "vel op", + "ct or", + "c tor", + "he ad", + "h ead", + "cl e", + "c le", + "▁b uild", + "▁bu ild", + "▁ build", + "in c", + "i nc", + ". '", + "b s", + "in fo", + "inf o", + "ch n", + "c hn", + "▁we ek", + "▁ week", + "▁b ook", + "▁bo ok", + "▁ book", + "H E", + "ba r", + "b ar", + "ic ense", + "▁W hat", + "▁Wh at", + "▁ What", + "▁qu est", + "▁que st", + "▁q uest", + "▁ quest", + "ur ch", + "at o", + "a to", + "le ft", + "l eft", + "▁m ar", + "▁ma r", + "▁ mar", + "▁t op", + "▁to p", + "▁ top", + "F F", + "▁f riend", + "▁ friend", + "▁b eh", + "▁be h", + "▁f ield", + "▁fi eld", + "▁ field", + "▁again st", + "ra ct", + "rac t", + "r act", + "iz ation", + "us er", + "use r", + "u ser", + "ch en", + "che n", + "c hen", + "▁ke ep", + "▁ keep", + "A D", + "it or", + "ito r", + "i tor", + "▁n on", + "▁no n", + "▁ non", + "ir d", + "i rd", + "op e", + "o pe", + "▁re st", + "▁r est", + "▁res t", + "▁ rest", + "▁d ev", + "▁de v", + "▁ dev", + "▁_ _", + "▁ __", + "▁u na", + "▁un a", + "▁ una", + "▁t erm", + "▁te rm", + "▁ter m", + "▁ term", + "I S", + "▁p op", + "▁po p", + "▁ pop", + "ri st", + "ris t", + "r ist", + "▁s ince", + "▁sin ce", + "▁sinc e", + "▁ since", + "ve s", + "v es", + "▁h ard", + "▁ha rd", + "▁har d", + "▁ hard", + "p i", + "ut il", + "uti l", + "u til", + "▁s oc", + "▁so c", + "▁ soc", + "en e", + "e ne", + "Ex ception", + "▁l ocal", + "▁loc al", + "▁lo cal", + "▁ local", + "▁d irect", + "▁di rect", + "▁dire ct", + "▁dir ect", + "▁ direct", + "▁s ure", + "▁su re", + "▁sur e", + "▁ sure", + "▁b ro", + "▁br o", + "▁ bro", + "▁d a", + "▁ da", + "▁< /", + "▁ ", + "ai m", + "a im", + "▁s ervice", + "▁serv ice", + "▁ service", + "▁with in", + "an gu", + "ang u", + "▁ Д", + "uf fer", + "uff er", + "A G", + "▁D o", + "▁ Do", + "▁in cre", + "▁inc re", + "▁under stand", + "} ^", + "▁look ed", + "▁lo oked", + "ge n", + "g en", + "ail ed", + "ai led", + "a iled", + "▁ е", + "ay er", + "aye r", + "a yer", + "▁O ne", + "▁On e", + "▁ One", + "▁b as", + "▁ba s", + "▁ bas", + "▁j ob", + "▁jo b", + "▁ job", + "m u", + "bu t", + "b ut", + "el ta", + "elt a", + "▁Ch rist", + "▁Chris t", + "▁ Christ", + "ur ation", + "▁re cord", + "▁rec ord", + "▁ record", + "▁Un ivers", + "▁ Univers", + "iv id", + "ivi d", + "i vid", + "val id", + "▁ Р", + "▁h old", + "▁hol d", + "▁ho ld", + "▁ hold", + "▁t able", + "▁tab le", + "▁ta ble", + "▁ table", + "on es", + "one s", + "o nes", + "lin k", + "l ink", + "▁G e", + "▁ Ge", + "▁of fer", + "▁off er", + "st er", + "ste r", + "s ter", + "For m", + "F orm", + "= {", + "▁н е", + "▁ не", + "st ance", + "stan ce", + "▁g overn", + "▁go vern", + "▁gover n", + "▁ govern", + "▁te chn", + "▁tech n", + "▁ techn", + "▁p rim", + "▁pr im", + "▁pri m", + "▁ prim", + "* .", + "ch o", + "c ho", + "ma x", + "m ax", + "▁f ore", + "▁for e", + "▁fo re", + "▁ fore", + "▁C an", + "▁Ca n", + "▁ Can", + "▁pol it", + "▁po lit", + "▁ polit", + "or ies", + "ori es", + "orie s", + "o ries", + "▁t imes", + "▁time s", + "▁tim es", + "▁ti mes", + "▁ times", + "▁d ans", + "▁da ns", + "▁dan s", + "▁a ir", + "▁ai r", + "▁ air", + "▁any thing", + "▁s ever", + "▁se ver", + "ac y", + "a cy", + "} _", + "H e", + "▁l east", + "▁le ast", + "ip s", + "i ps", + "EN T", + "E NT", + "d o", + "▁о т", + "▁ от", + "▁c ost", + "▁co st", + "▁cos t", + "▁ cost", + ". ”", + "▁child ren", + "▁ children", + "ab ility", + "abil ity", + "Bu t", + "B ut", + "▁p ath", + "▁pat h", + "▁pa th", + "▁ path", + "res ult", + "ac ter", + "act er", + "▁e lement", + "▁el ement", + "▁ele ment", + "▁elem ent", + "▁ element", + "e e", + "▁w ait", + "▁wa it", + "▁ wait", + "▁m oney", + "▁mon ey", + "▁mo ney", + "Ma p", + "M ap", + "t d", + "oi n", + "o in", + "iv ing", + "ivi ng", + "i ving", + "ic ht", + "ich t", + "i cht", + "ic y", + "i cy", + "sc h", + "s ch", + "st e", + "s te", + "д у", + "or ed", + "ore d", + "o red", + "ou d", + "o ud", + "il le", + "ill e", + "i lle", + "is ed", + "ise d", + "i sed", + "pl ication", + "plic ation", + "▁c ustom", + "▁cust om", + "▁ custom", + "▁h aving", + "▁ha ving", + "▁hav ing", + "pon ent", + "po nent", + "▁B y", + "▁ By", + "ul es", + "ule s", + "u les", + "ue d", + "u ed", + "at ter", + "att er", + "atte r", + "An d", + "A nd", + "it ive", + "iti ve", + "De f", + "D ef", + "▁m oment", + "▁mom ent", + "▁mo ment", + "▁ moment", + "at erial", + "ate rial", + "ater ial", + "Cl ass", + "C lass", + "og raph", + "ograp h", + "o graph", + "ik e", + "i ke", + "▁l arge", + "▁larg e", + "▁ large", + "▁# ###", + "▁## ##", + "▁### #", + "▁ ####", + "▁e ither", + "du ct", + "duc t", + "d uct", + "▁T hen", + "▁The n", + "▁Th en", + "▁ Then", + "▁G u", + "▁ Gu", + "ole an", + "o lean", + "pe rt", + "per t", + "p ert", + "▁G et", + "▁Ge t", + "▁ Get", + "▁A b", + "▁ Ab", + "▁sh ort", + "▁ short", + "O n", + "im ent", + "ime nt", + "imen t", + "i ment", + "▁pro ject", + "▁ project", + "cri pt", + "cr ipt", + "c ript", + "▁incl uding", + "▁includ ing", + "▁inclu ding", + "▁ including", + "ни я", + "▁m aking", + "▁ma king", + "▁ making", + "▁some one", + "▁F l", + "▁ Fl", + "▁s at", + "▁sa t", + "▁ sat", + "▁comp any", + "▁compan y", + "▁ company", + "oc us", + "p u", + "▁G od", + "▁Go d", + "▁ God", + "if ication", + "ific ation", + "N o", + "▁s n", + "▁ sn", + "an o", + "a no", + "g a", + "▁a u", + "▁ au", + "▁c ou", + "▁co u", + "▁ cou", + "á s", + "en ded", + "end ed", + "ende d", + "т у", + "ob er", + "obe r", + "o ber", + "▁n othing", + "▁not hing", + "▁no thing", + "▁n et", + "▁ne t", + "▁ net", + "▁p ot", + "▁po t", + "▁ pot", + "▁t yp", + "▁ty p", + "▁ typ", + "▁it em", + "▁i tem", + "▁ item", + "re w", + "r ew", + "At t", + "A tt", + "▁you ng", + "▁yo ung", + "} \r", + "nd er", + "nde r", + "n der", + "st art", + "sta rt", + "star t", + "▁S c", + "▁ Sc", + "* )", + "▁e nc", + "▁en c", + "▁ enc", + "▁w omen", + "▁wom en", + "▁wo men", + "▁look ing", + "▁lo oking", + "▁ looking", + "▁р о", + "▁ ро", + "▁he alth", + "▁heal th", + "▁ health", + "Pat h", + "P ath", + "▁A fter", + "▁Af ter", + "▁ After", + "▁m ult", + "▁mu lt", + "▁mul t", + "▁ mult", + "▁{ \\", + "▁ {\\", + "▁l and", + "▁la nd", + "▁lan d", + "▁ land", + "or ld", + "▁D es", + "▁De s", + "▁ Des", + "▁e ng", + "▁en g", + "▁ eng", + "in put", + "▁P ol", + "▁Po l", + "▁ Pol", + "\" \"", + "Co de", + "C ode", + "▁s upp", + "▁su pp", + "▁sup p", + "▁ supp", + "ain er", + "ai ner", + "aine r", + "a iner", + "he ck", + "▁m or", + "▁mo r", + "▁ mor", + "▁m ill", + "▁mil l", + "▁mi ll", + "▁ mill", + "▁a w", + "▁ aw", + "f s", + "▁do ing", + "ting s", + "t ings", + "ad es", + "ade s", + "a des", + "▁to get", + "▁c ertain", + "▁cert ain", + "▁cer tain", + "▁t ogether", + "▁toget her", + "C E", + "ide o", + "▁Amer ican", + "▁America n", + "▁ American", + "on y", + "o ny", + "id d", + "i dd", + "I I", + "ge d", + "g ed", + "ab les", + "able s", + "abl es", + "a bles", + "▁ide nt", + "▁id ent", + "▁ ident", + "io d", + "i od", + "▁p arent", + "▁par ent", + "▁pa rent", + "▁pare nt", + "▁ parent", + "F or", + "amb da", + "an do", + "and o", + "= \\", + "ag ed", + "age d", + "a ged", + "en ding", + "end ing", + "In t", + "I nt", + "▁poss ible", + "▁ possible", + "▁с о", + "▁ со", + "iv ity", + "ivi ty", + "nu m", + "n um", + "r t", + "aj or", + "ajo r", + "a jor", + "cre ate", + "creat e", + "c reate", + "ri de", + "rid e", + "r ide", + "▁k new", + "▁kn ew", + "▁kne w", + "bi t", + "b it", + "it ional", + "ition al", + "iti onal", + "▁l ik", + "▁li k", + "▁ lik", + "▁H er", + "▁He r", + "▁ Her", + "ens ion", + "\" .", + "ot o", + "o to", + "▁ex ist", + "▁ exist", + "ak en", + "ake n", + "a ken", + "▁act ually", + "▁actual ly", + "c a", + "▁ Г", + "х о", + "in n", + "i nn", + "Al l", + "A ll", + "bu f", + "b uf", + "▁M e", + "▁ Me", + "▁s een", + "▁se en", + "▁see n", + "▁ seen", + "op s", + "o ps", + "No t", + "N ot", + "▁cont rol", + "▁contr ol", + "▁contro l", + "▁ control", + "▁res pon", + "▁resp on", + "▁ respon", + "} ;", + "il t", + "i lt", + "is k", + "i sk", + "▁b ad", + "▁ba d", + "▁ bad", + "▁o ften", + "▁of ten", + "▁p ast", + "▁pas t", + "▁pa st", + "ap er", + "ape r", + "a per", + "▁re ason", + "▁ reason", + "et ers", + "eter s", + "ete rs", + "e ters", + "▁w anted", + "▁want ed", + "ur a", + "u ra", + "ta ble", + "tab le", + "t able", + "or mal", + "orm al", + "wid th", + "w idth", + "г а", + "pt r", + "p tr", + "▁d est", + "▁de st", + "▁des t", + "▁ dest", + "▁de sign", + "▁des ign", + "▁ design", + "▁s ound", + "▁so und", + "▁sou nd", + "▁ sound", + "▁p lan", + "▁pl an", + "▁ plan", + "▁b ase", + "▁bas e", + "▁ba se", + "▁ base", + "ha nd", + "han d", + "h and", + "g s", + "▁s ays", + "▁sa ys", + "▁say s", + "fun ction", + "f unction", + "▁t ri", + "▁tr i", + "▁ tri", + "m t", + "▁in vest", + "▁inv est", + "▁av ailable", + "▁ available", + "ay out", + "a yout", + "▁o ch", + "▁oc h", + "▁ och", + "▁l as", + "▁la s", + "▁ las", + "il led", + "ill ed", + "ille d", + "V al", + "▁ ф", + "ie ty", + "iet y", + "i ety", + "mo n", + "m on", + "Ha nd", + "H and", + "F r", + "ia m", + "i am", + "pa ce", + "p ace", + "▁O b", + "▁ Ob", + "▁p ara", + "▁par a", + "▁pa ra", + "▁ para", + "▁me et", + "▁s um", + "▁su m", + "▁ sum", + "M essage", + "ic i", + "i ci", + "▁k nown", + "▁kn own", + "▁know n", + "▁ known", + "▁g en", + "▁ge n", + "▁ gen", + "am ma", + "amm a", + "a mma", + "ar r", + "a rr", + "▁t re", + "▁tr e", + "▁ tre", + "ok e", + "o ke", + "ut h", + "u th", + "~ \\", + "▁exper ience", + "▁experi ence", + "ic le", + "icl e", + "i cle", + "▁I l", + "▁ Il", + "▁s ent", + "▁se nt", + "▁sen t", + "▁ sent", + "▁o thers", + "▁other s", + "▁ others", + "▁s oft", + "▁so ft", + "▁ soft", + "I P", + "▁m ax", + "▁ma x", + "▁ max", + "ba ll", + "bal l", + "b all", + "▁mark et", + "▁mar ket", + "▁ market", + "▁p our", + "▁po ur", + "▁pou r", + "pr ession", + "press ion", + "p ression", + "ep s", + "e ps", + "▁s aw", + "▁sa w", + "▁a cross", + "▁ac ross", + "▁S u", + "▁ Su", + "O ver", + "ни е", + "ul ation", + "u lation", + "▁R eg", + "▁Re g", + "▁ Reg", + "▁+ =", + "▁ +=", + "bo dy", + "b ody", + ") \\", + "▁pr int", + "▁pri nt", + "▁prin t", + "▁ print", + "▁п ри", + "▁пр и", + "▁ при", + "d b", + "our ces", + "ource s", + "ward s", + "war ds", + "w ards", + "▁bl ack", + "▁ black", + "с о", + "il i", + "i li", + "▁E d", + "▁ Ed", + "▁com plet", + "▁comp let", + "▁compl et", + "▁s ingle", + "▁sing le", + "▁sin gle", + "▁ single", + "▁I N", + "▁ IN", + "ac hed", + "ach ed", + "ache d", + "a ched", + "b t", + "▁c ode", + "▁co de", + "▁cod e", + "▁ code", + "▁b ool", + "▁bo ol", + "▁ bool", + "▁a rea", + "▁are a", + "▁ar ea", + "▁ area", + "▁re quire", + "▁requ ire", + "▁ require", + "▁pro blem", + "▁proble m", + "▁prob lem", + "ac ed", + "ace d", + "a ced", + "Eq u", + "E qu", + "▁con fig", + "▁conf ig", + "▁ config", + "ve c", + "v ec", + "ne y", + "n ey", + "c y", + "A l", + "▁acc ount", + "▁ac count", + "▁ account", + "ym bol", + "▁s te", + "▁st e", + "▁ ste", + "ge s", + "g es", + "Ar ray", + "Arr ay", + "em pl", + "emp l", + "con text", + "cont ext", + "De s", + "D es", + "Res ult", + "ec ut", + "e cut", + "▁t arget", + "▁tar get", + "▁ target", + "▁get ting", + "\" />", + "og le", + "o gle", + "▁him self", + "▁was n", + "▁wa sn", + "▁b lock", + "▁bl ock", + "▁blo ck", + "▁ block", + "▁a nt", + "▁an t", + "▁ ant", + "▁Y ork", + "▁be come", + "▁bec ome", + "if f", + "i ff", + "port s", + "por ts", + "p orts", + "re ate", + "reat e", + "rea te", + "= '", + "c d", + "loc ation", + "l ocation", + "е т", + "▁a ccess", + "▁acc ess", + "▁ac cess", + "▁ access", + "gr ess", + "gre ss", + "gres s", + "g ress", + "ro s", + "r os", + "U p", + "▁work ing", + "▁wor king", + "▁ working", + "▁A m", + "▁ Am", + "iq u", + "i qu", + "ce r", + "c er", + "▁( (", + "▁ ((", + "▁P er", + "▁Pe r", + "▁ Per", + "▁f unc", + "▁fun c", + "▁fu nc", + "▁ func", + "▁g irl", + "▁gi rl", + "▁gir l", + "▁ girl", + "▁ab ove", + "pe n", + "p en", + "п и", + "id o", + "i do", + "▁v ersion", + "▁vers ion", + "▁ version", + "T Y", + "▁ ;", + "ma ry", + "mar y", + "m ary", + "ab led", + "able d", + "abl ed", + "a bled", + "an nel", + "ann el", + "anne l", + "▁ex ample", + "▁exam ple", + "▁ example", + "▁con text", + "▁cont ext", + "▁ context", + "O P", + "▁re d", + "▁r ed", + "▁ red", + "▁c ir", + "▁ci r", + "▁ cir", + "s m", + "Lo g", + "L og", + "▁s pace", + "▁sp ace", + "▁ space", + "▁f ut", + "▁fu t", + "▁G ener", + "▁Ge ner", + "▁Gen er", + "▁Gene r", + "▁ Gener", + "il ls", + "ill s", + "▁d ri", + "▁dr i", + "_ .", + "▁f elt", + "▁fe lt", + "▁fel t", + "▁o ffic", + "▁of fic", + "▁off ic", + "▁= ==", + "▁== =", + "▁ ===", + "i i", + "▁start ed", + "▁star ted", + "▁ Т", + "▁} );", + "▁}) ;", + "▁ });", + "j s", + "▁fr ont", + "▁fro nt", + "▁ front", + "▁al most", + "ir m", + "i rm", + "! \"", + "sign ed", + "sig ned", + "s igned", + "▁y et", + "▁ye t", + "▁t rad", + "▁tr ad", + "▁tra d", + "ient s", + "ien ts", + "i ents", + "am a", + "a ma", + "▁in put", + "▁ input", + "li m", + "l im", + "п а", + "▁к а", + "▁ ка", + "▁c amp", + "▁cam p", + "▁ca mp", + "▁ camp", + "ib r", + "i br", + "fe ct", + "f ect", + "un t", + "u nt", + "▁h alf", + "▁hal f", + "▁ half", + "▁c over", + "▁co ver", + "▁cov er", + "▁ cover", + "angu age", + "▁b en", + "▁be n", + "▁ ben", + "h a", + "▁d iff", + "▁di ff", + "▁dif f", + "▁ diff", + "_ \\", + "▁о б", + "▁ об", + "] )", + "od es", + "ode s", + "o des", + "he l", + "h el", + "io s", + "i os", + "▁ О", + "▁m ot", + "▁mo t", + "▁ mot", + "▁s ocial", + "▁so cial", + "▁soc ial", + "▁soci al", + "▁ social", + "//// ////", + "▁s tre", + "▁st re", + "▁str e", + "▁ stre", + "gr ound", + "gro und", + "g round", + "і в", + "ob ject", + "obj ect", + "pl es", + "ple s", + "p les", + "re ed", + "ree d", + "r eed", + "▁e en", + "▁ een", + "▁b ased", + "▁bas ed", + "▁base d", + "▁ba sed", + "▁ based", + "▁r ange", + "▁ran ge", + "▁rang e", + "▁ range", + "A n", + "ur g", + "u rg", + "▁le arn", + "▁lear n", + "▁ learn", + "▁e xc", + "▁ex c", + "▁ exc", + "▁im p", + "▁i mp", + "▁ imp", + "▁me ans", + "▁mean s", + "▁w ur", + "en ds", + "end s", + "vo id", + "v oid", + "▁s td", + "▁st d", + "▁ std", + "▁part icular", + "▁partic ular", + "▁particul ar", + "▁parti cular", + "j a", + "▁s ource", + "▁sour ce", + "▁ source", + "def ault", + "p y", + "▁a ls", + "▁al s", + "▁ als", + "sc ri", + "scr i", + "s cri", + "st atus", + "stat us", + "▁st ory", + "▁stor y", + "▁sto ry", + "▁ story", + "▁b egin", + "▁be gin", + "▁beg in", + "▁ begin", + "▁pos ition", + "▁posit ion", + "▁ position", + "▁spec ial", + "▁spe cial", + "▁ special", + "ph p", + "p hp", + "▁b ar", + "▁ba r", + "▁ bar", + "▁p ract", + "▁pr act", + "▁pra ct", + "▁prac t", + "cal l", + "ca ll", + "c all", + "▁d as", + "▁da s", + "▁ das", + "▁r ad", + "▁ra d", + "▁ rad", + "▁cl ose", + "▁clos e", + "▁clo se", + "▁ close", + "ww w", + "w ww", + "ер е", + "е ре", + "g u", + "▁E r", + "▁ Er", + "▁d om", + "▁do m", + "▁ dom", + "A M", + "▁b ed", + "▁be d", + "▁ bed", + "▁sever al", + "au l", + "a ul", + "bo x", + "b ox", + "▁l ow", + "▁lo w", + "▁ low", + "pa ck", + "p ack", + "Re g", + "R eg", + "O f", + "at ures", + "ature s", + "atur es", + "atu res", + "é n", + "ed er", + "ede r", + "e der", + "uild er", + "ca st", + "cas t", + "c ast", + "con om", + "co nom", + "c onom", + "ra ft", + "raf t", + "r aft", + "▁m akes", + "▁make s", + "▁ma kes", + "Lo c", + "L oc", + "ht tp", + "htt p", + "h ttp", + "▁a bs", + "▁ab s", + "▁ abs", + "re sh", + "res h", + "r esh", + "▁W ill", + "▁Wil l", + "▁Wi ll", + "▁ Will", + "bre ak", + "b reak", + "▁o ptions", + "▁opt ions", + "▁option s", + "▁ options", + "fo rt", + "for t", + "f ort", + "▁и з", + "▁ из", + "▁a nal", + "▁an al", + "▁ anal", + "▁e nv", + "▁en v", + "▁ env", + "( {", + "ev ent", + "even t", + "eve nt", + "e vent", + "▁p age", + "▁pa ge", + "▁pag e", + "▁ page", + "ter nal", + "tern al", + "▁d istribut", + "▁dist ribut", + "▁f ood", + "▁fo od", + "▁foo d", + "▁ food", + "che ck", + "c heck", + "C K", + "▁в о", + "▁ во", + "as sert", + "ass ert", + "asse rt", + "á n", + "ba se", + "bas e", + "b ase", + "▁w hole", + "▁wh ole", + "▁who le", + "ac ión", + "ació n", + "aci ón", + "a ción", + "O D", + "▁turn ed", + "▁tur ned", + "ig ma", + "▁res ponse", + "▁respon se", + "▁respons e", + "▁ response", + "▁Univers ity", + "▁d iv", + "▁di v", + "▁ div", + "ap ter", + "apt er", + "▁result s", + "▁ results", + "▁re present", + "▁rep resent", + "▁every thing", + "▁C ent", + "▁Ce nt", + "▁ Cent", + "ut es", + "ute s", + "u tes", + "ri x", + "r ix", + "▁S ome", + "▁So me", + "▁Som e", + "▁ Some", + "▁be hind", + "▁beh ind", + "▁c reat", + "▁cre at", + "▁ creat", + "pl ace", + "plac e", + "p lace", + "s u", + "▁P art", + "▁Par t", + "▁Pa rt", + "▁ Part", + "um b", + "u mb", + "math bb", + "pi ng", + "pin g", + "p ing", + "▁m atch", + "▁mat ch", + "▁ match", + "O ut", + "do m", + "d om", + "▁s itu", + "▁sit u", + "▁si tu", + "d r", + "ar a", + "a ra", + "▁w indow", + "▁wind ow", + "▁ window", + "n s", + "lish ed", + "l ished", + "▁V er", + "▁Ve r", + "▁ Ver", + "▁m essage", + "▁mess age", + "▁ message", + "▁E m", + "▁ Em", + "▁h uman", + "▁hum an", + "▁ human", + "per ties", + "pert ies", + "л у", + "le m", + "l em", + "OR T", + "O RT", + "▁e arly", + "▁ear ly", + "▁qu ick", + "▁qui ck", + "▁ quick", + "▁т а", + "▁ та", + "ro id", + "r oid", + "▁c ountry", + "▁coun try", + "▁count ry", + "▁countr y", + "▁ country", + "▁d ue", + "▁du e", + "▁ due", + "▁D ie", + "▁Di e", + "▁ Die", + "▁t rying", + "▁tr ying", + "▁try ing", + "▁l ive", + "▁li ve", + "▁liv e", + "▁ live", + "▁p ress", + "▁pre ss", + "▁pr ess", + "▁pres s", + "▁ press", + "IN T", + "I NT", + "W ith", + "ov ed", + "ove d", + "o ved", + "▁spec ific", + "▁ specific", + "▁f all", + "▁fa ll", + "▁fal l", + "▁ fall", + "u k", + "y l", + "▁gener al", + "▁gen eral", + "▁gene ral", + "▁ general", + "м у", + "н у", + "▁n ames", + "▁name s", + "▁na mes", + "▁nam es", + "▁ names", + "wh ere", + "whe re", + "w here", + "▁The se", + "▁Th ese", + "▁ These", + "▁s il", + "▁si l", + "▁ sil", + "é t", + "▁e ner", + "▁en er", + "▁ ener", + "▁N ow", + "▁No w", + "▁ Now", + "▁add ress", + "▁addr ess", + "▁ address", + "Res ponse", + "▁M r", + "▁ Mr", + "▁an sw", + "▁ans w", + "▁fil m", + "▁fi lm", + "▁ film", + "▁str ong", + "▁stro ng", + "▁ strong", + "▁b ring", + "▁br ing", + "▁Un ited", + "▁Unit ed", + "▁g e", + "▁ ge", + "▁w oman", + "▁wom an", + "▁wo man", + "▁ woman", + "Ne w", + "N ew", + "et t", + "e tt", + ". )", + "en ame", + "ena me", + "e name", + "▁A N", + "▁ AN", + "▁de scrib", + "▁desc rib", + "з а", + "is ing", + "isi ng", + "i sing", + "E L", + "q l", + "▁f ur", + "▁fu r", + "▁ fur", + "y ing", + "▁C al", + "▁Ca l", + "▁ Cal", + "▁D r", + "▁ Dr", + "ER R", + "E RR", + "▁\\ \\", + "▁ \\\\", + "an gle", + "ang le", + "ur ope", + "uro pe", + "urop e", + "▁c ity", + "▁cit y", + "▁ci ty", + "▁ city", + "▁in dex", + "▁ind ex", + "▁inde x", + "▁ index", + "▁a ction", + "▁act ion", + "▁ action", + "▁How ever", + "▁ However", + "▁f ig", + "▁fi g", + "▁ fig", + "ia s", + "i as", + "▁quest ion", + "▁ question", + "▁J an", + "▁Ja n", + "▁ Jan", + "▁M ed", + "▁Me d", + "▁ Med", + "▁C ont", + "▁Con t", + "▁Co nt", + "▁ Cont", + "am ed", + "ame d", + "a med", + "Cal l", + "C all", + "pl ied", + "tt y", + "t ty", + "▁ind ivid", + "pa ge", + "pag e", + "p age", + "▁c omb", + "▁com b", + "▁co mb", + "▁ comb", + "se ction", + "sect ion", + "s ection", + "▁C omm", + "▁Com m", + "▁Co mm", + "▁ Comm", + "ue l", + "u el", + "▁h et", + "▁he t", + "▁ het", + "▁B ar", + "▁Ba r", + "▁ Bar", + "ag ement", + "age ment", + "agem ent", + "fi n", + "f in", + "▁m ajor", + "▁ma jor", + "▁maj or", + "▁ major", + "op er", + "ope r", + "o per", + "ap i", + "a pi", + "ro om", + "r oom", + "▁ „", + "▁h ab", + "▁ha b", + "▁ hab", + "з и", + "▁a uf", + "▁au f", + "▁ auf", + "cur rent", + "curr ent", + "n i", + "▁in clude", + "▁incl ude", + "▁includ e", + "▁inclu de", + "▁ include", + "▁qu i", + "▁q ui", + "v a", + "U E", + "▁ide a", + "▁id ea", + "▁ idea", + ", '", + "▁requ ired", + "▁require d", + "▁ required", + "▁he art", + "▁hear t", + "▁ heart", + "ib ility", + "ibil ity", + "ict ion", + "i ction", + "Mod el", + "Mode l", + "Mo del", + "wr ite", + "writ e", + "w rite", + "▁cont ent", + "▁conten t", + "▁ content", + "▁w er", + "▁we r", + "▁ wer", + "▁h ands", + "▁hand s", + "▁han ds", + "ze n", + "z en", + "ch ar", + "cha r", + "c har", + "}^ {", + "} ^{", + "▁m ass", + "▁ma ss", + "▁mas s", + "▁ mass", + "pl y", + "p ly", + "▁n at", + "▁na t", + "▁ nat", + "re l", + "r el", + "▁d at", + "▁da t", + "▁ dat", + "==== ============", + "======== ========", + "============ ====", + "im al", + "ima l", + "i mal", + "▁pro bably", + "▁prob ably", + "un ch", + "unc h", + "▁m er", + "▁me r", + "▁ mer", + "il ar", + "ila r", + "i lar", + "ir es", + "ire s", + "i res", + "▁w atch", + "▁wat ch", + "▁ watch", + "S I", + "▁c ult", + "▁cu lt", + "▁cul t", + "▁m other", + "▁mot her", + "▁mo ther", + "▁ mother", + "▁govern ment", + "or ding", + "ord ing", + "▁( )", + "▁ ()", + "▁p ri", + "▁pr i", + "▁l ink", + "▁lin k", + "▁ link", + "gr oup", + "gro up", + "g roup", + "O L", + "▁n ear", + "▁ne ar", + "▁S er", + "▁Se r", + "▁ Ser", + "Se r", + "S er", + "it o", + "i to", + "▁value s", + "▁val ues", + "▁ values", + "▁j ava", + "▁ja va", + "▁ java", + "ful ly", + "full y", + "f ully", + "Co unt", + "C ount", + "++ )", + "▁v i", + "▁ vi", + "▁wh ite", + "▁ white", + "ma t", + "m at", + "ct x", + "c tx", + "▁con c", + "▁co nc", + "▁ conc", + "▁st ay", + "▁sta y", + "gi ng", + "gin g", + "g ing", + "▁c lear", + "▁cl ear", + "▁cle ar", + "▁ clear", + "▁c opy", + "▁co py", + "▁cop y", + "▁ copy", + "sel ves", + "▁prov ide", + "▁w ords", + "▁wor ds", + "▁word s", + "▁ words", + "com p", + "co mp", + "c omp", + "ar gs", + "arg s", + "▁p ick", + "▁pi ck", + "▁pic k", + "▁ pick", + "ul y", + "u ly", + "▁v ari", + "▁var i", + "▁va ri", + "▁ vari", + "▁bel ieve", + "▁belie ve", + "▁C o", + "▁ Co", + "Pro perty", + "Gr oup", + "G roup", + "▁t en", + "▁te n", + "▁ ten", + "is chen", + "isch en", + "ische n", + "isc hen", + "i schen", + "et urn", + "e turn", + "iv al", + "iva l", + "i val", + "Sys tem", + "S ystem", + "C L", + "be d", + "b ed", + "▁t otal", + "▁to tal", + "▁tot al", + "▁ total", + "▁is t", + "▁i st", + "▁ ist", + "In put", + "um ents", + "ument s", + "umen ts", + "u ments", + "Man ager", + "ш и", + "▁w in", + "▁ win", + "le ep", + "lee p", + "P I", + "но го", + "н ого", + "ru ction", + "ruct ion", + "r uction", + "▁in te", + "▁i nte", + "▁int e", + "▁ inte", + "Ap p", + "A pp", + "av or", + "avo r", + "a vor", + "▁re spect", + "▁res pect", + "▁resp ect", + "▁ respect", + "at ors", + "ator s", + "ato rs", + "▁c omo", + "▁com o", + "▁co mo", + "▁c ut", + "▁cu t", + "▁ cut", + "F A", + "▁s us", + "▁su s", + "▁A pp", + "▁Ap p", + "▁ App", + "re ct", + "rec t", + "r ect", + "F I", + "▁be gan", + "▁beg an", + "op h", + "o ph", + "▁s ort", + "▁so rt", + "▁sor t", + "▁ sort", + "th ough", + "ј е", + "ic ro", + "i cro", + "Tr ans", + "Tra ns", + "л і", + "▁In st", + "▁Ins t", + "▁ Inst", + "re quest", + "requ est", + "req uest", + "о р", + "▁rel ations", + "▁relation s", + "- \\", + "St atus", + "Stat us", + "ж и", + "▁f ather", + "▁fa ther", + "▁fat her", + "▁ father", + "c s", + "▁s ex", + "▁se x", + "▁ sex", + "is ch", + "isc h", + "i sch", + "v o", + "}_ {", + "} _{", + "ave n", + "av en", + "a ven", + "▁N e", + "▁ Ne", + "AT E", + "A TE", + "it ten", + "itt en", + "itte n", + "▁e ss", + "▁es s", + "▁ ess", + "T H", + "ight s", + "igh ts", + "▁h om", + "▁ho m", + "▁ hom", + "▁t oday", + "▁to day", + "▁tod ay", + "▁toda y", + "▁z u", + "▁ zu", + "it a", + "i ta", + "▁is n", + "▁i sn", + "▁o pt", + "▁op t", + "▁ opt", + "og n", + "o gn", + "é r", + "▁wh ether", + "▁whe ther", + "ix ed", + "ph i", + "p hi", + "id ence", + "iden ce", + "al d", + "a ld", + "Cl ient", + "A t", + "▁de ath", + "▁L et", + "▁Le t", + "▁ Let", + "iu s", + "i us", + "г и", + "▁р е", + "▁ ре", + "be n", + "b en", + ") \r", + "b a", + ">< /", + "> ", + "▁ ->", + "▁J ust", + "▁Ju st", + "▁ Just", + "Wh at", + "W hat", + "at al", + "ata l", + "a tal", + "▁M in", + "▁Mi n", + "▁ Min", + "▁C or", + "▁Co r", + "▁ Cor", + "▁d ark", + "▁dar k", + "▁ dark", + "r l", + "▁l arg", + "▁la rg", + "▁ larg", + "di ng", + "d ing", + "ó n", + "ou ch", + "o uch", + "▁u m", + "▁ um", + "▁e lect", + "▁el ect", + "▁ele ct", + "▁ elect", + "▁d am", + "▁da m", + "▁ dam", + "▁ne eds", + "▁need s", + "▁m atter", + "▁mat ter", + "▁matt er", + "▁r ather", + "▁rat her", + "▁ra ther", + "fr om", + "f rom", + "ra m", + "r am", + "▁ і", + "▁t aken", + "▁take n", + "▁tak en", + "▁ta ken", + "▁de al", + "▁per iod", + "▁ period", + "▁M on", + "▁Mo n", + "▁ Mon", + "▁ Л", + "▁A ug", + "▁Au g", + "▁ Aug", + "ru n", + "r un", + "m m", + "el le", + "ell e", + "e lle", + "▁ex port", + "▁exp ort", + "▁ export", + "S c", + "vi s", + "v is", + "ab or", + "a bor", + "▁aut hor", + "▁auth or", + "▁ author", + "è re", + "▁re member", + "▁rem ember", + "▁remem ber", + "▁re du", + "▁r edu", + "▁red u", + "▁ redu", + "▁L ist", + "▁Li st", + "▁Lis t", + "▁ List", + "▁f ocus", + "▁ focus", + "▁char acter", + "▁ character", + "Tab le", + "T able", + "▁individ ual", + "▁need ed", + "bu m", + "b um", + "▁st yle", + "▁sty le", + "▁ style", + "in ary", + "ina ry", + "inar y", + "ers ion", + "ou te", + "out e", + "o ute", + "▁P e", + "▁ Pe", + "▁h on", + "▁ho n", + "▁ hon", + "mu t", + "m ut", + "se e", + "s ee", + "▁bec ame", + "▁d ire", + "▁di re", + "▁dir e", + "▁ dire", + "▁d ocument", + "▁doc ument", + "▁ document", + "se c", + "s ec", + "en ing", + "eni ng", + "e ning", + "▁vis it", + "▁ visit", + "▁f ac", + "▁fa c", + "▁ fac", + "t x", + "do wn", + "d own", + "pl it", + "p lit", + "▁ph ys", + "▁ phys", + "it ting", + "itt ing", + "jo y", + "j oy", + "▁h ig", + "▁hi g", + "Th is", + "T his", + "A d", + "▁B rit", + "▁Br it", + "▁em ploy", + "▁r é", + "▁ ré", + "▁ т", + "l ambda", + "▁im pro", + "▁imp ro", + "▁B o", + "▁ Bo", + "id ing", + "idi ng", + "i ding", + "▁on line", + "▁ online", + "me m", + "m em", + "at form", + "▁W ar", + "▁Wa r", + "▁ War", + "▁c as", + "▁ca s", + "▁ cas", + "as ure", + "a sure", + "▁p ur", + "▁pu r", + "▁ pur", + "me di", + "med i", + "m edi", + "Di s", + "D is", + "▁G erm", + "▁Ge rm", + "▁Ger m", + "p c", + "с а", + "▁friend s", + "▁M c", + "▁ Mc", + "D I", + "▁pl us", + "▁ plus", + "▁S et", + "▁Se t", + "▁ Set", + "idd le", + "it ut", + "itu t", + "▁de pend", + "▁dep end", + "▁ depend", + "re st", + "res t", + "r est", + "▁J e", + "▁ Je", + "▁h or", + "▁ho r", + "▁ hor", + "▁ent ire", + "Qu ery", + "Que ry", + "▁re fer", + "▁ref er", + "▁ refer", + "▁h ot", + "▁ho t", + "▁ hot", + "▁A ust", + "▁Aus t", + "▁Au st", + "▁com mon", + "▁comm on", + "▁ common", + "ц і", + "▁p ull", + "▁pu ll", + "▁pul l", + "▁ pull", + "▁A dd", + "▁Ad d", + "▁ Add", + "▁se ason", + "▁sea son", + "▁seas on", + "▁ season", + "▁in vol", + "▁inv ol", + "▁W orld", + "▁Wor ld", + "▁ World", + "cl ient", + "cli ent", + "no w", + "n ow", + "tr ue", + "ap pend", + "app end", + "appe nd", + "appen d", + "it ted", + "itt ed", + "itte d", + "em pt", + "emp t", + ") {", + "// /", + "/ //", + "▁p rop", + "▁pro p", + "▁pr op", + "▁ prop", + "im ate", + "ima te", + "imat e", + "i mate", + "S C", + "▁h ours", + "▁hour s", + "▁ho urs", + "▁h ope", + "▁hop e", + "▁ho pe", + "an dom", + "and om", + "ando m", + "і д", + "ist ic", + "isti c", + "▁pro perty", + "▁proper ty", + "▁ property", + "s g", + "> (", + "▁w rite", + "▁wr ite", + "▁writ e", + "▁ write", + "mar k", + "m ark", + "fin d", + "fi nd", + "f ind", + "▁person al", + "▁pers onal", + "▁persona l", + "▁ personal", + "] [", + "ro wn", + "row n", + "r own", + "P h", + "▁f oot", + "▁fo ot", + "▁foo t", + "▁ foot", + "▁re search", + "▁res earch", + "iron ment", + "▁n om", + "▁no m", + "▁ nom", + "▁in stance", + "▁inst ance", + "▁ instance", + "▁h eld", + "▁he ld", + "▁hel d", + "▁ held", + "D e", + "▁mem bers", + "▁member s", + "▁ members", + "▁f ire", + "▁fi re", + "▁fir e", + "▁ fire", + "▁hist ory", + "▁histor y", + "▁hi story", + "▁ history", + "▁m ap", + "▁ma p", + "▁ map", + "▁dis cuss", + "▁disc uss", + "▁e spec", + "▁es pec", + "▁esp ec", + "▁ espec", + "▁t aking", + "▁tak ing", + "▁ta king", + "▁s ervices", + "▁serv ices", + "▁service s", + "▁ services", + "▁ind ust", + "▁indu st", + "▁ indust", + "ig en", + "ige n", + "i gen", + "▁A ss", + "▁As s", + "▁ Ass", + "▁e xpected", + "▁ex pected", + "▁expect ed", + "▁ expected", + "▁wur de", + "di r", + "d ir", + "▁a mong", + "▁am ong", + "▁s ugg", + "▁su gg", + "▁sug g", + "re c", + "r ec", + "In ter", + "Int er", + "bl ock", + "blo ck", + "b lock", + "▁R ep", + "▁Re p", + "▁ Rep", + "▁p ain", + "▁pa in", + "▁f ive", + "▁fi ve", + "▁ five", + "▁f und", + "▁fun d", + "▁fu nd", + "▁ fund", + "ri d", + "r id", + "ar row", + "arr ow", + "▁t reat", + "▁tre at", + "▁he ard", + "▁hear d", + "▁de term", + "▁det erm", + "▁deter m", + "ic ult", + "▁s ense", + "▁sens e", + "▁sen se", + "es e", + "e se", + "F un", + "▁month s", + "▁mont hs", + "js on", + "j son", + ", ”", + "T I", + "or age", + "ora ge", + "o rage", + "▁ У", + "▁every one", + "▁c los", + "▁cl os", + "▁clo s", + "▁ clos", + "ie rs", + "ier s", + "i ers", + "air s", + "ai rs", + "a irs", + "def ine", + "I f", + "os p", + "o sp", + "▁w onder", + "▁won der", + "▁wo nder", + "N A", + "qu ery", + "que ry", + "quer y", + "p g", + "it es", + "ite s", + "i tes", + "▁m aterial", + "▁mat erial", + "▁mate rial", + "▁mater ial", + "▁ material", + "y d", + "Re ad", + "R ead", + "ht ml", + "h tml", + "T E", + "P r", + "^{ \\", + "^ {\\", + "▁g ave", + "▁ga ve", + "▁I S", + "▁ IS", + "▁s uggest", + "▁sugg est", + "▁sug gest", + "Over ride", + "ro du", + "rod u", + "Fr om", + "F rom", + "▁E urope", + "▁Europ e", + "▁Euro pe", + "▁ Europe", + "P O", + "▁s oon", + "▁so on", + "ho st", + "hos t", + "h ost", + "▁B er", + "▁Be r", + "▁ Ber", + ".. ..", + "... .", + ". ...", + "▁H ar", + "▁Ha r", + "▁ Har", + "▁e nergy", + "▁ener gy", + "▁energ y", + "▁ energy", + "> <", + "ave s", + "av es", + "a ves", + "▁e asy", + "▁eas y", + "▁b re", + "▁br e", + "▁ bre", + "fr ame", + "▁g round", + "▁gr ound", + "▁gro und", + "▁ ground", + "wi th", + "w ith", + "▁in side", + "▁ins ide", + "ie f", + "i ef", + "▁m o", + "▁ mo", + "p m", + "pa n", + "p an", + "ig r", + "i gr", + "▁o m", + "▁ om", + "ne xt", + "nex t", + "n ext", + "om et", + "ome t", + "o met", + "▁st atus", + "▁stat us", + "▁ status", + "▁} \r", + "▁ }\r", + "▁mus ic", + "or a", + "o ra", + "il es", + "ile s", + "i les", + "k i", + "▁e sc", + "▁es c", + "▁ esc", + "▁b es", + "▁be s", + "▁ bes", + "▁D is", + "▁Di s", + "▁ Dis", + "▁h ost", + "▁ho st", + "▁ host", + "▁c omes", + "▁com es", + "▁co mes", + "▁come s", + "▁ comes", + "us ed", + "use d", + "u sed", + "▁f uture", + "▁fut ure", + "▁ future", + "lic k", + "li ck", + "l ick", + "ai d", + "a id", + "▁com pet", + "▁comp et", + "▁ compet", + "▁v oice", + "▁vo ice", + "▁ voice", + "▁l oad", + "▁lo ad", + "▁ load", + "ev el", + "eve l", + "e vel", + "▁n eg", + "▁ne g", + "▁ neg", + "▁com mand", + "▁comm and", + "▁ command", + "▁f ür", + "▁p ie", + "▁pi e", + "▁ pie", + "▁qu ite", + "▁qui te", + "▁quit e", + "▁b lo", + "▁bl o", + "▁ blo", + "ag n", + "a gn", + "il on", + "ilo n", + "i lon", + "▁cl aim", + "▁ claim", + "▁t each", + "▁te ach", + "▁tea ch", + "▁pre vious", + "▁prev ious", + "▁ previous", + "▁s ite", + "▁sit e", + "▁si te", + "▁ site", + "co lor", + "col or", + "colo r", + "at tr", + "att r", + "▁ac cept", + "▁ accept", + "▁ex act", + ") }", + "af t", + "a ft", + "rol ler", + "roll er", + "о н", + "o o", + "Dat e", + "Da te", + "D ate", + "▁o u", + "▁ ou", + "s y", + "▁pre tty", + "▁pret ty", + "▁im age", + "▁imag e", + "▁ image", + "B U", + "▁term s", + "▁ter ms", + "▁s earch", + "▁se arch", + "▁sear ch", + "▁ search", + "▁ è", + "▁V al", + "▁Va l", + "▁ Val", + "▁ ‘", + "▁D av", + "▁Da v", + "M S", + "sr c", + "s rc", + "ma r", + "m ar", + "in cip", + "inc ip", + "▁could n", + "ad os", + "ado s", + "▁d ro", + "▁dr o", + "▁ dro", + "be ta", + "bet a", + "b eta", + "im um", + "▁min utes", + "▁minute s", + "▁minut es", + "▁g rand", + "▁gr and", + "▁gran d", + "▁gra nd", + "▁ grand", + "▁ »", + "▁O ur", + "▁ Our", + "St r", + "S tr", + "VE R", + "V ER", + "ma z", + "m az", + "▁or iginal", + "▁orig inal", + "▁origin al", + "▁ original", + "in i", + "i ni", + "▁c oll", + "▁col l", + "▁co ll", + "▁ coll", + "lo at", + "▁o s", + "▁ os", + "}) ;", + "} );", + "sum mary", + "▁w all", + "▁wa ll", + "▁wal l", + "▁ wall", + "Col or", + "Co lor", + "▁v ers", + "▁ver s", + "▁ve rs", + "▁ vers", + "▁d ella", + "▁de lla", + "▁del la", + "▁dell a", + "▁\" \"\"", + "▁\"\" \"", + "▁ \"\"\"", + "math bf", + "ze r", + "z er", + "au r", + "a ur", + "▁tr ack", + "▁tra ck", + "▁ track", + "▁ass oci", + "▁ associ", + "▁s uff", + "▁su ff", + "▁in de", + "▁i nde", + "▁ind e", + "▁ inde", + "ag ue", + "agu e", + "a gue", + "▁A pr", + "▁Ap r", + "▁ Apr", + "L e", + "ro ups", + "rou ps", + "roup s", + "bo ard", + "b oard", + "▁att ack", + "▁s eries", + "▁se ries", + "▁ser ies", + "▁serie s", + "▁ series", + "▁in stead", + "▁inst ead", + "ha m", + "h am", + "bo ok", + "b ook", + "▁s ix", + "▁si x", + "▁ six", + "▁R ec", + "▁Re c", + "▁ Rec", + "▁c oming", + "▁com ing", + "▁co ming", + "▁ coming", + "ur t", + "u rt", + "▁gl obal", + "▁glob al", + "▁glo bal", + "▁ global", + "▁ne cess", + "▁neces s", + "▁ necess", + "le ge", + "leg e", + "Po s", + "P os", + "▁le ave", + "▁ leave", + "▁p od", + "▁po d", + "▁ pod", + "ateg ory", + "ategor y", + "u z", + "▁de ep", + "▁ deep", + "▁k m", + "▁ km", + "▁out side", + "▁outs ide", + "ha s", + "h as", + "opt ions", + "option s", + "o ptions", + "▁S m", + "▁ Sm", + "Su b", + "S ub", + "ro ws", + "row s", + "r ows", + "▁в и", + "▁ ви", + "▁St ates", + "▁State s", + "▁Stat es", + "▁Sta tes", + "▁ States", + "▁wr ong", + "▁how ever", + "▁s em", + "▁se m", + "▁ sem", + "▁c atch", + "▁cat ch", + "▁ catch", + "\") ,", + "\" ),", + "mod el", + "mode l", + "mo del", + "▁h ttp", + "▁htt p", + "▁ http", + "▁o ption", + "▁opt ion", + "▁ option", + "ri e", + "r ie", + "▁с та", + "▁ст а", + "▁ ста", + "▁ä r", + "▁ är", + "▁en joy", + "▁enjo y", + "n u", + "▁p as", + "▁pa s", + "▁ pas", + "▁a mount", + "▁am ount", + "▁ amount", + "▁res pons", + "▁respon s", + "▁resp ons", + "▁ respons", + "▁In tern", + "▁Inter n", + "▁Int ern", + "▁ Intern", + "▁my self", + "▁o pp", + "▁op p", + "▁ opp", + "▁S im", + "▁Si m", + "▁ Sim", + "▁s ens", + "▁se ns", + "▁sen s", + "E d", + "▁( \\", + "▁ (\\", + "▁stud ents", + "▁student s", + "но в", + "н ов", + "▁point s", + "▁ points", + "ar ning", + "arn ing", + "U P", + "el ling", + "ell ing", + "elli ng", + "▁c annot", + "▁can not", + "B e", + "▁l ength", + "▁le ngth", + "▁ length", + "nu ll", + "n ull", + "ui nt", + "u int", + "wi se", + "w ise", + "▁d ouble", + "▁dou ble", + "▁doub le", + "▁ double", + "ig e", + "i ge", + "is ta", + "ist a", + "i sta", + "▁est ab", + "▁es tab", + "▁esta b", + "an ch", + "anc h", + "▁a go", + "▁ag o", + "▁ ago", + "▁b ound", + "▁bo und", + "▁bou nd", + "▁ bound", + "▁f a", + "▁ fa", + "▁c lean", + "▁cle an", + "▁ clean", + "▁sim ple", + "▁simpl e", + "▁ simple", + "m i", + "#### ####", + "if ier", + "ifi er", + "▁Gener al", + "▁Gen eral", + "▁Gene ral", + "▁ General", + "▁se emed", + "▁see med", + "▁seem ed", + "en a", + "e na", + "▁a ge", + "▁ag e", + "▁ age", + "но й", + "end if", + "A A", + "▁c aus", + "▁ca us", + "▁e duc", + "▁ed uc", + "▁ educ", + "▁c ell", + "▁ce ll", + "▁cel l", + "▁ cell", + "Ge ner", + "Gen er", + "G ener", + "sp ace", + "s pace", + "▁Y our", + "▁You r", + "▁ Your", + "▁be aut", + "g t", + "▁l imit", + "▁li mit", + "▁lim it", + "▁ limit", + "▁d ate", + "▁da te", + "▁dat e", + "▁ date", + "Ut il", + "U til", + "▁N ational", + "▁Nat ional", + "▁Nation al", + "▁ National", + "ow s", + "o ws", + "pa t", + "p at", + "qu ad", + "▁o k", + "▁ ok", + "▁ И", + "ar th", + "art h", + "ha t", + "h at", + "▁comm unity", + "▁commun ity", + "ou l", + "o ul", + "▁e conom", + "▁ec onom", + "▁ econom", + "Com ponent", + "bo r", + "b or", + "us ion", + "▁be low", + "▁bel ow", + "ear ch", + "e arch", + "or es", + "ore s", + "o res", + "ba n", + "b an", + "▁Aug ust", + "▁fur ther", + "sig ma", + "s igma", + "▁h a", + "▁ ha", + "j i", + "▁com put", + "▁comp ut", + "▁ comput", + "г ра", + "▁N one", + "▁No ne", + "▁Non e", + "▁ None", + "▁t er", + "▁te r", + "▁ ter", + "▁any one", + "▁t ask", + "▁ta sk", + "▁ task", + "en te", + "ent e", + "e nte", + "pos ition", + "pp ed", + "ppe d", + "p ped", + "▁a us", + "▁au s", + "▁ aus", + "Att ribute", + "Attrib ute", + "re q", + "r eq", + "ad dr", + "add r", + "li ght", + "lig ht", + "l ight", + "ш е", + "▁a rm", + "▁ar m", + "▁ arm", + "co ver", + "cov er", + "c over", + "up port", + "upp ort", + "▁G l", + "▁ Gl", + "▁S an", + "▁Sa n", + "▁ San", + "▁wr iting", + "▁writ ing", + "▁ writing", + "▁l ost", + "▁lo st", + "▁los t", + "▁M ark", + "▁Mar k", + "▁ Mark", + "▁g re", + "▁gr e", + "▁ gre", + "TY PE", + "T YPE", + "▁S outh", + "▁So uth", + "▁Sou th", + "▁Sout h", + "▁ South", + "▁per fect", + "▁perf ect", + "▁pack age", + "▁ package", + "▁in fl", + "▁inf l", + "▁ infl", + "ha ps", + "h aps", + "▁A ng", + "▁An g", + "▁ Ang", + "res pon", + "resp on", + "ri s", + "r is", + "pt ember", + "pte mber", + "▁build ing", + "▁ building", + "VA L", + "V AL", + "fr ee", + "fre e", + "f ree", + "▁c e", + "▁ ce", + "H T", + "▁F rom", + "▁Fr om", + "▁Fro m", + "▁ From", + "d s", + "ro y", + "r oy", + "ach ine", + "achi ne", + "no wn", + "now n", + "n own", + "▁sa ying", + "▁say ing", + "▁б ы", + "▁ бы", + "o e", + "Re f", + "R ef", + "▁net work", + "▁ network", + "par ent", + "pa rent", + "pare nt", + "paren t", + "p arent", + "ug e", + "u ge", + "▁sim ilar", + "> \r", + "Build er", + "B uilder", + "▁l iving", + "▁li ving", + "▁liv ing", + "▁contin ue", + "▁continu e", + "▁ continue", + "an ger", + "ang er", + "ange r", + "▁R ed", + "▁Re d", + "▁ Red", + "▁h air", + "▁ha ir", + "an ced", + "ance d", + "anc ed", + "ia ns", + "ian s", + "i ans", + "▁d ead", + "▁de ad", + "▁ dead", + "▁bo olean", + "▁ boolean", + "ic ation", + "▁д е", + "▁ де", + "▁cl ient", + "▁ client", + "uc t", + "u ct", + "▁ •", + "S P", + "ol der", + "old er", + "п е", + "ud io", + "udi o", + "▁d eg", + "▁de g", + "▁ deg", + "as ing", + "asi ng", + "a sing", + "▁st ep", + "▁ste p", + "▁ step", + "▁p ers", + "▁per s", + "▁pe rs", + "▁ pers", + "ç ão", + "ob j", + "o z", + "ul a", + "u la", + "▁r ound", + "▁ro und", + "▁rou nd", + "▁ round", + "▁u pon", + "▁up on", + "▁re source", + "▁res ource", + "▁ resource", + "▁val id", + "▁ valid", + "▁I I", + "▁ II", + "bu g", + "b ug", + "st d", + "s td", + "▁a ng", + "▁an g", + "▁ ang", + "sp an", + "s pan", + "po l", + "p ol", + "ial og", + "ia log", + "▁p hot", + "▁ph ot", + "? '", + "D B", + "▁F in", + "▁Fi n", + "▁ Fin", + "V E", + "E m", + "▁c am", + "▁ca m", + "▁ cam", + "tar get", + "t arget", + "pe cted", + "pect ed", + "pec ted", + "He l", + "H el", + "▁u t", + "▁ ut", + "▁T est", + "▁Te st", + "▁Tes t", + "▁ Test", + "▁t own", + "▁to wn", + "▁tow n", + "▁ town", + "al ign", + "ali gn", + "▁we bs", + "▁web s", + "in ner", + "inn er", + "au gh", + "aug h", + "a ugh", + "▁ex cept", + "▁ except", + "▁init ial", + "▁initi al", + "▁ initial", + "en ty", + "ent y", + "lic h", + "li ch", + "l ich", + "▁A ut", + "▁Au t", + "▁ Aut", + "to p", + "t op", + "▁f ail", + "▁fa il", + "▁ fail", + "on a", + "o na", + "▁ben ef", + "an ks", + "ank s", + "is che", + "isch e", + "isc he", + "i sche", + ". *", + "▁sign ific", + "▁cont act", + "▁ contact", + "Re c", + "R ec", + "ar io", + "ari o", + "a rio", + "ot tom", + "ott om", + "otto m", + "▁rel ationship", + "▁relations hip", + "▁relation ship", + "]) ;", + "] );", + "▁Н а", + "▁ На", + "He ad", + "H ead", + "form at", + "for mat", + "▁é t", + "▁ ét", + "▁M ore", + "▁Mor e", + "▁Mo re", + "▁ More", + "act ory", + "actor y", + "port un", + "+ \\", + "▁sim ply", + "▁simpl y", + "▁e p", + "▁ ep", + "▁R uss", + "▁Ru ss", + "▁Rus s", + "n í", + "u a", + "er c", + "e rc", + "▁long er", + "▁lon ger", + "in ition", + "init ion", + "ect or", + "ec tor", + "e ctor", + "apt ion", + "a ption", + "▁prof ess", + "▁profes s", + "▁M us", + "▁Mu s", + "▁ Mus", + "il ities", + "ili ties", + "è s", + "▁A ct", + "▁Ac t", + "▁ Act", + "off set", + "offs et", + "▁i ll", + "▁il l", + "▁ ill", + "ba nd", + "ban d", + "b and", + "▁A g", + "▁ Ag", + "▁П о", + "▁ По", + "б и", + "cont ent", + "ic on", + "ico n", + "i con", + "▁work s", + "▁wor ks", + "▁ works", + "yn am", + "yna m", + "y nam", + "pl ement", + "ple ment", + "p lement", + "Res ource", + "Re source", + "Act ion", + "A ction", + "▁diff icult", + "▁W est", + "▁We st", + "▁Wes t", + "▁ West", + "▁v ideo", + "▁vide o", + "▁ video", + "▁T HE", + "▁TH E", + "▁ THE", + "▁de cl", + "▁dec l", + "▁ decl", + "on don", + "ond on", + "ondo n", + "de d", + "d ed", + "}{ \\", + "} {\\", + "oc r", + "o cr", + "▁C ity", + "▁Cit y", + "▁Ci ty", + "▁ City", + "▁ я", + "ue r", + "u er", + "c z", + "▁im ag", + "▁i mag", + "▁ imag", + "c r", + "et e", + "e te", + "id get", + "idge t", + "▁M od", + "▁Mo d", + "▁ Mod", + "▁for ward", + "▁ forward", + "▁p ict", + "▁pi ct", + "▁pic t", + "or ge", + "org e", + "▁sub ject", + "▁ subject", + "up date", + "at tle", + "att le", + "s a", + "▁A nt", + "▁An t", + "▁ Ant", + "▁r unning", + "▁run ning", + "▁ running", + "▁s al", + "▁sa l", + "▁ sal", + "con ne", + "conn e", + "c onne", + "▁out put", + "▁ output", + "ad ata", + "ada ta", + "a data", + "M L", + "Che ck", + "C heck", + "led ge", + "l edge", + "▁p aper", + "▁pa per", + "▁pap er", + "▁ paper", + "param s", + "par ams", + "para ms", + "av y", + "a vy", + "▁a f", + "▁ af", + "▁e ine", + "▁ein e", + "▁j our", + "▁jo ur", + "▁jou r", + "▁ jour", + "A Y", + "▁it self", + "▁its elf", + "▁S tr", + "▁St r", + "▁ Str", + "st yle", + "sty le", + "Th at", + "T hat", + "▁m illion", + "▁mill ion", + "▁l anguage", + "▁ language", + "O S", + "vi ng", + "vin g", + "v ing", + "▁м а", + "▁ ма", + "▁т о", + "▁ то", + ") (", + "▁b uy", + "▁bu y", + ". /", + "▁. ..", + "▁.. .", + "▁ ...", + "▁t ried", + "▁tr ied", + "▁tri ed", + "▁com pl", + "▁comp l", + "▁act iv", + "▁ activ", + "ap ped", + "app ed", + "appe d", + "a pped", + "But ton", + "B utton", + "To ken", + "Tok en", + "T oken", + "▁prov ided", + "▁provide d", + "ib er", + "ibe r", + "i ber", + "▁c reated", + "▁cre ated", + "▁create d", + "▁creat ed", + "▁ created", + "cur ity", + "c urity", + "En d", + "E nd", + "a ł", + "us ter", + "ust er", + "u ster", + "iz ing", + "izi ng", + "i zing", + "om b", + "o mb", + "▁s ich", + "▁si ch", + "▁com pon", + "▁comp on", + "▁S ee", + "▁Se e", + "▁ See", + "▁u int", + "▁ui nt", + "▁ uint", + "▁l abel", + "▁la bel", + "▁lab el", + "▁ label", + "vo l", + "v ol", + "ó w", + "oc ol", + "oco l", + "o col", + "▁re ceived", + "▁rece ived", + "▁receive d", + "▁in tern", + "▁int ern", + "▁inter n", + "▁inte rn", + "▁ intern", + "ц е", + "R un", + "▁r oad", + "▁ro ad", + "▁ road", + "▁O ct", + "▁ Oct", + "▁C omp", + "▁Com p", + "▁Co mp", + "▁ Comp", + "▁stud y", + "▁т е", + "▁ те", + "Ac t", + "A ct", + "▁t our", + "▁to ur", + "▁tou r", + "▁St ate", + "▁Stat e", + "▁Sta te", + "▁ State", + "▁ad ded", + "▁add ed", + "▁ added", + "htt ps", + "http s", + "st ream", + "stre am", + "▁l ower", + "▁lo wer", + "▁low er", + "▁ lower", + "▁b ox", + "▁bo x", + "▁ box", + "▁S k", + "▁ Sk", + "▁them selves", + "▁c ross", + "▁cr oss", + "▁cro ss", + "▁ cross", + "▁e cho", + "▁ec ho", + "▁ echo", + "▁dev ice", + "▁ device", + "pos e", + "po se", + "p ose", + "▁g ames", + "▁game s", + "▁gam es", + "▁ga mes", + "P L", + "W indow", + "is es", + "ise s", + "i ses", + "ti tle", + "tit le", + "t itle", + "St ream", + "z t", + "▁S w", + "▁ Sw", + "▁r ole", + "▁ro le", + "▁ role", + "ia nt", + "ian t", + "i ant", + "k u", + "se qu", + "seq u", + "s equ", + "▁l ate", + "▁la te", + "▁lat e", + "▁ late", + "▁s old", + "▁so ld", + "▁sol d", + "р я", + "Com m", + "Co mm", + "C omm", + "▁en tre", + "▁ent re", + "▁entr e", + "▁ entre", + "▁d og", + "▁do g", + "▁ dog", + "dev ice", + "P ar", + "▁like ly", + "▁lik ely", + "▁ likely", + "^{ -", + "^ {-", + "▁l en", + "▁le n", + "▁ len", + "▁P aul", + "▁Pa ul", + "▁ Paul", + "▁t ool", + "▁to ol", + "▁too l", + "▁ tool", + "Of f", + "O ff", + "▁f amil", + "▁fam il", + "▁fa mil", + "▁d raw", + "▁dr aw", + "▁ draw", + "ap ping", + "app ing", + "a pping", + "▁ev ents", + "▁even ts", + "▁event s", + "▁ events", + "cre t", + "cr et", + "c ret", + "rou ght", + "rough t", + "r ought", + "Cont ent", + "▁soft ware", + "ri a", + "r ia", + "ms g", + "m sg", + "ga mma", + "g amma", + "▁h ear", + "▁he ar", + "Op er", + "O per", + "▁your self", + "▁yours elf", + "▁l iter", + "▁li ter", + "▁lit er", + "▁ liter", + "em p", + "e mp", + "▁se par", + "▁sep ar", + "▁ separ", + "▁ З", + "▁t itle", + "▁tit le", + "▁ti tle", + "▁ title", + "M ethod", + "math rm", + "▁s low", + "▁sl ow", + "▁R om", + "▁Ro m", + "▁ Rom", + "! !", + "▁t ax", + "▁ta x", + "▁ tax", + "ск а", + "с ка", + "empl ate", + "emp late", + "o i", + "▁A rt", + "▁Ar t", + "▁ Art", + "f alse", + "ast ic", + "ст ь", + "с ть", + "oc ket", + "ock et", + "▁e ns", + "▁en s", + "▁ ens", + "T O", + "am ente", + "ame nte", + "ament e", + "amen te", + "a mente", + "lo cal", + "loc al", + "l ocal", + "ch ie", + "chi e", + "▁p an", + "▁pa n", + "▁ pan", + "ни й", + "ch ema", + "che ma", + "chem a", + "▁N orth", + "▁Nor th", + "▁Nort h", + "з о", + "▁> =", + "▁ >=", + "A ut", + "▁d ig", + "▁di g", + "▁ dig", + "▁se ems", + "▁see ms", + "▁seem s", + "▁mor ning", + "so le", + "sol e", + "s ole", + "um er", + "ume r", + "u mer", + "del ta", + "d elta", + "it é", + "i té", + "ab ase", + "aba se", + "a base", + "ra f", + "r af", + "▁ob serv", + "▁obs erv", + "▁ observ", + "▁E st", + "▁Es t", + "▁ Est", + "▁s eg", + "▁se g", + "▁ seg", + "▁[ ]", + "▁ []", + "▁P res", + "▁Pr es", + "▁Pre s", + "▁ Pres", + "if ul", + "i ful", + "pu sh", + "pus h", + "p ush", + "▁O ff", + "▁Of f", + "▁ Off", + "ip e", + "i pe", + "at i", + "a ti", + "▁d im", + "▁di m", + "▁ dim", + "ce ed", + "c eed", + "En t", + "E nt", + "__ __", + "___ _", + "_ ___", + "en try", + "ent ry", + "entr y", + "▁f ight", + "▁fig ht", + "▁fi ght", + "▁c red", + "▁cre d", + "▁cr ed", + "▁ cred", + "▁O R", + "▁ OR", + "▁D ep", + "▁De p", + "▁ Dep", + "$ {", + "ле н", + "л ен", + "Creat e", + "C reate", + "▁Apr il", + "▁Ap ril", + "min istr", + "F L", + "▁A p", + "▁ Ap", + "▁H ere", + "▁He re", + "▁Her e", + "▁ Here", + "priv ate", + "p rivate", + "In stance", + "Inst ance", + "ie m", + "i em", + "▁off ice", + "▁offic e", + "▁th ird", + "▁ third", + "▁up date", + "▁ update", + "Lin e", + "Li ne", + "L ine", + "ta g", + "t ag", + "▁e specially", + "▁espec ially", + "▁especial ly", + "▁ especially", + "▁го да", + "▁год а", + "▁c u", + "▁ cu", + "▁k ill", + "▁kil l", + "▁ki ll", + "▁ kill", + "au ght", + "augh t", + "aug ht", + "▁s we", + "▁sw e", + "Option s", + "Opt ions", + "O ptions", + "I M", + "C C", + "▁com pan", + "▁comp an", + "ju st", + "j ust", + "▁Wh ile", + "▁ While", + "iz er", + "ize r", + "i zer", + "▁м о", + "▁ мо", + "к е", + "▁a uto", + "▁aut o", + "▁au to", + "▁ auto", + "▁b and", + "▁ban d", + "▁ba nd", + "▁ band", + "ме н", + "м ен", + "ique s", + "iqu es", + "iq ues", + "i ques", + "▁p le", + "▁pl e", + "▁ ple", + "N O", + "▁O F", + "▁ OF", + "▁s ong", + "▁so ng", + "▁son g", + "▁A cc", + "▁Ac c", + "▁ Acc", + "EX T", + "E XT", + "en sor", + "ens or", + "enso r", + "in ing", + "ini ng", + "i ning", + "▁l at", + "▁la t", + "▁ lat", + "bi g", + "b ig", + "▁K ing", + "▁Ki ng", + "▁Kin g", + "▁ King", + "oc h", + "o ch", + "s i", + "▁H ist", + "▁His t", + "▁Hi st", + "▁ Hist", + "▁qu ality", + "▁qual ity", + "▁ quality", + "mod e", + "mo de", + "m ode", + "▁op portun", + "▁would n", + ":* *", + ": **", + "out put", + "▁fe et", + "▁fee t", + "▁m is", + "▁mi s", + "d f", + "ag ing", + "agi ng", + "a ging", + "▁м е", + "▁ ме", + "▁t ro", + "▁tr o", + "▁d efined", + "▁def ined", + "▁define d", + "▁defin ed", + "▁ defined", + "▁re view", + "▁rev iew", + "▁ review", + "▁F il", + "▁Fi l", + "▁ Fil", + "> >", + "▁pr incip", + "▁prin cip", + "Bas e", + "B ase", + "di ct", + "d ict", + "ve rage", + "ver age", + "ic ient", + "ici ent", + "I F", + "▁h it", + "▁hi t", + "▁ hit", + "Pag e", + "P age", + "▁p erm", + "▁per m", + "▁pe rm", + "▁ perm", + "ce l", + "c el", + "í t", + "▁ex press", + "▁exp ress", + "▁expr ess", + "▁ express", + "▁ind ic", + "▁Se ptember", + "▁Sept ember", + "im age", + "ima ge", + "imag e", + "▁product s", + "▁ products", + "▁m edia", + "▁med ia", + "▁medi a", + "▁ media", + "ch ange", + "chan ge", + "ig ger", + "igg er", + "▁s end", + "▁se nd", + "▁sen d", + "▁ send", + "la st", + "las t", + "l ast", + "min g", + "mi ng", + "m ing", + "p a", + "ua ry", + "uar y", + "u ary", + "▁spe ak", + "ны й", + "щ е", + "ys is", + "y sis", + "ly ing", + "l ying", + "▁ ч", + "li ke", + "lik e", + "l ike", + "р ы", + "в і", + "▁M ich", + "▁Mic h", + "▁Mi ch", + "M O", + "▁J ah", + "▁Ja h", + "ens ive", + "▁sh are", + "▁shar e", + "▁sha re", + "▁ share", + "▁develop ment", + "C P", + "sp ec", + "spe c", + "s pec", + "▁f ast", + "▁fa st", + "▁ fast", + "he t", + "h et", + "H O", + "▁part icip", + "▁partic ip", + "▁parti cip", + "Bl ock", + "Blo ck", + "B lock", + "▁vi ol", + "▁fr ame", + "▁fra me", + "▁fram e", + "▁ frame", + "▁qu al", + "▁q ual", + "▁ qual", + "tr e", + "t re", + "▁ Ф", + "▁to ward", + "▁tow ard", + "f g", + "Bo x", + "B ox", + "Col umn", + "▁mil it", + "▁mi lit", + "▁M arch", + "▁Mar ch", + "▁Marc h", + "▁var ious", + "▁vari ous", + "pa ss", + "pas s", + "p ass", + "▁P ark", + "▁Par k", + "▁B en", + "▁Be n", + "▁ Ben", + "Fr ame", + "▁n ormal", + "▁nor mal", + "▁norm al", + "▁ normal", + "op en", + "ope n", + "o pen", + "p x", + "▁ph one", + "▁ phone", + "▁E ven", + "▁Ev en", + "▁Eve n", + "▁ Even", + "▁m a", + "▁ ma", + "ibr ary", + "St art", + "Star t", + "id den", + "idd en", + "rh o", + "r ho", + "gr aph", + "gra ph", + "g raph", + "ac ing", + "aci ng", + "a cing", + "' .", + "ar ter", + "art er", + "arte r", + "me s", + "m es", + "in st", + "ins t", + "▁i r", + "▁ ir", + "act ive", + "activ e", + "▁f em", + "▁fe m", + "▁ fem", + "▁m oved", + "▁mov ed", + "▁move d", + "▁mo ved", + "▁st ore", + "▁stor e", + "▁sto re", + "▁ store", + "▁p rice", + "▁pr ice", + "▁pri ce", + "▁ price", + "\") .", + "\" ).", + "ber g", + "be rg", + "b erg", + "▁n ov", + "▁no v", + "▁ nov", + "▁c ard", + "▁car d", + "▁ca rd", + "▁ card", + "el low", + "ell ow", + "ello w", + "▁part y", + "▁par ty", + "▁ party", + "▁M or", + "▁Mo r", + "ae l", + "a el", + "▁per cent", + "▁ percent", + "▁tr aining", + "▁tra ining", + "▁train ing", + "▁ training", + "▁in g", + "▁i ng", + "▁ ing", + "im er", + "ime r", + "i mer", + "▁S am", + "▁Sa m", + "▁ Sam", + "Def ault", + "▁f uck", + "▁fu ck", + "▁com plete", + "▁comp lete", + "▁complet e", + "▁compl ete", + "▁ complete", + "ui d", + "u id", + "▁det ails", + "▁detail s", + "▁ details", + "▁l ed", + "▁le d", + "▁ led", + "Po int", + "P oint", + "▁C ount", + "▁Co unt", + "▁Coun t", + "▁Cou nt", + "▁ Count", + "▁reg ard", + "z o", + "▁B ro", + "▁Br o", + "▁ Bro", + "▁rec ogn", + "▁ recogn", + "▁H ol", + "▁Ho l", + "▁ Hol", + "U M", + "el ement", + "ele ment", + "elem ent", + "e lement", + "Mod e", + "Mo de", + "M ode", + "▁ex am", + "▁E X", + "▁ EX", + "Im age", + "ver se", + "vers e", + "ri ter", + "rit er", + "rite r", + "r iter", + "so ft", + "s oft", + "▁int rodu", + "▁intro du", + "▁sur pr", + "Buf fer", + "Buff er", + "B uffer", + "le ctor", + "lect or", + "l ector", + "ar en", + "are n", + "a ren", + "an ged", + "ang ed", + "ange d", + "▁P at", + "▁Pa t", + "▁ Pat", + "▁P al", + "▁Pa l", + "▁ Pal", + "▁con tr", + "▁cont r", + "▁ contr", + "Hand ler", + "Handle r", + "▁fe atures", + "▁feature s", + "▁feat ures", + "▁ features", + "ip le", + "i ple", + "▁C ON", + "▁CO N", + "▁ CON", + "Fi l", + "F il", + "▁P ort", + "▁Po rt", + "▁Por t", + "▁ Port", + "▁th inking", + "▁think ing", + "▁thin king", + "do c", + "d oc", + "we r", + "w er", + "▁work ed", + "▁wor ked", + "P C", + "c m", + "da t", + "d at", + "PR O", + "P RO", + "▁E very", + "▁Ev ery", + "▁Ever y", + "▁Eve ry", + "▁ Every", + "▁e ra", + "▁er a", + "▁ era", + "▁F irst", + "▁ First", + "g n", + "▁im medi", + "▁imm edi", + "ov ember", + "ove mber", + "ap an", + "apa n", + "a pan", + "▁ex tra", + "▁ext ra", + "▁extr a", + "▁ extra", + "▁s ection", + "▁se ction", + "▁sect ion", + "▁ section", + "▁J une", + "▁Jun e", + "▁Ju ne", + "▁v ia", + "▁vi a", + "▁ via", + "▁g one", + "▁go ne", + "com e", + "co me", + "c ome", + "▁s tri", + "▁st ri", + "▁str i", + "▁ stri", + "^ \\", + "ant ly", + "▁ar ch", + "▁arc h", + "▁ arch", + "S ource", + "▁con v", + "▁co nv", + "▁ conv", + "▁L ondon", + "▁Lond on", + "▁ London", + "Num ber", + "N umber", + "▁quest ions", + "▁question s", + "an did", + "and id", + "▁play ed", + "en v", + "e nv", + "▁Sch ool", + "▁nat ural", + "▁natur al", + "▁ natural", + "ca n", + "c an", + "▁ne ws", + "▁new s", + "▁ news", + "D R", + "▁c hall", + "▁ch all", + "▁cha ll", + "▁S oc", + "▁So c", + "▁ э", + "▁att empt", + "* }", + "N ull", + "ro te", + "rot e", + "r ote", + "▁b i", + "▁ bi", + "▁wr itten", + "▁writ ten", + "▁ written", + "▁bl ood", + "▁blo od", + "▁happ ened", + "▁happen ed", + "▁c ause", + "▁caus e", + "▁ca use", + "as hing", + "ash ing", + "ashi ng", + "▁Will iam", + "ad em", + "ade m", + "a dem", + "▁b rought", + "▁br ought", + "▁dis play", + "▁displ ay", + "▁disp lay", + "▁ display", + "im a", + "i ma", + "▁fin ally", + "▁final ly", + "ta b", + "t ab", + "▁return ed", + "ны х", + "ni e", + "n ie", + "▁ q", + "▁h ers", + "▁he rs", + "▁her s", + "▁P re", + "▁Pr e", + "▁ Pre", + "▁d ou", + "▁do u", + "buf fer", + "buff er", + "b uffer", + "▁eff ort", + "ain e", + "ai ne", + "a ine", + "x y", + "▁his tor", + "▁hist or", + "en u", + "e nu", + "▁ar riv", + "▁arr iv", + "▁D em", + "▁De m", + "▁ Dem", + "▁f avor", + "▁fa vor", + "▁fav or", + "▁hand le", + "▁ handle", + "SE T", + "S ET", + "▁P ublic", + "▁Pub lic", + "▁Pu blic", + "▁ Public", + "ru pt", + "rup t", + "r upt", + "▁u r", + "▁ ur", + "▁for ce", + "▁ force", + "▁é s", + "▁ és", + "ub e", + "u be", + "Pr e", + "P re", + "р і", + "in y", + "i ny", + "th eta", + "the ta", + "is f", + "i sf", + "▁n ational", + "▁nat ional", + "▁nation al", + "Equ al", + "Eq ual", + "E qual", + "ren ch", + "▁w ife", + "▁c apt", + "▁cap t", + "▁ca pt", + "▁In ter", + "▁Int er", + "▁ Inter", + "ta u", + "t au", + "▁s leep", + "▁sle ep", + "▁ sleep", + "../ ../", + "▁iss ue", + "▁ issue", + "▁m ember", + "▁me mber", + "▁mem ber", + "▁ member", + "▁a wait", + "▁aw ait", + "▁ await", + "▁D an", + "▁Da n", + "▁ Dan", + "z i", + "in ate", + "ina te", + "i nate", + "▁s ym", + "▁sy m", + "▁ sym", + "ch an", + "cha n", + "c han", + "▁J ack", + "▁Jac k", + "▁Ja ck", + "▁ Jack", + "▁Eng lish", + "▁ English", + "▁s z", + "▁ sz", + "rib utes", + "ribut es", + "ribute s", + "ribu tes", + "▁i gn", + "▁ig n", + "▁ ign", + "á l", + "▁app ear", + "▁appe ar", + "ra d", + "r ad", + "id ge", + "▁co uple", + "▁cou ple", + "▁coup le", + "▁s hip", + "▁sh ip", + "▁ ship", + "li g", + "l ig", + "we b", + "w eb", + "▁us ually", + "▁usual ly", + "▁re ady", + "▁read y", + "▁ ready", + "▁v ill", + "▁vi ll", + "▁vil l", + "▁W hy", + "▁Wh y", + "▁ Why", + "eb ru", + "e bru", + "▁g rad", + "▁gr ad", + "▁gra d", + "▁ grad", + "or ds", + "ord s", + "▁in f", + "▁i nf", + "▁ inf", + "▁l oss", + "▁lo ss", + "▁los s", + "▁ loss", + "▁o d", + "▁ od", + "▁Ph il", + "▁ Phil", + "ser ver", + "serv er", + "serve r", + "▁U p", + "▁ Up", + "▁b uff", + "▁bu ff", + "▁buf f", + "▁ buff", + "▁fil ename", + "▁file name", + "▁ filename", + "AB LE", + "it ing", + "iti ng", + "i ting", + "ef ore", + "e fore", + "() ->", + "( )->", + "▁cond itions", + "▁condition s", + "▁ conditions", + "v m", + "el d", + "e ld", + "it z", + "i tz", + "▁Tr ans", + "▁Tra ns", + "▁ Trans", + "▁w eight", + "▁we ight", + "▁weigh t", + "▁ weight", + "▁high er", + "▁hig her", + "▁r ate", + "▁rat e", + "▁ra te", + "▁ rate", + "▁acc om", + "▁ac com", + "vi der", + "vid er", + "v ider", + "O M", + "▁w ays", + "▁way s", + "▁wa ys", + "▁ ways", + "com ing", + "co ming", + "c oming", + "▁l ock", + "▁loc k", + "▁lo ck", + "▁ lock", + "▁e tc", + "▁et c", + "▁ etc", + "▁a vec", + "▁av ec", + "▁ave c", + "▁t akes", + "▁take s", + "▁tak es", + "▁ta kes", + "▁C har", + "▁Ch ar", + "▁Cha r", + "▁ Char", + "▁N ovember", + "▁Nov ember", + "m ethod", + "▁A ustral", + "▁Aust ral", + "▁ Austral", + "▁Amer ica", + "▁ America", + "lo ng", + "lon g", + "l ong", + "ce mber", + "c ember", + "▁polit ical", + "fl ow", + "f low", + "▁may be", + "▁ maybe", + "▁a mb", + "▁am b", + "▁ amb", + "La yout", + "L ayout", + "il ed", + "ile d", + "i led", + "om en", + "ome n", + "o men", + "ol a", + "o la", + "ic ip", + "ici p", + "i cip", + "part ial", + "Tr ue", + "▁f loor", + "▁fl oor", + "▁flo or", + "▁ floor", + "▁D ef", + "▁De f", + "▁ Def", + "▁conc ern", + "▁conce rn", + "▁concer n", + "y r", + "▁sh ows", + "▁show s", + "i h", + "▁an swer", + "▁answ er", + "▁ans wer", + "▁ answer", + "ac c", + "a cc", + "▁b all", + "▁bal l", + "▁ba ll", + "▁ ball", + "▁R ev", + "▁Re v", + "▁ Rev", + "▁s un", + "▁su n", + "▁ sun", + "▁quick ly", + "▁s omet", + "▁so met", + "▁some t", + "▁som et", + "ment e", + "me nte", + "men te", + "m ente", + "▁M al", + "▁Ma l", + "▁ Mal", + "und red", + "▁iss ues", + "▁issue s", + "▁ issues", + "ec ause", + "eca use", + "pe s", + "p es", + "▁p layer", + "▁pl ayer", + "▁play er", + "▁ player", + "▁par ents", + "▁parent s", + "▁ parents", + "▁pop ular", + "▁popula r", + "▁popul ar", + "▁m ode", + "▁mod e", + "▁mo de", + "▁ mode", + "▁m ention", + "▁ment ion", + "N E", + "Lo ad", + "L oad", + "▁reg ular", + "▁regul ar", + "▁ regular", + "ave d", + "av ed", + "a ved", + "? :", + "ye ar", + "y ear", + "fun c", + "fu nc", + "f unc", + "▁per formance", + "▁perform ance", + "▁J uly", + "▁Jul y", + "▁Ju ly", + "th ern", + "ther n", + "the rn", + "▁we bsite", + "▁webs ite", + "▁web site", + "fo rd", + "for d", + "f ord", + "P R", + "el a", + "e la", + "le vel", + "lev el", + "l evel", + "ui t", + "u it", + "fl ags", + "flag s", + "▁w orth", + "▁wor th", + "▁ worth", + "▁cor respon", + "▁Brit ish", + "si m", + "s im", + "▁al one", + "▁ alone", + "▁h ar", + "▁ha r", + "▁ har", + "▁o nes", + "▁on es", + "▁one s", + "▁ ones", + "ob ile", + "obi le", + "obil e", + "▁d ru", + "▁dr u", + "▁ dru", + "ch i", + "c hi", + "▁D avid", + "▁Dav id", + "▁Da vid", + "▁ David", + "▁proble ms", + "▁problem s", + "▁col umn", + "▁ column", + "() ;\r", + "(); \r", + "( );\r", + "Z E", + "▁re lig", + "▁rel ig", + "▁reli g", + "olog ical", + "▁reg ion", + "▁ region", + "ad y", + "a dy", + "I O", + "an der", + "and er", + "ande r", + "a nder", + "Ne t", + "N et", + "▁bu ilt", + "▁ built", + "▁inst all", + "▁ install", + "▁appro ach", + "C ur", + "▁f ine", + "▁fin e", + "▁fi ne", + "▁talk ing", + "▁tal king", + "▁ch anges", + "▁chang es", + "▁change s", + "▁ changes", + "St yle", + "▁M art", + "▁Mar t", + "▁Ma rt", + "▁ Mart", + "л ю", + "res ponse", + "respon se", + "respons e", + "te ger", + "{ \r", + "ir it", + "iri t", + "i rit", + "▁prote cted", + "▁protect ed", + "▁ protected", + "▁re le", + "▁r ele", + "▁rel e", + "er ship", + "ers hip", + "те ль", + "тел ь", + "un signed", + "uns igned", + "ial ize", + "▁htt ps", + "▁http s", + "▁ https", + "T ag", + "▁$ (", + "▁ $(", + "mo re", + "mor e", + "m ore", + "ype s", + "yp es", + "y pes", + "▁st ream", + "▁stre am", + "▁ stream", + "et ch", + "etc h", + "▁eng ine", + "▁ engine", + "K E", + "cm d", + "c md", + "sc ript", + "scri pt", + "scr ipt", + "s cript", + "tt p", + "t tp", + "▁a void", + "▁av oid", + "▁t err", + "▁te rr", + "▁ter r", + "▁r ock", + "▁ro ck", + "▁ rock", + "▁f ul", + "▁fu l", + "▁ ful", + "Up date", + "▁env ironment", + "▁environ ment", + "▁ environment", + "▁p rec", + "▁pre c", + "▁pr ec", + "▁ prec", + "▁с а", + "▁ са", + "▁c ases", + "▁case s", + "▁cas es", + "▁ca ses", + "▁ cases", + "▁off set", + "▁ offset", + "▁r ais", + "▁ra is", + "▁ rais", + "li b", + "l ib", + "ée s", + "é es", + "a a", + "y t", + "▁a rr", + "▁ar r", + "▁ arr", + "opy right", + "f irst", + "▁u til", + "▁ut il", + "▁ util", + "▁fe ature", + "▁feat ure", + "▁ feature", + "pos ed", + "po sed", + "pose d", + "p osed", + "ff ect", + "f fect", + "ж а", + "it ude", + "itu de", + "itud e", + "em ents", + "ement s", + "emen ts", + "e ments", + "as c", + "a sc", + "ad or", + "ado r", + "le ctions", + "lect ions", + "lection s", + "▁cl ub", + "▁ club", + "] {", + "▁* )", + "▁ *)", + "ст во", + "ств о", + "с тво", + "▁im m", + "▁i mm", + "▁ imm", + "▁for mer", + "▁form er", + "▁forme r", + "▁ former", + "▁r ights", + "▁right s", + "▁dec ided", + "▁decide d", + "▁decid ed", + "▁re v", + "▁r ev", + "▁ rev", + "▁m ent", + "▁me nt", + "▁men t", + "▁ ment", + "an i", + "a ni", + "▁st ru", + "▁str u", + "▁ stru", + "▁att ention", + "art ment", + "▁I tal", + "▁It al", + "al le", + "all e", + "a lle", + "▁b is", + "▁bi s", + "▁ bis", + "ge ner", + "gen er", + "g ener", + "▁in tegr", + "▁int egr", + "▁inte gr", + "▁ integr", + "el lo", + "ell o", + "ry pt", + "▁a chie", + "ne s", + "n es", + "▁s tra", + "▁st ra", + "▁str a", + "▁ stra", + "s b", + "▁t ypes", + "▁type s", + "▁typ es", + "▁ty pes", + "▁ types", + "▁R E", + "▁ RE", + "In it", + "I nit", + "▁com ment", + "▁comm ent", + "▁comme nt", + "▁ comment", + "▁add ition", + "▁I D", + "▁ ID", + "AR T", + "A RT", + "F O", + "щ и", + "Con ne", + "Conn e", + "C onne", + "▁s qu", + "▁sq u", + "▁consider ed", + "▁consid ered", + "id ad", + "ida d", + "▁Oct ober", + "ci al", + "cia l", + "c ial", + "▁O f", + "▁ Of", + "▁tr avel", + "▁tra vel", + "▁trav el", + "▁b oy", + "▁bo y", + "▁ boy", + "') .", + "' ).", + "u y", + "il la", + "ill a", + "i lla", + "is try", + "ist ry", + "istr y", + "▁v a", + "▁ va", + "▁C he", + "▁Ch e", + "▁ Che", + "ER T", + "E RT", + "en de", + "end e", + "e nde", + "un gen", + "ung en", + "unge n", + "ab y", + "a by", + "▁R ober", + "▁Ro ber", + "▁Rob er", + "▁play ing", + "il s", + "i ls", + "▁s am", + "▁sa m", + "▁ sam", + "▁ex ecut", + "▁exec ut", + "▁ execut", + "▁U s", + "▁ Us", + "▁m ut", + "▁mu t", + "▁ mut", + "▁b al", + "▁ba l", + "▁ bal", + "as se", + "ass e", + "▁k ids", + "▁kid s", + "▁ki ds", + "▁fin anc", + "go r", + "g or", + "▁S ec", + "▁Se c", + "▁ Sec", + "ber t", + "be rt", + "b ert", + "▁H igh", + "▁Hig h", + "▁Hi gh", + "▁ High", + "▁ је", + "▁ke pt", + "but ton", + "b utton", + "it ory", + "itor y", + "ito ry", + "▁R em", + "▁Re m", + "▁ Rem", + "▁D E", + "▁ DE", + "▁re ach", + "▁r each", + "▁ reach", + "▁b ur", + "▁bu r", + "▁ bur", + "La bel", + "L abel", + "á t", + "ag o", + "a go", + "▁pass ed", + "▁pas sed", + "▁be hav", + "▁beh av", + "xF F", + "x FF", + "▁R eturn", + "▁Re turn", + "▁Ret urn", + "▁ Return", + "ST R", + "S TR", + "▁L es", + "▁Le s", + "▁ Les", + "▁o rd", + "▁or d", + "▁ ord", + "al a", + "a la", + "in ger", + "ing er", + "inge r", + "▁S ince", + "▁Sin ce", + "▁ Since", + "▁exper i", + "▁exp eri", + "▁s hall", + "▁sh all", + "▁sha ll", + "▁ shall", + "▁s tar", + "▁st ar", + "▁sta r", + "▁ star", + "no n", + "n on", + "▁g un", + "▁gu n", + "▁ gun", + "▁B el", + "▁Be l", + "▁ Bel", + "▁ob j", + "▁ obj", + "ar es", + "are s", + "a res", + "r s", + "▁we eks", + "▁week s", + "ne n", + "n en", + "▁S tre", + "▁St re", + "▁Str e", + "or ing", + "ori ng", + "o ring", + "▁ î", + "▁ser ious", + "time s", + "ti mes", + "tim es", + "t imes", + "▁H ouse", + "▁Ho use", + "▁Hou se", + "▁r oll", + "▁ro ll", + "▁ roll", + "▁reg ister", + "▁ register", + "▁mod ule", + "▁mo dule", + "▁ module", + "▁app lic", + "▁ap plic", + "▁appl ic", + "I R", + "▁c ook", + "▁co ok", + "▁ cook", + "au x", + "a ux", + "▁s ave", + "▁sa ve", + "▁sav e", + "▁ save", + "▁C r", + "▁ Cr", + ", \r", + "▁st ates", + "▁stat es", + "▁state s", + "▁sta tes", + "▁ states", + "▁em pty", + "▁emp ty", + "▁empt y", + "▁ empty", + "▁aut om", + "▁au tom", + "▁auto m", + "▁ autom", + "fig ure", + "ian ce", + "i ance", + "▁h appy", + "▁happ y", + "▁f n", + "▁ fn", + "▁j ud", + "▁ju d", + "▁ jud", + "▁h at", + "▁ha t", + "▁ hat", + "AC K", + "A CK", + "▁F e", + "▁ Fe", + "$ -", + "iv il", + "ivi l", + "i vil", + "ot ed", + "ote d", + "o ted", + "▁size of", + "▁ sizeof", + "▁sit uation", + "▁situ ation", + "▁l ives", + "▁li ves", + "▁live s", + "▁liv es", + "▁fe eling", + "▁feel ing", + "▁fee ling", + "▁r isk", + "▁ri sk", + "▁ris k", + "▁Jan uary", + "▁Januar y", + "▁Ob ject", + "▁ Object", + "▁re comm", + "▁rec omm", + "▁в ы", + "▁ вы", + "▁pot ential", + "ea h", + "e ah", + "▁com plex", + "▁comp lex", + "▁compl ex", + "▁ complex", + "print f", + "ist ance", + "istan ce", + "i stance", + "ir th", + "irt h", + "li k", + "l ik", + "as te", + "ast e", + "a ste", + "▁wh ose", + "▁who se", + "Ar g", + "A rg", + "▁mod ern", + "▁mo dern", + "▁mode rn", + "▁moder n", + "ion es", + "io nes", + "ione s", + "i ones", + "▁ч е", + "▁ че", + "▁s ett", + "▁se tt", + "▁set t", + "▁M ag", + "▁Ma g", + "▁ Mag", + "a e", + "▁cond ition", + "▁ condition", + "Le ngth", + "L ength", + "▁f it", + "▁fi t", + "▁ fit", + "ound s", + "oun ds", + "▁ch anged", + "▁chang ed", + "▁change d", + "▁ changed", + "▁g uy", + "▁gu y", + "fil ter", + "at ever", + "ate ver", + "é d", + "re move", + "rem ove", + "▁h op", + "▁ho p", + "▁ hop", + "▁O ut", + "▁ Out", + "▁R ich", + "▁Ric h", + "▁ Rich", + "ch ild", + "chi ld", + "▁in cluded", + "▁incl uded", + "▁includ ed", + "▁include d", + "▁inclu ded", + "$ \\", + "▁T om", + "▁To m", + "▁ Tom", + "el ine", + "eli ne", + "elin e", + "e line", + "▁s ometimes", + "▁some times", + "▁somet imes", + "▁sometime s", + "▁dr ink", + "▁qu ant", + "▁ quant", + "▁p lease", + "▁ple ase", + "▁I nt", + "▁In t", + "▁ Int", + "ri ef", + "rie f", + "r ief", + "▁ex actly", + "▁exact ly", + "ci ng", + "cin g", + "c ing", + "▁all owed", + "▁allow ed", + "▁ allowed", + "bu ild", + "b uild", + "▁beaut iful", + "▁W ell", + "▁We ll", + "▁Wel l", + "▁ Well", + "▁look s", + "▁lo oks", + "▁ ü", + "▁ch ance", + "▁w rote", + "▁wr ote", + "▁n or", + "▁no r", + "▁ nor", + "▁f ailed", + "▁fa iled", + "▁fail ed", + "▁ failed", + "Me t", + "M et", + "▁p rior", + "▁pr ior", + "▁pri or", + "▁h undred", + "ско й", + "с кой", + "or ia", + "ori a", + "o ria", + "▁c y", + "▁ cy", + "▁w eb", + "▁we b", + "▁ web", + "▁m ess", + "▁me ss", + "▁mes s", + "le q", + "l eq", + "d y", + "te x", + "t ex", + "▁a nim", + "▁an im", + "▁ anim", + "at ur", + "atu r", + "▁str ucture", + "▁struct ure", + "▁ structure", + "opt ion", + "o ption", + "▁act ual", + "▁ actual", + "▁Fr anc", + "▁Fra nc", + "▁Fran c", + "en ced", + "ence d", + "enc ed", + ".< /", + ". ", + "▁ />", + "▁p roduction", + "▁produ ction", + "▁product ion", + "▁prod uction", + "▁ production", + "ig er", + "ige r", + "i ger", + "▁с т", + "▁ ст", + "sh ow", + "s how", + "▁pop ulation", + "▁popul ation", + "▁p ark", + "▁par k", + "▁ park", + "▁Z e", + "▁necess ary", + "▁ necessary", + "▁t rust", + "▁tr ust", + "▁sh own", + "▁show n", + "mod ule", + "mo dule", + "G E", + "▁l ay", + "▁la y", + "▁ lay", + "▁ann oun", + "▁class Name", + "▁ className", + "▁cal cul", + "▁calc ul", + "Fun ction", + "F unction", + "▁S al", + "▁Sa l", + "▁ Sal", + "O K", + "T P", + "▁en try", + "▁ent ry", + "▁entr y", + "▁ entry", + "▁St ud", + "▁ Stud", + "▁it ems", + "▁item s", + "▁ items", + "▁se curity", + "▁sec urity", + "▁secur ity", + "▁ security", + "En try", + "Ent ry", + "f loat", + "l s", + "ib ly", + "▁cont ribut", + "▁C heck", + "▁Che ck", + "▁ Check", + "M D", + "▁impro ve", + "Par t", + "P art", + "▁system s", + "▁syst ems", + "B l", + "▁pol icy", + "▁polic y", + "▁ policy", + "▁s creen", + "▁sc reen", + "▁scr een", + "▁ screen", + "▁A ny", + "▁An y", + "▁ Any", + "▁op ened", + "▁open ed", + "al loc", + "all oc", + "allo c", + "▁De cember", + "▁Dec ember", + "▁ É", + "▁e mail", + "▁em ail", + "▁ email", + "ad er", + "ade r", + "a der", + "= >", + "▁H en", + "▁He n", + "▁ Hen", + "▁in fo", + "▁inf o", + "▁ info", + "▁f loat", + "▁flo at", + "▁ float", + "▁sw itch", + "▁ switch", + "ра н", + "р ан", + "ur ance", + "▁as sum", + "▁ass um", + "us tr", + "ust r", + "u str", + "▁g roups", + "▁group s", + "▁gro ups", + "▁ groups", + "▁R ead", + "▁Re ad", + "▁ Read", + "▁w at", + "▁wa t", + "S p", + "ве р", + "в ер", + "RA N", + "R AN", + "hi b", + "h ib", + "AL L", + "A LL", + "▁h us", + "▁ hus", + "Sp ec", + "Spe c", + "S pec", + "\") )", + "\" ))", + "▁F rench", + "▁C lass", + "▁Cl ass", + "▁ Class", + "▁pres ident", + "▁presid ent", + "▁def init", + "▁defin it", + "▁N or", + "▁No r", + "▁T hom", + "▁Th om", + "ai gn", + "a ign", + "W idth", + "D o", + "▁{ @", + "ag on", + "ago n", + "a gon", + "▁L u", + "▁ Lu", + "▁follow ed", + "M M", + "as ons", + "ason s", + "tm p", + "t mp", + "▁th rows", + "▁throw s", + "▁thr ows", + "▁thro ws", + "▁ throws", + "IT Y", + "I TY", + "но м", + "▁f air", + "▁fa ir", + "▁p en", + "▁pe n", + "▁ pen", + "é g", + "▁inter face", + "▁ interface", + "▁s af", + "▁sa f", + "oo n", + "o on", + "B ack", + "▁s peed", + "▁sp eed", + "▁spe ed", + "▁ speed", + "▁ext ends", + "▁extend s", + "em pty", + "empt y", + "emp ty", + "▁п ере", + "▁пер е", + "▁пе ре", + "▁pro per", + "▁pr oper", + "▁prop er", + "▁d riv", + "▁dr iv", + "▁dri v", + "ф и", + "▁c enter", + "▁cent er", + "▁ center", + "he ader", + "head er", + "▁} )", + "▁ })", + "w a", + "▁m iddle", + "▁ middle", + "▁ch oose", + "▁cho ose", + "▁St ad", + "▁Sta d", + "S O", + "Fact ory", + "Factor y", + "F actory", + "De v", + "D ev", + "ic les", + "icle s", + "icl es", + "i cles", + "▁ap plication", + "▁applic ation", + "▁appl ication", + "▁ application", + "▁mod els", + "▁model s", + "▁mode ls", + "▁ models", + "pi te", + "pit e", + "p ite", + "ca p", + "c ap", + "x i", + "osp ital", + "▁d ream", + "▁dre am", + "EN D", + "E ND", + "▁con tract", + "▁cont ract", + "▁contr act", + "▁contra ct", + "▁ contract", + "icro soft", + "▁th ous", + "▁thou s", + "iz es", + "ize s", + "i zes", + "▁д а", + "▁ да", + "▁C O", + "▁ CO", + "▁d irection", + "▁di rection", + "▁direct ion", + "▁dire ction", + "▁dir ection", + "▁ direction", + "▁` `", + "▁ ``", + "▁d rive", + "▁dr ive", + "▁dri ve", + "▁driv e", + "▁ drive", + "Ma x", + "M ax", + "ci a", + "c ia", + "▁contin u", + "▁A lex", + "▁Al ex", + "▁Ale x", + "▁ Alex", + "▁g old", + "▁go ld", + "▁gol d", + "▁ gold", + "▁p rep", + "▁pre p", + "▁pr ep", + "▁or igin", + "▁orig in", + "▁ origin", + "▁r ap", + "▁ra p", + "▁ rap", + "O p", + "ous ly", + "▁are as", + "▁area s", + "PO RT", + "P ORT", + "он а", + "о на", + "▁sa fe", + "▁saf e", + "▁ safe", + "▁profess ional", + "▁profession al", + "ap ache", + "apa che", + "▁t emper", + "▁tem per", + "▁temp er", + "s z", + "▁u nit", + "▁un it", + "▁ unit", + "▁c op", + "▁co p", + "▁ cop", + "eq n", + "List ener", + "Listen er", + "▁for mat", + "▁form at", + "▁forma t", + "▁ format", + "se lect", + "sel ect", + "s elect", + "▁com fort", + "▁ comfort", + "▁me ant", + "▁mean t", + "id ay", + "ida y", + "i day", + "em e", + "e me", + "▁act ive", + "▁activ e", + "▁ active", + "▁n ote", + "▁not e", + "▁no te", + "▁ note", + "▁M il", + "▁Mi l", + "▁ Mil", + "on ly", + "▁< =", + "▁ <=", + "▁ne igh", + "▁nei gh", + "a o", + "▁bl ue", + "▁ blue", + "▁T V", + "▁ TV", + "Ch ild", + "▁re ached", + "▁reach ed", + "Add ress", + "Addr ess", + "ст в", + "▁cl osed", + "▁close d", + "▁clos ed", + "▁clo sed", + "▁ closed", + "in der", + "ind er", + "inde r", + "i nder", + "ol o", + "o lo", + "▁a lt", + "▁al t", + "▁ alt", + "▁a dm", + "▁ad m", + "Form at", + "For mat", + "U I", + "▁H am", + "▁Ha m", + "▁f requ", + "▁fr equ", + "▁fre qu", + "▁in depend", + "▁inde pend", + "▁ independ", + "▁eas ily", + "▁L and", + "▁La nd", + "▁Lan d", + "▁ Land", + "▁t or", + "▁to r", + "▁ tor", + "ograph y", + "ograp hy", + "in fty", + "inf ty", + "▁W ork", + "▁Wor k", + "▁ Work", + "iv en", + "ive n", + "i ven", + "▁Count y", + "▁Coun ty", + "▁s rc", + "▁ src", + "}$ ,", + "} $,", + "par se", + "pars e", + "p arse", + "C D", + "▁C our", + "▁Co ur", + "▁Cou r", + "▁f ol", + "▁fo l", + "▁ fol", + "Ent ity", + "pg f", + "▁Ch ina", + "▁Chi na", + "▁S ub", + "▁Su b", + "▁ Sub", + "ho od", + "h ood", + "▁field s", + "▁ fields", + "▁y es", + "▁ye s", + "▁ yes", + "re nd", + "ren d", + "r end", + "▁to wards", + "▁toward s", + "▁tow ards", + "▁st aff", + "▁sta ff", + "▁ staff", + "▁A ir", + "▁ Air", + "▁st ation", + "▁stat ion", + "▁ station", + "at ives", + "ative s", + "ati ves", + "ativ es", + "▁imp act", + "в ы", + "▁direct ly", + "iss ions", + "ission s", + "iv a", + "i va", + "| \\", + "Pt r", + "P tr", + "▁S ant", + "▁San t", + "▁Sa nt", + "Po l", + "P ol", + "▁pro gress", + "▁ progress", + "it ar", + "ita r", + "i tar", + "▁p arts", + "▁part s", + "▁par ts", + "▁ parts", + "▁pl ant", + "▁plan t", + "▁ plant", + "▁abs olut", + "▁gu ess", + "eq ref", + "▁t im", + "▁ti m", + "▁ tim", + "▁L ou", + "▁Lo u", + "▁ Lou", + "▁c ool", + "▁co ol", + "al u", + "a lu", + "▁m outh", + "▁mo uth", + "▁mou th", + "▁ mouth", + "ни х", + "▁h eight", + "▁he ight", + "▁ height", + "ge st", + "ges t", + "g est", + "▁P ost", + "▁Po st", + "▁Pos t", + "▁ Post", + "▁b oard", + "▁bo ard", + "▁ board", + "▁t it", + "▁ti t", + "▁ tit", + "▁h our", + "▁ho ur", + "▁ hour", + "▁ser ver", + "▁serv er", + "▁serve r", + "▁ server", + "▁p layers", + "▁play ers", + "▁player s", + "ri er", + "rie r", + "r ier", + "Lin k", + "L ink", + "▁Pres ident", + "] (", + "▁con struct", + "▁const ruct", + "▁constr uct", + "▁constru ct", + "▁ construct", + "hand le", + "}$ .", + "} $.", + "ry ing", + "r ying", + "▁s hop", + "▁sh op", + "▁ shop", + "ia na", + "ian a", + "i ana", + "ex p", + "e xp", + "Hel per", + "Help er", + "Off set", + "ac hes", + "ach es", + "ache s", + "a ches", + "▁conne ction", + "▁connect ion", + "▁conn ection", + "▁ connection", + "▁d ifference", + "▁dif ference", + "▁differ ence", + "serv ice", + "s ervice", + "▁g as", + "▁ga s", + "▁ gas", + "▁p riv", + "▁pr iv", + "▁pri v", + "▁ priv", + "▁un ivers", + "▁ univers", + "▁w ish", + "▁wis h", + "Re m", + "R em", + "U rl", + "ge b", + "g eb", + "S o", + "ens ions", + "ension s", + "Mod ule", + "Mo dule", + "SI ZE", + "▁p rem", + "▁pre m", + "▁pr em", + "wind ow", + "w indow", + "▁d ies", + "▁di es", + "▁die s", + "de l", + "d el", + "▁r ow", + "▁ro w", + "▁ row", + "▁a verage", + "▁aver age", + "▁ave rage", + "xi m", + "x im", + "▁p u", + "▁ pu", + "an ç", + "De t", + "D et", + "ke r", + "k er", + "y a", + "▁D et", + "▁De t", + "▁ Det", + "▁p å", + "▁n amed", + "▁name d", + "▁na med", + "▁nam ed", + "▁ named", + "▁dec ision", + "▁decis ion", + "wi n", + "w in", + "▁Ge orge", + "▁Georg e", + "ar ily", + "ari ly", + "▁s olution", + "▁sol ution", + "▁mult iple", + "▁multi ple", + "▁multip le", + "▁ multiple", + "at egy", + "ate gy", + "ateg y", + "▁le arning", + "▁learn ing", + "▁lear ning", + "▁ learning", + "▁se cret", + "▁sec ret", + "▁secre t", + "▁ secret", + "D O", + "▁n ice", + "▁ni ce", + "▁nic e", + "▁ nice", + "//////// ////////", + "S u", + "it ation", + "itat ion", + "▁j oin", + "▁jo in", + "▁ join", + "▁el ements", + "▁element s", + "▁ele ments", + "▁elem ents", + "▁ elements", + "▁e mer", + "▁em er", + "til de", + "t ilde", + "▁d ep", + "▁de p", + "▁ dep", + "▁s hot", + "▁sh ot", + "▁ shot", + "▁pl atform", + "▁plat form", + "▁ platform", + "ot hing", + "oth ing", + "o thing", + "M y", + "ed ia", + "edi a", + "om s", + "o ms", + "ail y", + "ai ly", + "a ily", + "( [", + "▁d ress", + "▁dr ess", + "▁dre ss", + "▁off icial", + "▁offic ial", + "es tern", + "est ern", + "ester n", + "este rn", + "▁dis cover", + "▁disc over", + "▁disco ver", + "▁m i", + "▁ mi", + "ны е", + "C A", + "od ing", + "odi ng", + "o ding", + "▁F ound", + "▁Fou nd", + "▁Fo und", + "▁ Found", + "▁a ffect", + "▁aff ect", + "▁af fect", + "Vi s", + "V is", + "st ract", + "str act", + "stra ct", + "s tract", + "ic ed", + "ice d", + "i ced", + "de bug", + "d ebug", + "▁rel ated", + "▁relate d", + "▁ related", + "▁s pect", + "▁sp ect", + "▁spec t", + "▁spe ct", + "▁ spect", + "us hed", + "ush ed", + "сь ко", + "▁b ank", + "▁ban k", + "▁ bank", + "▁c ele", + "▁ce le", + "▁cel e", + "AN D", + "A ND", + "ol f", + "е м", + "▁f ill", + "▁fil l", + "▁fi ll", + "▁ fill", + "▁g ives", + "▁giv es", + "▁give s", + "▁gi ves", + "▁б у", + "▁ бу", + "ar on", + "aro n", + "a ron", + "▁J es", + "▁Je s", + "RE G", + "▁s udd", + "▁su dd", + "▁sud d", + "date d", + "da ted", + "dat ed", + "d ated", + "v i", + "▁g i", + "▁ gi", + "se nd", + "sen d", + "s end", + "cp p", + "c pp", + "▁s pent", + "▁sp ent", + "▁spe nt", + "an de", + "and e", + "a nde", + "▁oper ation", + "▁ operation", + "pro cess", + "proc ess", + "▁in form", + "▁inf orm", + "▁info rm", + "▁F ree", + "▁Fr ee", + "▁Fre e", + "▁ Free", + "yo nd", + "y ond", + "▁per haps", + "▁su rv", + "▁sur v", + "▁L oc", + "▁Lo c", + "▁ Loc", + "▁con cl", + "▁conc l", + "▁ра з", + "▁ раз", + "▁O ver", + "▁ Over", + "ho l", + "h ol", + "ra z", + "r az", + "Wr ite", + "Writ e", + "W rite", + "▁g iving", + "▁giv ing", + "▁gi ving", + "r d", + "in stance", + "inst ance", + "▁re leased", + "▁rele ased", + "▁release d", + "▁R o", + "▁ Ro", + "R A", + "▁pract ice", + "▁g raph", + "▁gr aph", + "▁gra ph", + "▁grap h", + "▁ graph", + "▁incre ase", + "▁fig ure", + "▁ figure", + "Fil ter", + "HE CK", + "id x", + "i dx", + "▁g lass", + "▁gl ass", + "▁ glass", + "sk i", + "s ki", + "com es", + "co mes", + "come s", + "c omes", + "▁c at", + "▁ca t", + "▁ cat", + "▁c old", + "▁col d", + "▁co ld", + "go to", + "got o", + "g oto", + "uf act", + "u fact", + "▁C opyright", + "▁Copy right", + "▁ Copyright", + "}} \\", + "} }\\", + "▁str eng", + "▁stre ng", + "▁d ir", + "▁di r", + "▁ dir", + "to ken", + "tok en", + "t oken", + "▁occ ur", + "▁oc cur", + "arl ier", + "▁me asure", + "▁meas ure", + "▁ measure", + "▁s ec", + "▁se c", + "▁ sec", + "▁m ás", + "▁má s", + "▁N et", + "▁Ne t", + "▁ Net", + "▁arg ument", + "▁ argument", + "▁s ou", + "▁so u", + "▁m oving", + "▁mov ing", + "▁mo ving", + "▁p refer", + "▁pre fer", + "▁pref er", + "ma sk", + "mas k", + "m ask", + "< <", + "▁bre ath", + "▁breat h", + "▁phys ical", + "▁pos itive", + "▁posit ive", + "▁s or", + "▁so r", + "▁ sor", + "▁de part", + "▁dep art", + "▁re move", + "▁rem ove", + "▁ remove", + "▁k it", + "▁ki t", + "▁ kit", + "▁me eting", + "▁meet ing", + "▁D ata", + "▁Da ta", + "▁Dat a", + "▁ Data", + "og raf", + "act ions", + "action s", + "a ctions", + "▁param eters", + "▁parameter s", + "▁ parameters", + "▁A tt", + "▁At t", + "▁ Att", + "es ch", + "esc h", + "e sch", + "▁inv olved", + "▁invol ved", + "▁involve d", + "ä t", + "L L", + "B ar", + "▁с и", + "▁ си", + "ec h", + "e ch", + "GE T", + "G ET", + "▁pre vent", + "▁pr event", + "▁prev ent", + "▁ prevent", + "▁be yond", + "▁O ther", + "▁Ot her", + "▁ Other", + "ä n", + "by te", + "▁sudd en", + "▁sud den", + "ol ve", + "olv e", + "▁н о", + "▁ но", + "LO G", + "L OG", + "un it", + "uni t", + "u nit", + "▁tr uth", + "ra t", + "r at", + "S D", + "▁e at", + "▁M ad", + "▁Ma d", + "▁ Mad", + "▁prov ides", + "▁provide s", + "▁s ession", + "▁ session", + "De le", + "Del e", + "D ele", + "▁con vers", + "▁conv ers", + "▁conver s", + "▁conve rs", + "cent er", + "cen ter", + "c enter", + "▁contin ued", + "▁continue d", + "▁continu ed", + "ot ion", + "oti on", + "ca che", + "c ache", + "dis play", + "disp lay", + "▁prote ct", + "▁prot ect", + "am s", + "a ms", + "▁p ow", + "▁po w", + "▁ pow", + "CT ION", + "C TION", + "▁M ac", + "▁Ma c", + "▁ Mac", + "m o", + "х а", + "▁d istance", + "▁di stance", + "▁dist ance", + "▁ distance", + "▁T ime", + "▁Tim e", + "▁Ti me", + "▁ Time", + "g i", + "▁s equ", + "▁se qu", + "▁seq u", + "▁ sequ", + "T arget", + "с ле", + "Ser ver", + "Serv er", + "▁w ide", + "▁wid e", + "▁ wide", + "cl ose", + "clos e", + "▁c ru", + "▁cr u", + "Ex t", + "E xt", + "▁s elect", + "▁se lect", + "▁sel ect", + "▁sele ct", + "▁ select", + "▁pat tern", + "▁ pattern", + "\") );", + "\")) ;", + "\" ));", + "Pro vider", + "Prov ider", + "UR L", + "U RL", + "▁g reen", + "▁gr een", + "▁gre en", + "▁ green", + "▁wait ing", + "▁wa iting", + "pro to", + "pr oto", + "prot o", + "▁immedi ately", + "▁immediate ly", + "com mon", + "comm on", + "az ione", + "azi one", + "a zione", + "ri ver", + "riv er", + "rive r", + "r iver", + "▁s en", + "▁se n", + "▁ sen", + "▁! ==", + "▁!= =", + "▁Febru ary", + "▁Februar y", + "ur b", + "u rb", + "▁S en", + "▁Se n", + "de st", + "des t", + "d est", + "< ?", + "▁ed ge", + "▁ edge", + "▁m ais", + "▁ma is", + "▁mai s", + "gor ith", + "cp u", + "c pu", + "▁educ ation", + "▁associ ated", + "▁associate d", + "No ne", + "Non e", + "N one", + "h i", + "▁p oor", + "▁po or", + "se m", + "s em", + "▁W il", + "▁Wi l", + "▁b ud", + "▁bu d", + "▁ bud", + "▁a uch", + "▁au ch", + "▁ auch", + "el ler", + "ell er", + "elle r", + "▁L ife", + "▁Li fe", + "▁ Life", + "▁f iles", + "▁fil es", + "▁file s", + "▁fi les", + "▁ files", + "▁le ading", + "▁lead ing", + "▁ leading", + "▁ob tain", + "▁obt ain", + "▁J ul", + "▁Ju l", + "at ory", + "ator y", + "ato ry", + "г у", + "it able", + "ita ble", + "i table", + "▁on to", + "▁ont o", + "▁ onto", + "▁b orn", + "▁bo rn", + "▁bor n", + "▁ born", + "or em", + "ore m", + "o rem", + "▁Stre et", + "▁m aint", + "▁main t", + "▁ma int", + "▁mai nt", + "Param s", + "Par ams", + "ri p", + "r ip", + "▁S T", + "▁ ST", + "u v", + "ma in", + "m ain", + "▁re cent", + "▁rec ent", + "▁rece nt", + "We b", + "W eb", + "ov a", + "o va", + "ц а", + "ais e", + "ai se", + "a ise", + "yle s", + "yl es", + "y les", + "▁de scribed", + "▁desc ribed", + "▁describ ed", + "▁describe d", + "▁begin ning", + "▁D ay", + "▁Da y", + "▁ Day", + "▁V ol", + "▁Vo l", + "▁ Vol", + "▁h uge", + "▁hug e", + "Ha s", + "H as", + "an cy", + "anc y", + "He ader", + "Head er", + "▁a ren", + "▁are n", + "▁ar en", + "▁ aren", + "ва н", + "в ан", + "▁en sure", + "▁ens ure", + "▁ ensure", + "▁p et", + "▁pe t", + "▁ pet", + "mu lt", + "mul t", + "m ult", + "▁L ike", + "▁Li ke", + "▁ Like", + "▁man agement", + "▁manage ment", + "▁ management", + "P S", + "wh ile", + "▁back ground", + "▁ background", + "ount er", + "oun ter", + "o unter", + "bo ol", + "b ool", + "F C", + "N um", + "R L", + "▁ex cl", + "▁exc l", + "▁e ye", + "▁ey e", + "im g", + "i mg", + "▁r om", + "▁ro m", + "▁ rom", + "▁H el", + "▁He l", + "▁ Hel", + "Opt ion", + "O ption", + "▁stop ped", + "▁sto pped", + "▁th read", + "▁thr ead", + "▁ thread", + "to type", + "tot ype", + "t otype", + ")) )", + ") ))", + "▁st age", + "▁stag e", + "▁sta ge", + "▁ stage", + "▁ü ber", + "▁ über", + "▁al though", + "▁ although", + "Type s", + "Ty pes", + "Typ es", + "T ypes", + "▁O h", + "▁ Oh", + "▁e ight", + "▁ eight", + "▁de scription", + "▁des cription", + "▁ description", + "' '", + "ö n", + "▁sur face", + "▁surf ace", + "▁ surface", + "▁Intern ational", + "▁ch arg", + "▁char g", + "▁cha rg", + "▁ charg", + "▁col lection", + "▁coll ection", + "▁collect ion", + "▁colle ction", + "▁ collection", + "▁us ers", + "▁use rs", + "▁user s", + "▁ users", + "▁ob vious", + "▁cent ury", + "▁ century", + "ic ks", + "ick s", + "i cks", + "▁art icle", + "▁artic le", + "▁ article", + "▁\" \\", + "▁ \"\\", + "di m", + "d im", + "▁s in", + "▁si n", + "▁ sin", + "en ge", + "eng e", + "Cont rol", + "▁com mit", + "▁comm it", + "▁ commit", + "ens ity", + "▁t ra", + "▁tr a", + "▁ tra", + "cript or", + "▁N OT", + "▁NO T", + "▁ NOT", + "we ll", + "w ell", + "▁M ichael", + "▁Mich ael", + "▁n od", + "▁no d", + "▁ nod", + "▁m ort", + "▁mor t", + "▁mo rt", + "iv o", + "i vo", + "is ation", + "▁P o", + "▁ Po", + "▁P aris", + "▁Par is", + "▁Pa ris", + "▁ad ministr", + "▁admin istr", + "▁ administr", + "bu rg", + "bur g", + "b urg", + "cd ot", + "c dot", + "▁mil itary", + "▁milit ary", + "▁militar y", + "▁B est", + "▁Be st", + "▁Bes t", + "▁ Best", + "▁К а", + "▁ Ка", + "IN E", + "I NE", + "▁through out", + "S l", + "▁im pl", + "▁imp l", + "▁ impl", + "cont rol", + "contr ol", + "▁ Ч", + "▁u it", + "▁ui t", + "▁ uit", + "▁un signed", + "▁uns igned", + "▁ unsigned", + "▁M ary", + "▁Mar y", + "▁Ma ry", + "Ch ar", + "C har", + "м і", + "▁th reat", + "▁c ourt", + "▁co urt", + "▁cour t", + "▁cou rt", + "▁ court", + "vi lle", + "vil le", + "v ille", + "▁ ш", + "▁C am", + "▁Ca m", + "▁ Cam", + ". \r", + "▁current ly", + "▁curr ently", + "ro t", + "r ot", + "▁D ate", + "▁Da te", + "▁Dat e", + "▁ Date", + "▁s hit", + "▁sh it", + "▁ shit", + "▁$ {\\", + "▁${ \\", + "un n", + "u nn", + "U s", + "▁b uffer", + "▁buff er", + "▁buf fer", + "▁ buffer", + "▁s ont", + "▁so nt", + "▁son t", + "▁let ter", + "▁lett er", + "▁ letter", + "in ated", + "ina ted", + "inate d", + "Ch ange", + "▁h ref", + "▁hr ef", + "▁ href", + "▁l ack", + "▁la ck", + "▁lac k", + "▁o il", + "▁C ons", + "▁Con s", + "▁Co ns", + "▁ Cons", + "▁J er", + "▁Je r", + "BU G", + "B UG", + "if orn", + "▁pro perties", + "▁proper ties", + "▁ properties", + "▁r andom", + "▁ran dom", + "▁rand om", + "▁ random", + "▁br other", + "▁bro ther", + "▁p iece", + "▁pie ce", + "▁ piece", + "б у", + "ist ics", + "istic s", + "isti cs", + "▁techn ology", + "gl obal", + "glob al", + "▁trans form", + "▁ transform", + "er d", + "e rd", + "▁B ecause", + "▁ Because", + "PE CT", + "P ECT", + "pr et", + "pre t", + "p ret", + "▁го ду", + "▁год у", + "▁M et", + "▁Me t", + "▁ Met", + "▁p sy", + "▁ps y", + "▁ psy", + "▁о д", + "▁g od", + "▁go d", + "▁ god", + "▁D el", + "▁De l", + "▁ Del", + "base d", + "ba sed", + "bas ed", + "b ased", + "▁v oor", + "▁vo or", + "▁C all", + "▁Cal l", + "▁Ca ll", + "▁ Call", + "S A", + "▁fil ter", + "▁ filter", + "▁incl udes", + "▁includ es", + "▁include s", + "▁inclu des", + "▁ includes", + "olut ions", + "olution s", + "f d", + "▁w ind", + "▁win d", + "▁ wind", + "▁б о", + "▁ бо", + "▁ab ility", + "▁ ability", + "ca rd", + "car d", + "c ard", + "▁n umer", + "▁num er", + "▁nu mer", + "▁ numer", + "add ress", + "addr ess", + "▁go al", + "ash ington", + "ashing ton", + "▁s light", + "▁sl ight", + "ab a", + "a ba", + "▁L og", + "▁Lo g", + "▁ Log", + "Set tings", + "Setting s", + "ad ow", + "ado w", + "▁p i", + "▁ pi", + "ir ing", + "iri ng", + "i ring", + "F T", + "▁number s", + "▁num bers", + "con f", + "co nf", + "ta sk", + "t ask", + "▁î n", + "т ы", + "▁re ceive", + "▁rece ive", + "▁r oot", + "▁ro ot", + "▁ root", + "▁Ind ia", + "pat ch", + "p atch", + "é l", + "▁sum mer", + "▁method s", + "▁ methods", + "▁pl aces", + "▁place s", + "▁plac es", + "▁М а", + "▁ Ма", + "▁cap ital", + "▁capit al", + "▁ev idence", + "▁G erman", + "▁Germ an", + "▁Ger man", + "\\ ,", + "D A", + "ec ute", + "ecut e", + "col umn", + "▁fun ctions", + "▁function s", + "▁ functions", + "▁c ounter", + "▁co unter", + "▁coun ter", + "▁count er", + "▁ counter", + "▁ar ms", + "▁arm s", + "▁ arms", + "▁f eed", + "▁fe ed", + "▁fee d", + "▁ feed", + "ve y", + "v ey", + "he nt", + "hen t", + "h ent", + "MA X", + "M AX", + "▁ac qu", + "▁app ly", + "▁ap ply", + "▁appl y", + "▁ apply", + "▁hus band", + "▁k illed", + "▁kill ed", + "▁kil led", + "▁S pec", + "▁Sp ec", + "▁Spe c", + "▁ Spec", + "ent ity", + "enti ty", + "▁e arlier", + "▁M iss", + "▁Mi ss", + "▁Mis s", + "▁ Miss", + "▁set ting", + "▁sett ing", + "▁ setting", + "it ect", + "ite ct", + "▁d ed", + "▁de d", + "▁ ded", + "Ro w", + "R ow", + "▁r an", + "▁ra n", + "▁ ran", + "▁Y es", + "▁Ye s", + "▁ Yes", + "▁fin ancial", + "▁financ ial", + "s ession", + "le ar", + "l ear", + "is hing", + "ish ing", + "ishi ng", + "▁ne arly", + "▁near ly", + "▁d ur", + "▁du r", + "▁m achine", + "▁mach ine", + "▁ machine", + "xf f", + "x ff", + "br o", + "b ro", + "▁s ymbol", + "▁sym bol", + "▁ symbol", + "land s", + "lan ds", + "l ands", + "Ac c", + "A cc", + "d i", + "▁Rober t", + "▁Ro bert", + "▁Rob ert", + "pro p", + "pr op", + "p rop", + "ur ity", + "uri ty", + "▁# ####", + "▁## ###", + "▁### ##", + "▁#### #", + "▁walk ed", + "▁wal ked", + "▁intern ational", + "▁internation al", + "▁ Е", + "Y es", + "▁re lease", + "▁rele ase", + "▁ release", + "▁start ing", + "▁star ting", + "st atic", + "stat ic", + "▁b ei", + "▁be i", + "al low", + "all ow", + "allo w", + "▁Pe ople", + "▁ People", + "e z", + "▁param eter", + "▁ parameter", + "C ache", + "▁$ $", + "▁ $$", + "amp ions", + "ampion s", + "▁M er", + "▁Me r", + "▁ Mer", + "▁k om", + "▁ko m", + "▁ kom", + "le ted", + "let ed", + "lete d", + "l eted", + "oi s", + "o is", + "▁O pen", + "▁Op en", + "▁ Open", + "ty pes", + "type s", + "typ es", + "t ypes", + "▁f ue", + "▁fu e", + "ac ters", + "act ers", + "acter s", + "▁re ference", + "▁refer ence", + "▁ reference", + "Equ als", + "Equal s", + "Eq uals", + "▁a ware", + "▁aw are", + "▁ aware", + "▁h ol", + "▁ho l", + "▁ hol", + "▁de mand", + "▁dem and", + "lo r", + "l or", + "▁v eh", + "▁ve h", + "▁ veh", + "▁not ice", + "▁ notice", + "▁com ponent", + "▁compon ent", + "▁ component", + "f n", + "▁anal ysis", + "▁analy sis", + "▁analys is", + "▁ analysis", + "mat ch", + "m atch", + "▁effect ive", + "▁ effective", + "pro duct", + "produ ct", + "prod uct", + "ни к", + "▁le gal", + "▁leg al", + "▁ legal", + "е й", + "se mb", + "sem b", + "s emb", + "▁loc ated", + "▁locate d", + "▁с у", + "▁ су", + "Q L", + "in ct", + "inc t", + "et o", + "e to", + "Dr aw", + "D raw", + "▁sc ale", + "▁scal e", + "▁ scale", + "ро в", + "р ов", + "▁w ants", + "▁want s", + "H ow", + "▁w el", + "▁we l", + "is ions", + "ision s", + "isi ons", + "▁de liver", + "▁del iver", + "un der", + "und er", + "unde r", + "u nder", + "▁d eb", + "▁de b", + "▁j u", + "▁ ju", + "val ues", + "value s", + "▁s ister", + "▁si ster", + "▁sist er", + "ко в", + "к ов", + "▁C reate", + "▁Creat e", + "▁Cre ate", + "▁ Create", + "▁I nc", + "▁In c", + "▁a ux", + "▁au x", + "▁ aux", + "▁Wh ite", + "▁Whit e", + "▁ White", + "Me nu", + "Men u", + "M enu", + "au d", + "a ud", + "re source", + "res ource", + "▁c ab", + "▁ca b", + "▁l if", + "▁li f", + "▁ lif", + "▁c ulture", + "▁cult ure", + "ic he", + "ich e", + "i che", + "▁wh atever", + "▁what ever", + "▁de signed", + "▁des igned", + "▁design ed", + "▁re pe", + "▁rep e", + "▁M ont", + "▁Mon t", + "▁Mo nt", + "▁ Mont", + "▁ch arge", + "▁char ge", + "▁charg e", + "▁ charge", + "Name s", + "Na mes", + "N ames", + "▁in sp", + "▁ins p", + "▁custom ers", + "▁customer s", + "os a", + "o sa", + "▁d aughter", + "▁E ast", + "E Q", + "▁o pin", + "▁op in", + "▁F re", + "▁Fr e", + "▁se ek", + "▁see k", + "▁ seek", + "▁p ush", + "▁pu sh", + "▁ push", + "▁n av", + "▁na v", + "▁ nav", + "▁b urn", + "▁bu rn", + "▁bur n", + "▁ burn", + "ar den", + "ard en", + "arde n", + "ha sh", + "has h", + "h ash", + "▁opportun ity", + "▁M at", + "▁Ma t", + "▁ Mat", + "oy al", + "oya l", + "o yal", + "▁p un", + "▁pu n", + "sc ale", + "scal e", + "yn amic", + "ynam ic", + "yna mic", + "▁T ype", + "▁Ty pe", + "▁Typ e", + "▁ Type", + "il ing", + "ili ng", + "i ling", + "▁qu ery", + "▁que ry", + "▁quer y", + "▁ query", + "▁m ist", + "▁mis t", + "▁mi st", + "ro r", + "r or", + "for ce", + "▁On ce", + "▁ Once", + "▁med ical", + "▁medic al", + "▁medi cal", + "li e", + "l ie", + "▁stud ent", + "▁ student", + "ed eral", + "eder al", + "ede ral", + "▁l ov", + "▁lo v", + "▁ lov", + "if orm", + "i form", + "▁al tern", + "▁alt ern", + "▁alter n", + "▁ altern", + "bi n", + "b in", + "od er", + "ode r", + "o der", + "▁return s", + "▁ returns", + "reg ister", + "ut s", + "u ts", + "C I", + "▁T or", + "▁To r", + "▁ Tor", + "C R", + "▁L os", + "▁Lo s", + "▁ Los", + "am ily", + "ami ly", + "amil y", + "air e", + "ai re", + "a ire", + "++ ;", + "Cont roller", + "Control ler", + "wi de", + "wid e", + "w ide", + "x x", + "row ser", + "rows er", + "▁B ook", + "▁Bo ok", + "▁ Book", + "Cont ainer", + "pl oad", + "plo ad", + "p load", + "▁E v", + "▁ Ev", + "▁t al", + "▁ta l", + "▁ tal", + "▁the ory", + "eqn array", + "б е", + "▁rep orted", + "▁report ed", + "▁me aning", + "▁mean ing", + "▁s y", + "▁ sy", + "ri be", + "rib e", + "r ibe", + "ic ate", + "ica te", + "ho ld", + "hol d", + "h old", + "▁of fers", + "▁off ers", + "▁offer s", + "▁t empl", + "▁tem pl", + "▁temp l", + "cs s", + "c ss", + "▁p icture", + "▁pict ure", + "▁ picture", + "▁a sync", + "▁as ync", + "▁ async", + "▁st ock", + "▁sto ck", + "▁ stock", + "▁in ternal", + "▁inter nal", + "▁intern al", + "▁ internal", + "t i", + "B O", + "V er", + "с по", + "▁d emon", + "▁de mon", + "▁dem on", + "▁demo n", + "▁l augh", + "▁la ugh", + "▁laug h", + "▁E nd", + "▁En d", + "▁ End", + "▁k on", + "▁ko n", + "▁ kon", + "▁ide as", + "▁idea s", + "▁c andid", + "▁can did", + "▁cand id", + "Me m", + "M em", + "iz z", + "i zz", + "re fix", + "ref ix", + "▁A ND", + "▁AN D", + "▁ AND", + "eg en", + "e gen", + "E l", + "▁camp aign", + "H ttp", + "▁R ob", + "▁Ro b", + "▁ Rob", + "д і", + "▁b ul", + "▁bu l", + "▁ bul", + "▁К о", + "▁ Ко", + "▁count ries", + "▁countr ies", + "» .", + "▁ex pression", + "▁exp ression", + "▁express ion", + "▁expr ession", + "▁ expression", + "▁Eng land", + "s f", + "▁certain ly", + "ag en", + "age n", + "a gen", + "▁ч а", + "▁ ча", + "▁A NY", + "▁AN Y", + "▁ ANY", + "▁conne ct", + "▁conn ect", + "▁ connect", + "F E", + "▁and roid", + "▁ android", + "▁G old", + "▁Go ld", + "▁Gol d", + "▁ Gold", + "▁op pos", + "▁opp os", + "ov ern", + "ove rn", + "over n", + "o vern", + "▁Com mun", + "▁Comm un", + ", _", + "as ion", + "asi on", + "L a", + "▁f irm", + "▁fi rm", + "▁fir m", + "▁Al though", + "▁G ood", + "▁Go od", + "▁ Good", + "▁L aw", + "▁La w", + "er ve", + "erv e", + "▁b rand", + "▁br and", + "▁bra nd", + "▁ brand", + "M in", + "fil l", + "fi ll", + "f ill", + "'] ,", + "' ],", + "▁J ew", + "▁Je w", + "il er", + "ile r", + "i ler", + "in gle", + "ing le", + "it hub", + "ith ub", + "▁D iv", + "▁Di v", + "▁ Div", + "▁c ert", + "▁ce rt", + "▁cer t", + "▁ cert", + "He ight", + "H eight", + "ra el", + "r ael", + "The re", + "Th ere", + "T here", + "it ute", + "itut e", + "itu te", + "▁a maz", + "▁am az", + "▁ amaz", + "lo ok", + "l ook", + "▁S E", + "▁ SE", + "▁j o", + "▁ jo", + "▁pull ed", + "▁pul led", + "▁re sources", + "▁res ources", + "▁resource s", + "▁ resources", + "▁M ax", + "▁Ma x", + "▁ Max", + "▁ag reed", + "▁agree d", + "▁agre ed", + "as y", + "a sy", + "▁treat ment", + "\"> < /", + "\" > >", + "▁ >>", + "com mand", + "comm and", + "at z", + "a tz", + "▁m al", + "▁ma l", + "▁ mal", + "ста в", + "▁P ress", + "▁Pr ess", + "▁Pres s", + "▁Pre ss", + "▁ Press", + "▁char acters", + "▁character s", + "▁z ero", + "▁ze ro", + "▁ zero", + "AG E", + "A GE", + "rap per", + "▁kit chen", + "am ing", + "ami ng", + "amin g", + "a ming", + "▁re str", + "▁r estr", + "▁res tr", + "▁rest r", + "X X", + "▁Col lege", + "▁Ar ray", + "▁Arr ay", + "▁ Array", + "▁f resh", + "▁fr esh", + "▁fre sh", + "▁fres h", + "▁sh ift", + "▁ shift", + "▁spec ified", + "pl ete", + "ple te", + "plet e", + "p lete", + "IT E", + "I TE", + "▁C amp", + "▁Cam p", + "▁Ca mp", + "▁ Camp", + "ri al", + "ria l", + "r ial", + "c b", + "▁T H", + "▁ TH", + "I B", + "os en", + "ose n", + "o sen", + "▁ ú", + "▁par ams", + "▁param s", + "▁para ms", + "▁ params", + "ign ment", + "ad ding", + "add ing", + "▁deg ree", + "▁ degree", + "Loc al", + "Lo cal", + "L ocal", + "O h", + "▁z ur", + "▁zu r", + "▁level s", + "▁lev els", + "C S", + "fin ished", + "finish ed", + "C ase", + "ri age", + "ria ge", + "Vec tor", + "V ector", + "▁s ea", + "▁se a", + "▁ sea", + "ant ic", + "anti c", + "▁Le ague", + "▁there fore", + "▁ther efore", + "On e", + "O ne", + "Re turn", + "Ret urn", + "R eturn", + "Acc ess", + "Ac cess", + "A ccess", + "va s", + "v as", + "▁о с", + "▁r at", + "▁ra t", + "▁ rat", + "Bi g", + "B ig", + "▁be havior", + "▁behav ior", + "▁behavi or", + "k r", + "▁un defined", + "▁und efined", + "▁ undefined", + "▁E s", + "▁ Es", + "▁appe ared", + "▁appear ed", + "el es", + "ele s", + "e les", + "▁W AR", + "▁WA R", + "▁ WAR", + "St at", + "S tat", + "▁Go ogle", + "▁ Google", + "▁c redit", + "▁cre dit", + "▁cr edit", + "▁cred it", + "▁F ile", + "▁Fil e", + "▁Fi le", + "▁ File", + "an ging", + "ang ing", + "ho use", + "hou se", + "h ouse", + "rom ise", + "ge nt", + "gen t", + "g ent", + "▁hab it", + "▁ha bit", + "▁soc iety", + "▁soci ety", + "▁societ y", + "▁enc our", + "▁p aint", + "▁pain t", + "▁pa int", + "pe t", + "p et", + "▁U K", + "▁ UK", + "aw s", + "a ws", + "on om", + "ono m", + "o nom", + "G l", + "}_ {\\", + "}_{ \\", + "} _{\\", + "el ess", + "ele ss", + "eles s", + "e less", + "em y", + "e my", + "▁C ong", + "▁Con g", + "▁Co ng", + "▁develop ed", + "▁im ages", + "▁image s", + "▁imag es", + "▁ images", + "▁ ö", + "▁f ont", + "▁fo nt", + "▁fon t", + "▁ font", + "cl ear", + "cle ar", + "c lear", + "gi n", + "g in", + "▁L ord", + "▁Lo rd", + "▁Lor d", + "▁trans port", + "▁ transport", + "▁: :", + "▁ ::", + "▁c up", + "▁cu p", + "▁ cup", + "ul ate", + "ula te", + "u late", + "▁D uring", + "▁Du ring", + "▁Dur ing", + "pr iv", + "p riv", + "▁ext rem", + "▁extr em", + "▁D i", + "▁ Di", + "▁d oubt", + "▁dou bt", + "▁doub t", + "P y", + "if ying", + "ify ing", + "sp lit", + "spl it", + "s plit", + "eg o", + "e go", + "git hub", + "g ithub", + "▁) ,", + "▁ ),", + "RO M", + "R OM", + "▁ch air", + "▁cha ir", + "▁ chair", + "▁t rade", + "▁tr ade", + "▁trad e", + "▁tra de", + "▁n icht", + "▁ni cht", + "▁nic ht", + "To p", + "T op", + "St ore", + "▁p arte", + "▁part e", + "▁par te", + "pro ject", + "ni a", + "n ia", + "▁в ід", + "▁ві д", + "wa r", + "w ar", + "▁Pro f", + "▁Pr of", + "▁c aught", + "Th read", + "ст ва", + "ств а", + "с тва", + "aut hor", + "auth or", + "▁d oll", + "▁do ll", + "▁dol l", + "▁h arm", + "▁ha rm", + "▁har m", + "▁ harm", + "▁G en", + "▁Ge n", + "▁ Gen", + "tr ee", + "tre e", + "t ree", + "et ime", + "eti me", + "e time", + "cf g", + "c fg", + "▁gu ys", + "▁guy s", + "▁Cal ifornia", + "▁G reen", + "▁Gr een", + "▁Gre en", + "▁Gree n", + "▁ Green", + "▁mov ement", + "▁move ment", + "▁mo vement", + "ie j", + "i ej", + "▁stat ement", + "▁state ment", + "▁ statement", + "▁se eing", + "▁see ing", + "▁h aven", + "▁have n", + "▁ha ven", + "▁hav en", + "vent ion", + "v ention", + "S L", + "ched ul", + "ie rt", + "ier t", + "i ert", + "▁pr imary", + "▁prim ary", + "▁pri mary", + "▁prima ry", + "▁ primary", + "▁c ivil", + "▁ci vil", + "▁civ il", + "ri an", + "ria n", + "r ian", + "▁b utton", + "▁but ton", + "▁butt on", + "▁ button", + "▁l ived", + "▁li ved", + "▁live d", + "▁liv ed", + "P ass", + "so r", + "s or", + "▁watch ing", + "▁wat ching", + "▁sk ills", + "▁skill s", + "te e", + "t ee", + "Le vel", + "L evel", + "▁sc ient", + "h s", + "▁a gre", + "▁ag re", + "ca t", + "c at", + "▁t end", + "▁te nd", + "▁ten d", + "▁M ill", + "▁Mil l", + "▁Mi ll", + "▁ Mill", + "▁C ap", + "▁Ca p", + "▁ Cap", + "OR D", + "O RD", + "gl e", + "g le", + "▁с во", + "» ,", + "▁a head", + "▁ah ead", + "ve st", + "ves t", + "v est", + "▁J ose", + "▁Jo se", + "▁Jos e", + "is cher", + "isch er", + "ische r", + "isc her", + "ș i", + "▁le aving", + "▁д ля", + "▁s outh", + "▁so uth", + "▁sou th", + "▁sout h", + "▁con sum", + "▁cons um", + "▁ consum", + "R ange", + "▁activ ities", + "Se c", + "S ec", + "▁s ales", + "▁sa les", + "▁sal es", + "▁sale s", + "▁f ix", + "▁fi x", + "▁ fix", + "▁j ed", + "▁je d", + "▁ jed", + "ru m", + "r um", + "ve ctor", + "vec tor", + "v ector", + "▁s pot", + "▁sp ot", + "▁spo t", + "▁ spot", + "▁man ufact", + "к т", + "or row", + "orr ow", + "si gn", + "sig n", + "s ign", + "▁col lege", + "▁colle ge", + "▁colleg e", + "▁d river", + "▁dr iver", + "▁dri ver", + "▁driv er", + "▁drive r", + "▁ driver", + "▁def initely", + "▁definit ely", + "▁s pend", + "▁sp end", + "▁spe nd", + "miss ion", + "m ission", + "з у", + "at ively", + "ative ly", + "ativ ely", + "b i", + "Call back", + "▁particular ly", + "▁particul arly", + "▁h ell", + "▁he ll", + "▁hel l", + "▁ hell", + "▁p ool", + "▁po ol", + "▁ pool", + "PR E", + "P RE", + "▁cle arly", + "▁clear ly", + "P T", + "ot hes", + "oth es", + "othe s", + "▁I d", + "▁ Id", + "Loc ation", + "L ocation", + "▁R un", + "▁Ru n", + "▁ Run", + "▁f ixed", + "▁fix ed", + "▁ fixed", + "▁H and", + "▁Ha nd", + "▁Han d", + "▁ Hand", + "ba l", + "b al", + "d ouble", + "C an", + "Om ega", + "▁chall eng", + "▁stand ing", + "▁stan ding", + "▁ standing", + "it en", + "ite n", + "i ten", + "▁me chan", + "▁d urch", + "▁dur ch", + "▁d ell", + "▁de ll", + "▁del l", + "▁rais ed", + "▁raise d", + "▁ra ised", + "▁we ak", + "▁ weak", + "▁D u", + "▁ Du", + "gr ad", + "gra d", + "g rad", + "▁sc ene", + "▁scen e", + "▁ scene", + "pos s", + "po ss", + "p oss", + "▁t on", + "▁to n", + "▁ ton", + "▁e arth", + "▁ear th", + "ul ations", + "ulation s", + "▁str ength", + "▁stre ngth", + "▁streng th", + "ak ed", + "ake d", + "a ked", + "▁re main", + "▁rem ain", + "▁B i", + "▁ Bi", + "▁custom er", + "▁cust omer", + "▁ customer", + "ran ge", + "r ange", + "▁inter ested", + "▁interest ed", + "ON E", + "O NE", + "▁c off", + "▁co ff", + "re quire", + "requ ire", + "▁On ly", + "▁ Only", + "▁W eb", + "▁We b", + "▁ Web", + "▁f arm", + "▁far m", + "▁fa rm", + "▁act ivity", + "▁activ ity", + "▁ activity", + "▁r out", + "▁ro ut", + "▁rou t", + "bl ing", + "b ling", + "S Y", + "▁Rich ard", + "▁Ric hard", + "▁R ef", + "▁Re f", + "▁ Ref", + "▁ко н", + "▁к он", + "▁ кон", + "▁j un", + "▁ju n", + "bo rn", + "bor n", + "b orn", + "ij n", + "Config uration", + "um an", + "uma n", + "u man", + "E E", + "▁mar ried", + "▁З а", + "▁ За", + "▁f at", + "▁fa t", + "▁k id", + "▁ki d", + "▁T ur", + "▁Tu r", + "▁ Tur", + "▁off ered", + "▁offer ed", + "ni c", + "n ic", + "▁B ig", + "▁Bi g", + "▁ Big", + "Ga mma", + "G amma", + "▁He alth", + "▁ Health", + "▁T R", + "▁ TR", + "▁s ię", + "▁si ę", + "▁const ruction", + "▁construct ion", + "▁constr uction", + "▁constru ction", + "▁ construction", + "▁Ch urch", + "▁B et", + "▁Be t", + "▁ Bet", + "bu s", + "b us", + "▁e arn", + "▁ear n", + "ri ct", + "ric t", + "r ict", + "▁п ра", + "▁пр а", + "▁ пра", + "▁br ain", + "▁bra in", + "▁f ra", + "▁fr a", + "▁O p", + "▁ Op", + "FI G", + "F IG", + "em a", + "e ma", + "▁Europe an", + "▁S aint", + "▁Sa int", + "▁ Saint", + "AR E", + "A RE", + "ur i", + "u ri", + "▁R iver", + "{ }", + "▁s itting", + "▁sit ting", + "▁under standing", + "▁understand ing", + "▁pl ans", + "▁plan s", + "rop ri", + "▁old er", + "▁ol der", + "▁ older", + "▁pres sure", + "▁press ure", + "Im pl", + "Imp l", + "▁pe ace", + "Conne ction", + "Conn ection", + "Connect ion", + "▁f i", + "▁ fi", + "ri ch", + "ric h", + "r ich", + "▁sh ut", + "ap ers", + "ape rs", + "aper s", + "a pers", + "Po rt", + "P ort", + "▁L ook", + "▁Lo ok", + "▁ Look", + "ri m", + "r im", + "au th", + "aut h", + "a uth", + "au to", + "aut o", + "a uto", + "▁high ly", + "▁un less", + "▁W al", + "▁Wa l", + "▁re n", + "▁r en", + "▁ ren", + "w s", + "▁c ore", + "▁co re", + "▁cor e", + "▁ core", + "( -", + "▁c lim", + "▁cl im", + "ru it", + "r uit", + "▁call back", + "▁ callback", + "he st", + "hes t", + "h est", + "▁Char les", + "▁Charl es", + "▁L ong", + "▁Lo ng", + "▁ Long", + "} =", + "ъ р", + "▁sh ared", + "▁share d", + "▁shar ed", + "▁sha red", + "▁ shared", + "ul ated", + "ula ted", + "ulate d", + "gorith m", + "▁H ome", + "▁Ho me", + "▁Hom e", + "▁ Home", + "▁vill age", + "▁vil lage", + "ee s", + "e es", + "s v", + "▁rest aur", + "re y", + "r ey", + "▁C ast", + "▁Cas t", + "▁Ca st", + "▁ Cast", + "▁P erson", + "▁Per son", + "▁Pers on", + "▁ Person", + "ки й", + "▁organ iz", + "▁R ad", + "▁Ra d", + "▁ Rad", + "pon ents", + "ponent s", + "▁wer den", + "▁werd en", + "▁b ow", + "▁bo w", + "▁ bow", + "se n", + "s en", + "am i", + "a mi", + "Inter face", + "▁b asis", + "▁bas is", + "▁ba sis", + "▁Comp any", + "▁Compan y", + "▁ Company", + "er nel", + "ern el", + "erne l", + "it u", + "i tu", + "Has h", + "Ha sh", + "H ash", + "▁a an", + "▁ х", + "▁s mile", + "▁sm ile", + "x ml", + "▁s cen", + "▁sc en", + "am m", + "a mm", + "to ol", + "too l", + "t ool", + "ar ia", + "ari a", + "a ria", + "▁acc ur", + "▁ac cur", + "▁ accur", + "set tings", + "setting s", + "▁Jes us", + "ac ement", + "ace ment", + "po wer", + "pow er", + "p ower", + "( !", + "▁c alls", + "▁call s", + "▁cal ls", + "▁ calls", + "▁bas ic", + "▁ basic", + "▁set tings", + "▁sett ings", + "▁setting s", + "▁ settings", + "ri pt", + "rip t", + "r ipt", + "po ol", + "p ool", + "ct ors", + "ctor s", + "▁Found ation", + "▁ Foundation", + "▁we ap", + "KE Y", + "K EY", + "fo ot", + "foo t", + "f oot", + "▁r adio", + "▁rad io", + "▁radi o", + "▁ radio", + "▁hel ped", + "▁help ed", + "ma nn", + "man n", + "m ann", + "▁j ump", + "▁ju mp", + "▁t ick", + "▁ti ck", + "▁ tick", + "▁gr owing", + "▁grow ing", + "▁gro wing", + "at en", + "ate n", + "a ten", + "re al", + "rea l", + "▁incre asing", + "Dev ice", + "var epsilon", + "vare psilon", + "▁s ets", + "▁se ts", + "▁set s", + "▁ sets", + "▁adv ant", + "Op en", + "O pen", + "▁re asons", + "▁reason s", + "▁sup posed", + "▁supp osed", + "▁suppose d", + "oe s", + "o es", + "ed e", + "e de", + "te en", + "tee n", + "t een", + "if def", + "▁de lete", + "▁del ete", + "▁delet e", + "▁ delete", + "▁& =", + "▁ &=", + "▁B ill", + "▁Bi ll", + "▁Bil l", + "▁ Bill", + "▁a im", + "▁ai m", + "▁ aim", + "▁O k", + "▁ Ok", + "▁A v", + "▁ Av", + "re ci", + "rec i", + "ac ks", + "ack s", + "a cks", + "is te", + "ist e", + "i ste", + "Pro perties", + "▁t mp", + "▁tm p", + "▁ tmp", + "▁d ei", + "▁de i", + "PE R", + "P ER", + "D C", + "st a", + "s ta", + "ни и", + "▁lim ited", + "▁limit ed", + "▁ limited", + "▁great er", + "▁gre ater", + "de scription", + "des cription", + "or i", + "o ri", + "ain ts", + "aint s", + "▁h y", + "▁ hy", + "▁M el", + "▁Me l", + "▁C H", + "▁ CH", + "con s", + "co ns", + "c ons", + "▁sur round", + "▁W ho", + "▁Wh o", + "▁ Who", + "ar c", + "a rc", + "▁te lev", + "▁tele v", + "▁tel ev", + "it ution", + "itut ion", + "▁e qual", + "▁equ al", + "▁eq ual", + "▁ equal", + "к і", + "▁Is rael", + "ä h", + "▁C aption", + "▁Capt ion", + "▁Ca ption", + "▁ex erc", + "em por", + "emp or", + "▁+ +", + "▁ ++", + "▁l ib", + "▁li b", + "▁ lib", + "ma ke", + "m ake", + "▁M A", + "▁ MA", + "co py", + "cop y", + "c opy", + "f riend", + "▁ко то", + "▁ кото", + "▁dam age", + "▁\\ ,", + "▁ \\,", + "od ed", + "ode d", + "o ded", + "▁n one", + "▁no ne", + "▁non e", + "▁ none", + "▁ev alu", + "▁eval u", + "▁ evalu", + "st on", + "sto n", + "s ton", + "> ,", + "FO R", + "F OR", + "▁n orm", + "▁no rm", + "▁nor m", + "▁ norm", + "ap pe", + "app e", + "a ppe", + "S ession", + "▁ad ult", + "▁h ospital", + "▁hosp ital", + "▁recomm end", + "pro perty", + "ste in", + "fin al", + "fi nal", + "f inal", + "▁n u", + "▁ nu", + "se cond", + "sec ond", + "▁a spect", + "▁as pect", + "▁asp ect", + "\") ]", + "\" )]", + "же н", + "ж ен", + "am ento", + "ament o", + "amen to", + "▁r ac", + "▁ra c", + "▁ rac", + "sa ve", + "s ave", + "▁foot ball", + "A b", + "un gs", + "ung s", + "ab il", + "abi l", + "a bil", + "▁Ar ch", + "▁Arc h", + "▁ Arch", + "sys tem", + "s ystem", + "hi st", + "his t", + "h ist", + "▁l uck", + "▁lu ck", + "▁luc k", + "re nder", + "ren der", + "rend er", + "r ender", + "▁se in", + "▁sei n", + "ion i", + "io ni", + "i oni", + "▁r ot", + "▁ro t", + "▁ rot", + "▁cor ner", + "▁corn er", + "▁app ropri", + "▁ap propri", + "▁ appropri", + "▁Soft ware", + "▁t ele", + "▁te le", + "▁tel e", + "▁ tele", + "De lete", + "Dele te", + "Del ete", + "▁Acc ording", + "▁pr ison", + "▁pri son", + "▁ prison", + "▁l ic", + "▁li c", + "▁ lic", + "▁м и", + "▁ ми", + "ter m", + "te rm", + "t erm", + "se ts", + "set s", + "s ets", + "▁v el", + "▁ve l", + "▁ vel", + "▁r ank", + "▁ran k", + "▁ rank", + "▁ex isting", + "▁exist ing", + "▁ existing", + "▁V ir", + "▁Vi r", + "▁t rip", + "▁tr ip", + "▁tri p", + "▁м у", + "▁ му", + "av ax", + "ava x", + "▁r is", + "▁ri s", + "▁ ris", + "▁def ine", + "▁defin e", + "▁ define", + "▁he at", + "ca r", + "c ar", + "▁con vert", + "▁conv ert", + "▁conver t", + "▁conve rt", + "▁ convert", + "em ail", + "ema il", + "e mail", + "▁U nder", + "▁Un der", + "▁Und er", + "▁ Under", + "▁ Ш", + "▁G rand", + "▁Gr and", + "▁Gran d", + "▁Gra nd", + "▁ex ists", + "▁exist s", + "▁ exists", + "sy s", + "s ys", + "ef f", + "e ff", + "▁T op", + "▁To p", + "▁ Top", + "▁ č", + "▁t empor", + "▁tem por", + "▁temp or", + "▁tempo r", + "▁arg uments", + "▁argument s", + "▁ arguments", + "▁support ed", + "▁supp orted", + "▁ supported", + "en sed", + "ens ed", + "ense d", + "▁Franc is", + "▁co ord", + "▁ coord", + "▁achie ve", + "▁N ame", + "▁Na me", + "▁Nam e", + "▁ Name", + "▁J ahr", + "▁Jah r", + "▁Ja hr", + "▁G i", + "sh e", + "s he", + "▁D ev", + "▁De v", + "▁ Dev", + "▁a lla", + "▁al la", + "▁all a", + "▁ alla", + "▁W IT", + "ag ment", + "c ustom", + "al ls", + "all s", + "& &", + "W E", + "▁h olding", + "▁hold ing", + "▁hol ding", + "pro totype", + "proto type", + "prot otype", + "▁f ing", + "▁fin g", + "▁fi ng", + "▁b ag", + "▁ba g", + "▁ bag", + "▁Par ty", + "▁Part y", + "st ack", + "sta ck", + "▁econom ic", + "▁G al", + "▁Ga l", + "id ents", + "ident s", + "iden ts", + "▁J un", + "▁Ju n", + "▁sh owed", + "▁show ed", + "os h", + "o sh", + "▁B ay", + "▁Ba y", + "▁ Bay", + "ma il", + "m ail", + "▁S O", + "▁ SO", + "▁\" <", + "graph ics", + "▁f u", + "▁ fu", + "cl ick", + "cli ck", + "c lick", + "▁b attle", + "▁batt le", + "▁bat tle", + "{ {", + "▁E vent", + "▁Even t", + "▁Ev ent", + "▁Eve nt", + "▁ Event", + "ri or", + "rio r", + "r ior", + "ch aft", + "cha ft", + "▁f avorite", + "▁favor ite", + "us ive", + "sup port", + "supp ort", + "s upport", + "b m", + "K ind", + "▁saf ety", + "▁safe ty", + "▁E nt", + "▁En t", + "▁ Ent", + "cu p", + "c up", + "▁Austral ia", + "▁dest roy", + "▁destro y", + "▁ destroy", + "▁organ ization", + "▁organiz ation", + "id en", + "ide n", + "i den", + "######## ########", + "de c", + "d ec", + "▁z a", + "▁ za", + "▁s even", + "▁se ven", + "▁ seven", + "ar ely", + "are ly", + "arel y", + "▁f lag", + "▁fl ag", + "▁ flag", + "Di r", + "D ir", + "▁C arl", + "▁Car l", + "▁Ca rl", + "▁do ctor", + "▁doc tor", + "▁var iety", + "▁vari ety", + "▁L in", + "▁Li n", + "▁ Lin", + "▁t om", + "▁to m", + "▁ tom", + "^{ (", + "^ {(", + "B o", + "an tes", + "ant es", + "ante s", + "▁m ine", + "▁min e", + "▁mi ne", + "▁ mine", + "▁M it", + "▁Mi t", + "▁de scribe", + "▁desc ribe", + "▁describ e", + "Ar gs", + "Arg s", + "L S", + "AP I", + "A PI", + "▁L uc", + "▁Lu c", + "▁ Luc", + "ph one", + "▁sc ience", + "▁ science", + "▁O per", + "▁Op er", + "▁ Oper", + "Ne xt", + "N ext", + "▁invest ig", + "▁demon str", + "▁G overn", + "▁Go vern", + "▁object s", + "▁ objects", + "▁Lou is", + "▁Lo uis", + "▁Return s", + "▁ Returns", + "▁h an", + "▁ha n", + "▁ han", + "na m", + "n am", + "▁com me", + "▁comm e", + "▁pres ence", + "▁p el", + "▁pe l", + "▁ pel", + "▁det ect", + "▁ detect", + ") =", + "▁Ch inese", + "▁r ich", + "▁ri ch", + "▁ric h", + "▁ rich", + "▁class es", + "▁classe s", + "▁clas ses", + "▁ classes", + "▁exp and", + "▁ expand", + "▁D om", + "▁Do m", + "▁ Dom", + "▁D ec", + "▁De c", + "▁ Dec", + "s n", + "pe ed", + "p eed", + "▁J im", + "▁Ji m", + "sh ould", + "▁Sm ith", + "▁p ages", + "▁page s", + "▁pa ges", + "▁pag es", + "▁ pages", + "▁Je an", + "ri cs", + "ric s", + "r ics", + "▁S und", + "▁Su nd", + "▁Sun d", + "ad s", + "a ds", + "▁The ir", + "un icip", + "uni cip", + "unic ip", + "в у", + "▁down load", + "▁ download", + "▁st ress", + "▁str ess", + "▁stre ss", + "▁P et", + "▁Pe t", + "▁ Pet", + "me nu", + "men u", + "m enu", + "re me", + "rem e", + "r eme", + "▁com pared", + "▁comp ared", + "▁compar ed", + "▁compare d", + "St e", + "S te", + "IN D", + "I ND", + "cont ainer", + "▁Ind ian", + "▁India n", + "or en", + "ore n", + "o ren", + "▁s es", + "▁se s", + "▁ ses", + "▁W he", + "▁Wh e", + "▁ Whe", + "▁r oku", + "▁ro ku", + "▁estab lished", + "▁establish ed", + "▁gener ally", + "▁general ly", + "▁f le", + "▁fl e", + "__ (", + "_ _(", + "=\" +", + "= \"+", + "V ar", + "▁M ake", + "▁Ma ke", + "▁Mak e", + "▁ Make", + "▁rem oved", + "▁remove d", + "▁ removed", + "z z", + "ü n", + "▁m ix", + "▁mi x", + "▁ mix", + "er k", + "iat ion", + "i ation", + "ou ter", + "out er", + "oute r", + "o uter", + "S K", + "▁be comes", + "▁bec omes", + "▁become s", + "▁H all", + "▁Ha ll", + "▁Hal l", + "sc ious", + "▁w atched", + "▁watch ed", + "▁wat ched", + "▁g ather", + "▁ga ther", + "▁ gather", + "▁Res ult", + "▁ Result", + "pro of", + "pa y", + "p ay", + "▁produ ced", + "▁produce d", + "▁prod uced", + "▁| =", + "▁b order", + "▁bord er", + "▁bor der", + "▁ border", + "▁d in", + "▁di n", + "▁s cript", + "▁sc ript", + "▁scr ipt", + "▁ script", + "▁a ctions", + "▁act ions", + "▁action s", + "▁ actions", + "▁m as", + "▁ma s", + "▁ mas", + "щ а", + "oot h", + "oo th", + "o oth", + "▁Te chn", + "▁Tech n", + "Js on", + "J son", + "▁f illed", + "▁fil led", + "▁fill ed", + "▁ filled", + "де н", + "д ен", + "und le", + "ст у", + "с ту", + "To ol", + "Too l", + "T ool", + "▁k ing", + "▁ki ng", + "▁kin g", + "▁ king", + "▁v en", + "▁ve n", + "▁ ven", + "st ra", + "str a", + "s tra", + "▁pre dict", + "▁pred ict", + "▁ predict", + "▁l ui", + "▁lu i", + "▁WAR RAN", + "▁F un", + "▁Fu n", + "▁ Fun", + "Sc ript", + "S cript", + "▁power ful", + "▁l ose", + "▁lo se", + "▁los e", + "at ically", + "atic ally", + "▁d aily", + "▁da ily", + "▁dai ly", + "▁r ing", + "▁ri ng", + "▁ ring", + "▁ar rived", + "▁arriv ed", + "▁arr ived", + "▁arrive d", + "St ack", + "sc ope", + "s cope", + "▁B ack", + "▁Ba ck", + "▁ Back", + "el ij", + "eli j", + "e lij", + "▁z e", + "▁ ze", + "ke ys", + "key s", + "{ \"", + "VI D", + "V ID", + "▁l icense", + "▁lic ense", + "▁ license", + "wh at", + "w hat", + "▁pro ced", + "▁proc ed", + "ra nt", + "ran t", + "r ant", + "est ival", + "ag ram", + "agr am", + "agra m", + "a gram", + "▁L O", + "▁ LO", + "▁Hen ry", + "▁fl ags", + "▁flag s", + "▁ flags", + "Do wn", + "D own", + "scri ption", + "script ion", + "s cription", + "▁famil ies", + "▁familie s", + "is se", + "iss e", + "bo ur", + "b our", + "▁B ur", + "▁Bu r", + "— \"", + "▁b rief", + "▁br ief", + "▁ brief", + "▁cre ating", + "▁creat ing", + "▁cl ients", + "▁client s", + "ran gle", + "r angle", + "▁amaz ing", + "▁s ind", + "▁si nd", + "▁sin d", + "▁cover ed", + "▁cov ered", + "▁ covered", + "We ll", + "W ell", + "ст е", + "с те", + "то р", + "т ор", + "▁B as", + "▁Ba s", + "▁ Bas", + "to tal", + "tot al", + "t otal", + "▁I nit", + "▁In it", + "▁ Init", + "▁s and", + "▁sa nd", + "▁san d", + "Un it", + "U nit", + "▁mur der", + "▁b right", + "▁br ight", + "▁brig ht", + "▁t rav", + "▁tr av", + "▁tra v", + "ic ans", + "ica ns", + "ican s", + "▁att ribute", + "▁attribut e", + "▁ attribute", + "f c", + "▁pl aced", + "▁place d", + "▁plac ed", + "ES T", + "E ST", + "Var i", + "V ari", + "▁c os", + "▁co s", + "▁ cos", + "▁at tract", + "▁att ract", + "▁attr act", + "▁attra ct", + "an el", + "ane l", + "a nel", + "}) .", + "} ).", + "by tes", + "byte s", + "▁p arse", + "▁par se", + "▁ parse", + "▁be long", + "▁bel ong", + "B N", + "▁S ol", + "▁So l", + "P o", + "` ,", + "▁c alling", + "▁call ing", + "▁cal ling", + "▁? >", + "▁ ?>", + "▁it er", + "▁i ter", + "▁ iter", + "▁u rl", + "▁ur l", + "▁ url", + "▁ev ening", + "▁even ing", + "re ek", + "ree k", + "▁hon est", + "▁direct or", + "▁dire ctor", + "▁dir ector", + "R C", + "▁s olid", + "▁sol id", + "▁ solid", + "▁ph il", + "ie ne", + "ien e", + "i ene", + "FA ULT", + "co pe", + "cop e", + "c ope", + "▁Hist ory", + "▁Histor y", + "▁Hi story", + "▁ History", + "▁Te am", + "▁ Team", + "ree dom", + "reed om", + "▁r u", + "▁ ru", + "U B", + "▁w orse", + "▁wor se", + "im o", + "i mo", + "Ma t", + "M at", + "▁M ex", + "▁Me x", + "ac tor", + "act or", + "a ctor", + "▁v or", + "▁vo r", + "▁ vor", + "ть ся", + "▁exper iment", + "▁experi ment", + "▁P lay", + "▁Pl ay", + "▁ Play", + "▁An other", + "▁happ ens", + "▁happen s", + "ua n", + "u an", + "▁pat ients", + "▁patient s", + "▁re nd", + "▁r end", + "▁ren d", + "▁ rend", + "▁M o", + "▁ Mo", + "▁T ex", + "▁Te x", + "▁ Tex", + "▁w ed", + "▁we d", + "▁ wed", + "t n", + "in sert", + "ins ert", + "▁п а", + "▁ па", + "▁an ti", + "▁ant i", + "▁ anti", + "Mat ch", + "M atch", + "ampions hip", + "ampion ship", + "▁for ces", + "▁force s", + "▁H ot", + "▁Ho t", + "▁ Hot", + "▁ph ase", + "▁ phase", + "▁t emplate", + "▁templ ate", + "▁temp late", + "▁ template", + "st op", + "sto p", + "s top", + "ic ated", + "ica ted", + "icate d", + "▁man aged", + "▁manage d", + "▁ managed", + "wa it", + "w ait", + "▁* (", + "▁ *(", + "G B", + "▁app oint", + "▁ap point", + "▁ appoint", + "ł a", + "▁s tick", + "▁st ick", + "▁ stick", + "▁F OR", + "▁FO R", + "▁ FOR", + "▁V is", + "▁Vi s", + "▁ Vis", + "to r", + "t or", + "▁p ř", + "qu est", + "que st", + "ques t", + "q uest", + "us es", + "use s", + "u ses", + "\"); \r", + "\") ;\r", + "\" );\r", + "▁sudden ly", + "▁sud denly", + "é c", + "N D", + "ur op", + "uro p", + "u rop", + "ре д", + "▁ins urance", + "ac cess", + "acc ess", + "a ccess", + "un finished", + "▁t amb", + "▁ta mb", + "▁tam b", + "▁s ac", + "▁sa c", + "▁C ourt", + "▁Co urt", + "▁Cour t", + "▁Cou rt", + "▁miss ing", + "▁mis sing", + "▁ missing", + "▁W here", + "▁Wh ere", + "▁Whe re", + "▁ Where", + "▁S um", + "▁Su m", + "▁ Sum", + "}^ {\\", + "}^{ \\", + "} ^{\\", + "▁s ua", + "▁su a", + "_ ,", + "▁th ick", + "▁Tr ump", + "▁Tru mp", + "▁oper ations", + "▁operation s", + "▁ operations", + "F S", + "▁de ux", + "d z", + "Temp late", + "T emplate", + "▁\" /", + "▁o dd", + "▁od d", + "▁ odd", + "▁re ality", + "▁real ity", + "▁te ams", + "▁team s", + "▁tea ms", + "▁c er", + "▁ce r", + "▁ cer", + "om a", + "o ma", + "▁ și", + "▁cl oud", + "▁clo ud", + "▁ cloud", + "▁Dep artment", + "N e", + "▁requ ires", + "▁require s", + "it ems", + "ite ms", + "item s", + "▁I II", + "▁II I", + "▁ III", + "right arrow", + ")- >", + ") ->", + "▁w riter", + "▁wr iter", + "▁writ er", + "▁write r", + "▁ writer", + "re place", + "rep lace", + "▁t hr", + "▁th r", + "je n", + "j en", + "▁o t", + "▁ ot", + "▁occ up", + "▁oc cup", + "▁ occup", + "▁event ually", + "▁M ath", + "▁Mat h", + "▁Ma th", + "▁ Math", + "▁con serv", + "▁cons erv", + "▁conse rv", + "am er", + "ame r", + "a mer", + "▁F ort", + "▁For t", + "▁Fo rt", + "▁d ry", + "▁dr y", + "▁sex ual", + "▁co sts", + "▁cost s", + "▁cos ts", + "▁for ms", + "▁form s", + "▁ forms", + "▁V ict", + "▁Vi ct", + "▁Vic t", + "PA R", + "P AR", + "frame work", + "▁д и", + "▁ ди", + "Oper ation", + "з на", + "wh ich", + "▁t ight", + "▁ti ght", + "In valid", + "▁part ner", + "▁п ред", + "▁пре д", + "▁th ank", + "▁than k", + "▁gu ard", + "▁ guard", + "he m", + "h em", + "Bo dy", + "B ody", + "▁e mot", + "▁em ot", + "I X", + "fa st", + "fas t", + "f ast", + "щ о", + "ñ o", + "ni ght", + "n ight", + "▁S ci", + "▁Sc i", + "ни ка", + "ник а", + "▁T O", + "▁ TO", + "▁individ uals", + "▁individual s", + "сс и", + "с си", + "}) ,", + "} ),", + "F alse", + "(\" %", + "( \"%", + "▁op tim", + "▁opt im", + "▁ optim", + "▁- ->", + "▁-- >", + "▁ -->", + "▁f actor", + "▁fact or", + "▁fac tor", + "▁fa ctor", + "▁ factor", + "▁sm aller", + "▁small er", + "▁con tain", + "▁cont ain", + "sp ect", + "spec t", + "spe ct", + "s pect", + "Eng ine", + "▁ann ounced", + "▁announ ced", + "▁announce d", + "▁Dem ocr", + "▁r ob", + "▁ro b", + "▁ rob", + "▁f lat", + "▁fl at", + "▁ flat", + "os oph", + "oso ph", + "Se arch", + "S earch", + "ah l", + "a hl", + "▁Ex ception", + "▁Except ion", + "▁ Exception", + "▁O l", + "equ als", + "eq uals", + "equal s", + "▁un ter", + "▁unt er", + "▁ unter", + "sh ape", + "sha pe", + "N S", + "Ob j", + "▁spec ies", + "▁spe cies", + "we ight", + "wei ght", + "w eight", + "yo u", + "y ou", + "▁e ste", + "▁est e", + "▁es te", + "▁ este", + "▁V iew", + "▁Vi ew", + "▁ View", + "▁m ission", + "▁miss ion", + "▁ mission", + "▁j ournal", + "▁jour nal", + "▁ journal", + "Value s", + "Val ues", + "▁ein em", + "▁eine m", + "is mo", + "ism o", + "▁project s", + "▁ projects", + "▁D as", + "▁Da s", + "ri ble", + "rib le", + "r ible", + "▁s erve", + "▁ser ve", + "▁serv e", + "▁ serve", + "▁op ening", + "▁open ing", + "▁h ur", + "▁program s", + "▁U SA", + "▁US A", + "▁ USA", + "il iar", + "ili ar", + "ilia r", + "id os", + "ido s", + "B r", + "est amp", + "esta mp", + "▁t ools", + "▁to ols", + "▁too ls", + "▁tool s", + "▁ tools", + "an ner", + "ann er", + "anne r", + "R T", + "▁St art", + "▁Star t", + "▁Sta rt", + "▁ Start", + "▁b ath", + "▁bat h", + "▁ba th", + "▁coff ee", + "or ter", + "ort er", + "orte r", + "in ternal", + "inter nal", + "intern al", + "file s", + "fil es", + "fi les", + "f iles", + "IN VAL", + "ak o", + "a ko", + "d t", + "▁Se cond", + "▁Sec ond", + "▁ Second", + "▁al loc", + "▁all oc", + "▁ alloc", + "▁en ded", + "▁end ed", + "▁ende d", + "▁ ended", + "ac ional", + "aci onal", + "acion al", + "acio nal", + "▁man ager", + "▁manage r", + "▁ manager", + "▁S un", + "▁Su n", + "▁ Sun", + "ag g", + "a gg", + "▁le ader", + "▁lead er", + "ol ved", + "olve d", + "olv ed", + "▁ч то", + "▁trad itional", + "▁tradition al", + "sh ot", + "s hot", + "ru p", + "r up", + "C F", + "▁E ach", + "▁ Each", + "w r", + "▁S om", + "▁So m", + "▁ Som", + "▁material s", + "▁mater ials", + "▁m sg", + "▁ms g", + "▁ msg", + "▁s yn", + "▁sy n", + "▁ syn", + "▁produ ce", + "▁prod uce", + "▁st orage", + "▁stor age", + "▁sto rage", + "▁ storage", + "sub section", + "▁S ie", + "▁Si e", + "▁I P", + "▁ IP", + "CE SS", + "▁w a", + "▁ wa", + "Re cord", + "Rec ord", + "▁mark eting", + "▁market ing", + "pl et", + "ple t", + "p let", + "D ialog", + "▁mention ed", + "▁ment ioned", + "▁N a", + "▁ Na", + "▁Un ion", + "▁ Union", + "▁A PI", + "▁AP I", + "▁ API", + "▁neg ative", + "▁ negative", + "tx t", + "t xt", + "▁eas ier", + "le gal", + "leg al", + "De p", + "D ep", + "▁no vel", + "▁nov el", + "▁nove l", + "eu r", + "e ur", + "ac ió", + "aci ó", + "a ció", + "▁B ud", + "▁Bu d", + "▁c arry", + "▁car ry", + "sch aft", + "s chaft", + "▁br oken", + "▁bro ken", + "▁broke n", + "▁t rees", + "▁tr ees", + "▁tre es", + "▁tree s", + ">( );", + ">() ;", + "> ();", + "▁e mb", + "▁em b", + "▁ emb", + "ie der", + "ied er", + "i eder", + "▁r oute", + "▁ro ute", + "▁rout e", + "▁rou te", + "▁ route", + "ik el", + "ike l", + "i kel", + "▁l isten", + "▁li sten", + "▁list en", + "▁ listen", + "ash ion", + "ashi on", + "▁M rs", + "▁Mr s", + "▁equip ment", + "ag ger", + "agg er", + "▁T hus", + "▁Th us", + "▁mat rix", + "▁ matrix", + "al la", + "all a", + "a lla", + "▁T our", + "▁To ur", + "▁con versation", + "▁convers ation", + "Mo n", + "M on", + "our nal", + "▁min ute", + "▁minut e", + "▁ minute", + "A m", + "Ap i", + "A pi", + "▁for get", + "▁forg et", + "M e", + "lev ant", + "te mp", + "tem p", + "t emp", + "▁t elling", + "▁tell ing", + "▁tel ling", + "mo ve", + "mov e", + "m ove", + "▁in dependent", + "▁independ ent", + "to String", + "ed it", + "edi t", + "e dit", + "▁J ac", + "▁Ja c", + "az z", + "a zz", + "re act", + "rea ct", + "▁c in", + "▁ci n", + "▁ cin", + "▁P rov", + "▁Pro v", + "▁Pr ov", + "▁ Prov", + "is ted", + "ist ed", + "iste d", + "i sted", + "▁h ash", + "▁has h", + "▁ha sh", + "▁ hash", + "on na", + "ik i", + "i ki", + "▁gener ated", + "▁generate d", + "▁gene rated", + "▁ generated", + "Re nder", + "Rend er", + "R ender", + "▁psy ch", + "▁ps ych", + "na v", + "n av", + "▁en tr", + "▁ent r", + "▁ entr", + "п ра", + "r x", + "AT H", + "A TH", + "▁ass ume", + "▁assum e", + "Tr ee", + "T ree", + "semb ly", + "sembl y", + "▁M att", + "▁Mat t", + "▁Ma tt", + "ca ption", + "c aption", + "▁s olutions", + "▁solution s", + "▁fa ith", + "▁fait h", + "▁dig ital", + "▁digit al", + "▁ex cell", + "▁exc ell", + "▁V ersion", + "▁Vers ion", + "▁ Version", + "De bug", + "D ebug", + "▁ж и", + "▁ жи", + "▁car ried", + "re set", + "res et", + "▁slow ly", + "an cing", + "anc ing", + "▁own er", + "▁ owner", + "▁T er", + "▁Te r", + "▁D id", + "▁Di d", + "▁ Did", + "▁g est", + "▁ge st", + "▁ges t", + "▁ gest", + "▁é té", + "▁ét é", + "▁ été", + "▁pro of", + "▁ proof", + "F ont", + "▁n ob", + "▁no b", + "▁ nob", + "C o", + "▁G NU", + "▁l iber", + "▁li ber", + "▁lib er", + "it ness", + "▁h ij", + "▁hi j", + "▁v ert", + "▁ver t", + "▁ve rt", + "▁ vert", + "ш а", + "FL AG", + "ME NT", + "M ENT", + "▁S on", + "▁So n", + "Mu lt", + "M ult", + "▁d istrict", + "▁di strict", + "▁dist rict", + "conne ct", + "conn ect", + "ject ion", + "je ction", + "j ection", + "ly mp", + "▁real ized", + "▁realize d", + "▁realiz ed", + "mo s", + "m os", + "y e", + "▁re nder", + "▁r ender", + "▁ren der", + "▁rend er", + "▁ render", + "ri o", + "r io", + "▁inter pret", + "▁ interpret", + "▁slight ly", + "fi x", + "f ix", + "▁stud ies", + "▁r id", + "▁ri d", + "▁ rid", + "at re", + "atr e", + "a tre", + "▁benef its", + "▁benefit s", + "▁F ace", + "▁Fa ce", + "▁Fac e", + "▁ Face", + "iv ery", + "ive ry", + "iver y", + "i very", + "ри я", + "doc ument", + "d ocument", + "▁as king", + "▁ask ing", + "La st", + "L ast", + "ar ante", + "ara nte", + "aran te", + "▁Mart in", + "▁E ll", + "▁El l", + "▁v ector", + "▁ve ctor", + "▁vec tor", + "▁ vector", + "▁for ced", + "▁force d", + "▁ forced", + "о ло", + "P H", + "W R", + "▁K l", + "▁s ky", + "▁sk y", + "▁ sky", + "▁str ategy", + "▁strateg y", + "▁strat egy", + "oc ked", + "ock ed", + "▁ne ck", + "ś ci", + "O UT", + ")) ,", + ") ),", + "C ustom", + "▁w ie", + "▁ wie", + "▁s weet", + "▁swe et", + "▁t emp", + "▁te mp", + "▁tem p", + "▁ temp", + "▁fore ign", + "▁h all", + "▁ha ll", + "▁hal l", + "▁ hall", + "as tr", + "ast r", + "a str", + "As s", + "A ss", + "MO DE", + "MOD E", + "▁max imum", + "▁maxim um", + "an nels", + "ann els", + "annel s", + "anne ls", + "▁t ip", + "▁ti p", + "▁ tip", + "▁second s", + "▁sec onds", + "▁ seconds", + "▁st ack", + "▁sta ck", + "▁ stack", + "ig a", + "i ga", + "▁r aise", + "▁rais e", + "▁ra ise", + "▁ raise", + "en able", + "ena ble", + "oi r", + "o ir", + "▁s oul", + "▁so ul", + "▁sou l", + "K e", + ")$ .", + ") $.", + "▁T im", + "▁Ti m", + "▁ Tim", + "AL SE", + "is er", + "ise r", + "i ser", + "cont in", + "be l", + "b el", + "▁m ad", + "▁ma d", + "▁ mad", + "lic hen", + "li chen", + "lich en", + "liche n", + "l ichen", + "ab e", + "a be", + "sa fe", + "▁con cent", + "▁conc ent", + "▁conce nt", + "bo und", + "b ound", + "▁R equ", + "▁Re qu", + "▁ Requ", + "sw itch", + "▁st one", + "▁sto ne", + "▁ stone", + "▁trans l", + "▁ transl", + "▁v ac", + "▁va c", + "an don", + "and on", + "ando n", + "▁F ore", + "▁For e", + "▁Fo re", + "▁ Fore", + "▁s ounds", + "▁sound s", + "▁P op", + "▁Po p", + "▁ Pop", + "▁H T", + "▁ HT", + "li a", + "l ia", + "en ter", + "ent er", + "ente r", + "▁hel ps", + "▁help s", + "ed y", + "e dy", + "ст вен", + "ств ен", + "стве н", + "an ted", + "ant ed", + "ante d", + "▁I ts", + "▁It s", + "▁St ep", + "▁Ste p", + "▁ Step", + "I con", + "▁EX PECT", + "▁ EXPECT", + "ial ized", + "ialize d", + "Pos t", + "Po st", + "P ost", + "az e", + "a ze", + "▁Car ol", + "▁Ca rol", + "▁re q", + "▁r eq", + "▁ req", + "▁crit ical", + "▁critic al", + "D S", + "▁se at", + "▁sea t", + "ap ed", + "ape d", + "a ped", + "▁up per", + "▁upp er", + "▁ upper", + "▁S y", + "▁ Sy", + "▁ex plain", + "▁expl ain", + "▁' ./", + "▁'. /", + "ut ils", + "util s", + "uti ls", + "poss ible", + "▁d ont", + "▁do nt", + "▁don t", + "H ost", + "▁appro xim", + "▁approx im", + "As ync", + "A sync", + "▁g rab", + "▁gr ab", + "▁gra b", + "▁s ources", + "▁source s", + "▁sour ces", + "▁ sources", + "▁M os", + "▁Mo s", + "▁Germ any", + "▁German y", + "▁Ger many", + "▁r ub", + "▁ru b", + "▁ rub", + "CH AN", + "▁r ain", + "▁ra in", + "▁tr uly", + "▁join ed", + "▁jo ined", + "▁< ?", + "▁ ", + "_ ->", + "ag nost", + "agn ost", + "▁pro posed", + "▁prop osed", + "▁propos ed", + "▁propose d", + "▁G ame", + "▁Ga me", + "▁Gam e", + "▁ Game", + "▁eff orts", + "▁effort s", + "в я", + "t c", + "с к", + "▁int ent", + "▁inte nt", + "▁ intent", + "▁B re", + "▁Br e", + "is c", + "i sc", + "▁pro test", + "▁prote st", + "▁prot est", + "▁h olds", + "▁hold s", + "▁hol ds", + "▁ holds", + "om etry", + "ome try", + "omet ry", + "o metry", + "▁H ave", + "▁Ha ve", + "▁Hav e", + "▁ Have", + "▁de tail", + "▁det ail", + "▁ detail", + "▁WIT HOUT", + "▁WITH OUT", + "ye r", + "y er", + "▁K on", + "▁Ko n", + "▁not iced", + "▁notice d", + "▁require ments", + "▁requirement s", + "DE BUG", + "ki ns", + "kin s", + "k ins", + "▁S pan", + "▁Sp an", + "▁ Span", + "▁c ars", + "▁car s", + "▁ca rs", + "me ta", + "met a", + "m eta", + "▁k il", + "▁ki l", + "▁ kil", + "▁B ron", + "▁Br on", + "▁Bro n", + "▁experience d", + "▁experi enced", + "▁re mind", + "▁rem ind", + "our se", + "ours e", + "▁W estern", + "▁West ern", + "▁Wes tern", + "ter ed", + "te red", + "tere d", + "t ered", + "▁dev ices", + "▁device s", + "▁ devices", + "▁pict ures", + "▁picture s", + "▁t ut", + "▁tu t", + "\" `", + "▁im possible", + "▁r ail", + "▁ra il", + "▁fe els", + "▁feel s", + "▁fee ls", + "ic as", + "ica s", + "i cas", + "il ling", + "ill ing", + "▁acc ident", + "▁' @", + "____ ____", + "▁n otes", + "▁not es", + "▁no tes", + "▁note s", + "▁ notes", + "om an", + "oma n", + "o man", + "Par ser", + "Parse r", + "Pars er", + "▁dis covered", + "▁discover ed", + "▁R oman", + "▁Rom an", + "▁Ro man", + "▁Roma n", + "▁bud get", + "▁gu ide", + "▁guid e", + "ki ng", + "kin g", + "k ing", + "▁in cred", + "▁inc red", + "▁incre d", + "ol ar", + "ola r", + "o lar", + "en den", + "end en", + "ende n", + "Des c", + "De sc", + "D esc", + "▁w ave", + "▁wa ve", + "▁ wave", + "б ли", + "ig t", + "i gt", + "▁re strict", + "▁rest rict", + "▁restr ict", + "▁R et", + "▁Re t", + "▁ Ret", + "▁m ac", + "▁ma c", + "▁ mac", + "у р", + "B S", + "í s", + "▁gener ation", + "de m", + "d em", + "al o", + "a lo", + "б ра", + "▁order ed", + "▁ord ered", + "▁ ordered", + "dr op", + "dro p", + "d rop", + "▁p p", + "▁ pp", + "▁Re view", + "▁Rev iew", + "▁ Review", + "▁liter ally", + "▁literal ly", + "▁S ir", + "▁Si r", + "▁ Sir", + "▁Y eah", + "▁Ye ah", + "▁ Yeah", + "▁d ensity", + "▁dens ity", + "▁ density", + "ri z", + "r iz", + "in de", + "ind e", + "i nde", + "▁g ain", + "▁ga in", + "▁ gain", + "▁p anel", + "▁pan el", + "▁pa nel", + "▁ panel", + "je t", + "j et", + "▁T imes", + "▁Time s", + "▁Tim es", + "▁Ti mes", + "▁ Times", + "▁n ella", + "▁ne lla", + "▁nel la", + "▁nell a", + "▁pre viously", + "▁previous ly", + "▁prev iously", + "point s", + "Se nd", + "S end", + "▁B rown", + "▁Br own", + "▁Bro wn", + "▁Brow n", + "ea ch", + "e ach", + "▁tr igger", + "▁ trigger", + "ome times", + "omet imes", + "ic os", + "ico s", + "i cos", + "G R", + "Pane l", + "Pan el", + "P anel", + "og en", + "oge n", + "o gen", + "▁c m", + "▁ cm", + "ru ctions", + "ruct ions", + "ruction s", + "▁k iss", + "▁ki ss", + "▁s olo", + "▁so lo", + "▁sol o", + "▁f amous", + "▁fam ous", + "ra n", + "r an", + "п ро", + "▁th ro", + "▁thr o", + "Gr aph", + "G raph", + "im it", + "imi t", + "i mit", + "▁V alue", + "▁Val ue", + "▁ Value", + "▁st arts", + "▁start s", + "▁star ts", + "ip eline", + "ipe line", + "h d", + "T C", + "▁dis cussion", + "▁discuss ion", + "▁tr uck", + "ak a", + "a ka", + "On ly", + "▁E qu", + "▁Eq u", + "▁ Equ", + "▁k ö", + "▁ kö", + "▁B es", + "▁Be s", + "▁crit ic", + "▁pro pos", + "▁prop os", + "▁b att", + "▁bat t", + "▁ba tt", + "▁S ection", + "▁Se ction", + "▁ Section", + "Sh ow", + "S how", + "g p", + "ST ATE", + "STAT E", + "PO ST", + "POS T", + "P OST", + "▁N ord", + "▁No rd", + "▁Nor d", + "▁in nov", + "▁inn ov", + "▁c rim", + "▁cr im", + "▁cri m", + "▁ crim", + "ax is", + "a xis", + "▁T urn", + "▁Tur n", + "▁Tu rn", + "▁ Turn", + "con n", + "co nn", + "Run time", + "▁rem aining", + "▁remain ing", + "os ton", + "ost on", + "osto n", + "o ston", + "▁ Э", + "▁window s", + "▁wind ows", + "▁ windows", + "▁R oyal", + "▁Ro yal", + "▁Roy al", + "▁v ide", + "▁vi de", + "▁vid e", + "P P", + "ch ron", + "chr on", + "▁s an", + "▁sa n", + "▁ san", + "▁r ise", + "▁ri se", + "▁ris e", + "▁ rise", + "▁d elle", + "▁de lle", + "▁del le", + "▁dell e", + "▁D ur", + "▁Du r", + "▁rap id", + "▁ra pid", + "ce rt", + "cer t", + "c ert", + "L A", + "ed ge", + "▁\\ ]", + "▁ \\]", + "▁en tered", + "▁ent ered", + "▁enter ed", + "▁l aws", + "▁la ws", + "▁law s", + "▁ph oto", + "▁phot o", + "▁ photo", + "▁ap plications", + "▁applic ations", + "▁application s", + "▁appl ications", + "▁Ber lin", + "▁ar rest", + "▁arr est", + "▁f ederal", + "▁fed eral", + "▁feder al", + "▁R ussia", + "▁Russ ia", + "▁us ual", + "▁r aw", + "▁ra w", + "▁ raw", + "▁pi ù", + "êt re", + "ê tre", + "JS ON", + "J SON", + "SI ON", + "S ION", + "xt ure", + "ist ent", + "iste nt", + "isten t", + "▁P ower", + "▁Po wer", + "▁Pow er", + "▁ Power", + "Bi t", + "B it", + "▁cap acity", + "▁capac ity", + "▁ capacity", + "▁c ards", + "▁car ds", + "▁card s", + "▁ cards", + "UI D", + "U ID", + "im ents", + "iment s", + "imen ts", + "i ments", + "▁d ar", + "▁da r", + "▁ dar", + "▁Ch icago", + "▁comfort able", + "ti p", + "t ip", + "ba s", + "b as", + "▁m u", + "▁ mu", + "▁en emy", + "▁enem y", + "ya n", + "y an", + "▁ф и", + "▁ фи", + "▁up dated", + "▁update d", + "▁ updated", + "an go", + "ang o", + "E v", + "E ffect", + "os ing", + "osi ng", + "o sing", + "ren ce", + "r ence", + "▁Con gress", + "▁Cong ress", + "▁d efe", + "▁de fe", + "▁def e", + "▁i p", + "▁ ip", + "▁t out", + "▁to ut", + "▁tou t", + "▁f reedom", + "▁free dom", + "▁freed om", + "▁a o", + "▁ ao", + "▁There fore", + "▁Ther efore", + "Ed it", + "E dit", + "▁Vir gin", + "RE E", + "R EE", + "ar go", + "arg o", + "▁D am", + "▁Da m", + "▁ Dam", + "▁tra ffic", + "▁traff ic", + "ño s", + "ñ os", + "▁a lle", + "▁al le", + "▁all e", + "▁ alle", + "▁dep th", + "▁ depth", + "No w", + "N ow", + "▁s ides", + "▁side s", + "▁si des", + "▁sid es", + "▁го ди", + "▁год и", + "Des criptor", + "▁art ikel", + "▁n arrow", + "▁narr ow", + "▁nar row", + "__ _", + "_ __", + "k w", + "ut o", + "u to", + "▁Face book", + "▁Fac ebook", + "te gr", + "t egr", + "bo olean", + "ni k", + "n ik", + "b d", + "Tr ack", + "Tra ck", + "▁g ran", + "▁gr an", + "▁gra n", + "res hold", + "resh old", + "ве т", + "в ет", + "wr ap", + "w rap", + "▁n oise", + "▁no ise", + "ig u", + "i gu", + "▁B on", + "▁Bo n", + "▁ Bon", + "▁w y", + "▁ wy", + "lin ux", + "ck s", + "c ks", + "▁f ans", + "▁fa ns", + "▁fan s", + "▁m ach", + "▁ma ch", + "▁mac h", + "▁p rices", + "▁pr ices", + "▁pri ces", + "▁price s", + "é v", + "ou ts", + "out s", + "o uts", + "stand ing", + "stan ding", + "▁c ateg", + "▁cat eg", + "; \\", + "▁de cre", + "▁dec re", + "▁S aturday", + "▁m enu", + "▁me nu", + "▁men u", + "▁ menu", + "▁N ov", + "▁No v", + "▁Y et", + "▁Ye t", + "▁та к", + "lic he", + "li che", + "lich e", + "l iche", + "▁Ac adem", + "▁commun ication", + "us ing", + "u sing", + "▁Soc iety", + "▁Soci ety", + "▁n uc", + "▁nu c", + "pect ive", + "or ial", + "oria l", + "ori al", + "o rial", + "▁af raid", + "▁an imal", + "▁anim al", + "▁turn ing", + "▁tur ning", + "ds t", + "d st", + "math frak", + "le rs", + "ler s", + "l ers", + "▁l ots", + "▁lo ts", + "▁lot s", + "▁ á", + "▁T ra", + "▁Tr a", + "▁ Tra", + "n p", + "▁r ose", + "▁ro se", + "▁ rose", + "▁G L", + "▁ GL", + "▁hel ping", + "▁help ing", + "▁w inter", + "▁win ter", + "▁ко м", + "▁ ком", + "Mo ck", + "M ock", + "▁invest ment", + "Us e", + "U se", + "▁Can ad", + "н д", + "Co py", + "Cop y", + "C opy", + "▁f ly", + "▁fl y", + "▁ fly", + "SE R", + "S ER", + "▁F ar", + "▁Fa r", + "▁R os", + "▁Ro s", + "am il", + "ami l", + "a mil", + "▁fight ing", + "▁rel igious", + "▁relig ious", + "su per", + "sup er", + "s uper", + "sc reen", + "scr een", + "s creen", + "▁f urn", + "▁fur n", + "▁fu rn", + "▁surpr ised", + "▁surprise d", + "▁re plied", + "▁repl ied", + "Act ivity", + "Activ ity", + "▁D own", + "▁Do wn", + "▁Dow n", + "▁ Down", + "▁in sert", + "▁ins ert", + "▁ insert", + "▁O lymp", + "▁point ed", + "▁po inted", + "▁C ard", + "▁Car d", + "▁Ca rd", + "▁ Card", + "dr iver", + "drive r", + "d river", + "▁D a", + "▁ Da", + "! --", + "ro ud", + "rou d", + "r oud", + "un do", + "und o", + "▁m essages", + "▁message s", + "▁mess ages", + "▁ messages", + "▁P oint", + "▁Po int", + "▁ Point", + "V M", + "▁p lane", + "▁pl ane", + "▁plan e", + "▁ plane", + "x c", + "▁telev ision", + "▁tele vision", + "▁televis ion", + "ё н", + "▁thous ands", + "▁thousand s", + "▁c ris", + "▁cr is", + "▁cri s", + "▁de lay", + "▁del ay", + "▁ delay", + "▁N ext", + "▁Ne xt", + "▁ Next", + "▁no mbre", + "▁nom bre", + "▁t u", + "▁ tu", + "▁sk ip", + "▁ski p", + "▁ skip", + "ro ad", + "r oad", + "istr ation", + "▁t ur", + "▁tu r", + "▁De velop", + "▁Devel op", + "▁П а", + "▁д ру", + "▁др у", + "▁wonder ful", + "> &", + "▁L iber", + "▁Li ber", + "▁Lib er", + "▁s cope", + "▁sc ope", + "▁ scope", + "▁man age", + "▁ma nage", + "▁d ass", + "▁da ss", + "▁das s", + "▁re call", + "▁rec all", + "P M", + "▁re levant", + "▁relev ant", + "▁E arth", + "▁ка к", + "▁a pr", + "▁ap r", + "▁A SS", + "▁AS S", + "▁ ASS", + "ié n", + "i én", + "▁S H", + "▁ SH", + "oo m", + "o om", + "it et", + "ite t", + "no ne", + "non e", + "n one", + "as i", + "a si", + "▁mot or", + "▁mo tor", + "▁S how", + "▁Sh ow", + "▁ Show", + "n b", + "▁fact ors", + "▁fa ctors", + "▁factor s", + "▁f orest", + "▁for est", + "▁fore st", + "▁fo rest", + "▁в ре", + "th m", + "t hm", + "▁m unicip", + "▁turn s", + "▁tur ns", + "▁Div ision", + "▁Di vision", + "E C", + "▁dis appe", + "struct or", + "stru ctor", + "▁some where", + "▁Afr ican", + "▁Africa n", + "▁Inst itute", + "▁Institut e", + "Gr id", + "G rid", + "▁te acher", + "▁teach er", + "▁tea cher", + "ur ies", + "uri es", + "u ries", + "▁respect ively", + "▁respective ly", + "▁S D", + "▁ SD", + "▁a live", + "▁al ive", + "▁ali ve", + "▁p ou", + "▁po u", + "▁W ater", + "▁Wat er", + "▁Wa ter", + "▁ Water", + "ф е", + "▁ch anging", + "▁chang ing", + "▁ changing", + "▁after noon", + "▁or ders", + "▁order s", + "▁ord ers", + "▁ orders", + "Re t", + "R et", + "Point er", + "Po inter", + "▁s av", + "▁sa v", + "er g", + "e rg", + "ok ed", + "oke d", + "o ked", + "ess ions", + "ession s", + "▁F ire", + "▁Fi re", + "▁ Fire", + "ar et", + "are t", + "a ret", + "im m", + "i mm", + "▁des ire", + "▁ що", + "▁De sign", + "▁Des ign", + "▁ Design", + "ut ure", + "▁Off ice", + "▁c md", + "▁cm d", + "▁ cmd", + "▁e ating", + "▁eat ing", + "Net work", + "▁r ough", + "▁ro ugh", + "▁rou gh", + "▁ rough", + "oper ator", + "IG N", + "I GN", + "▁s ports", + "▁sp orts", + "▁sport s", + "▁w eren", + "▁we ren", + "▁were n", + "▁wer en", + "▁n oted", + "▁not ed", + "▁no ted", + "▁note d", + "▁tw ice", + "II I", + "I II", + "▁a nx", + "▁an x", + "▁e lim", + "▁el im", + "▁а в", + "▁i o", + "▁ io", + "▁spe ech", + "▁con du", + "▁cond u", + "el les", + "ell es", + "elle s", + "id ade", + "ida de", + "idad e", + "▁adv ance", + "R I", + "oc a", + "o ca", + "/ \\", + "ap shot", + "aps hot", + "▁t ail", + "▁ta il", + "▁ tail", + "mod els", + "model s", + "mode ls", + "og y", + "o gy", + "▁J eff", + "▁Je ff", + "ir ation", + "irat ion", + "▁K ore", + "▁Ko re", + "▁Kor e", + "▁le ads", + "▁lead s", + "ba t", + "b at", + "Ad apter", + "c ategory", + "ang ular", + "angu lar", + "▁s aved", + "▁sa ved", + "▁save d", + "▁sav ed", + "▁ saved", + "▁un iform", + "▁ uniform", + "▁n é", + "▁ né", + "▁business es", + "His t", + "Hi st", + "H ist", + "▁а р", + "▁ ар", + "do main", + "dom ain", + "▁S i", + "▁ Si", + "ra ise", + "rais e", + "rai se", + "r aise", + "▁w arn", + "▁war n", + "▁wa rn", + "▁ warn", + "het ic", + "h etic", + "▁G ro", + "▁Gr o", + ")) .", + ") ).", + "} >", + "з е", + "▁Amaz on", + "▁Or gan", + "▁ Organ", + "▁L ake", + "▁La ke", + "▁ag reement", + "▁agree ment", + "▁agre ement", + "x a", + "▁p erman", + "▁per man", + "▁perm an", + "▁cont aining", + "▁contain ing", + "▁st range", + "▁str ange", + "▁strang e", + "ст і", + "с ті", + "▁st upid", + "▁spe aking", + "▁speak ing", + "▁Intern et", + "▁Inter net", + "pre fix", + "pref ix", + "p refix", + "es c", + "e sc", + "As sert", + "Ass ert", + "pro te", + "pr ote", + "prot e", + "p rote", + "▁m anner", + "▁man ner", + "▁S z", + "un te", + "unt e", + "u nte", + "io t", + "i ot", + "Pro file", + "ov en", + "ove n", + "o ven", + "▁for med", + "▁form ed", + "▁forme d", + "▁ formed", + "▁l it", + "▁li t", + "▁ lit", + "▁econom y", + "▁ec onomy", + "▁c z", + "▁ cz", + "wi d", + "w id", + "RE Q", + "R EQ", + "▁ch osen", + "▁cho sen", + "▁chose n", + "▁P rodu", + "▁Pro du", + "▁ Produ", + "os ter", + "ost er", + "o ster", + "st ances", + "stance s", + "stan ces", + "aw a", + "a wa", + "▁R en", + "▁Re n", + "▁conf irm", + "▁ confirm", + "▁Б о", + "▁b illion", + "▁bill ion", + "▁d éc", + "▁dé c", + "ý ch", + "▁ill ustr", + "TI ES", + "T IES", + "▁P ub", + "▁Pu b", + "▁ Pub", + "▁b an", + "▁ba n", + "▁ ban", + "ad ed", + "ade d", + "a ded", + "ah n", + "a hn", + "▁C ath", + "▁Cat h", + "▁Ca th", + "no number", + "non umber", + "▁wor st", + "▁М е", + "▁sugg ested", + "▁suggest ed", + "st ats", + "stat s", + "sta ts", + "▁c ant", + "▁can t", + "▁ca nt", + "▁al ign", + "▁ali gn", + "▁ align", + "kap pa", + "k appa", + "▁h en", + "▁he n", + "▁ hen", + "▁in iti", + "▁init i", + "'] )", + "' ])", + "B I", + "▁g arden", + "▁gar den", + "▁gard en", + "▁sec ure", + "▁secur e", + "▁ secure", + "▁\\ [", + "▁ \\[", + "hand ler", + "handle r", + "el li", + "ell i", + "e lli", + "ld ots", + "l dots", + "se cut", + "sec ut", + "s ecut", + "▁ext ended", + "▁extend ed", + "} -", + "an ie", + "ani e", + "a nie", + "▁F ind", + "▁Fin d", + "▁Fi nd", + "▁ Find", + "▁M useum", + "▁Muse um", + "▁C onne", + "▁Con ne", + "▁ Conne", + "y y", + "▁pass ion", + "ak ers", + "ake rs", + "aker s", + "a kers", + "ah r", + "a hr", + "olog ies", + "ologie s", + "▁equ ation", + "▁eq uation", + "▁ equation", + "▁occ asion", + "▁occas ion", + "Le t", + "L et", + "'] ['", + "'][ '", + "' ]['", + "Pr int", + "an es", + "ane s", + "a nes", + "ie nte", + "ient e", + "ien te", + "i ente", + "▁T oday", + "▁To day", + "▁Tod ay", + "LE CT", + "L ECT", + "▁A f", + "▁ Af", + ", ,", + "▁Т а", + "▁` ``", + "▁`` `", + "ev en", + "eve n", + "e ven", + "si n", + "s in", + "ur er", + "ure r", + "u rer", + "▁ °", + "ot imes", + "oti mes", + "o times", + "▁I O", + "▁ IO", + "▁po et", + "() ));", + "()) );", + "())) ;", + "( )));", + "▁ −", + "▁ad opt", + "ph ere", + "pher e", + "p here", + "# [", + "▁c entre", + "▁cent re", + "ov es", + "ove s", + "o ves", + "▁a ns", + "▁an s", + "▁ ans", + "d p", + "▁K ir", + "▁Ki r", + "▁applic able", + "f p", + "▁vis ual", + "▁ok ay", + "or o", + "o ro", + "▁opportun ities", + "Re pository", + "Rep ository", + "▁l l", + "▁ ll", + "▁R od", + "▁Ro d", + "▁s hel", + "▁sh el", + "▁she l", + "▁la unch", + "▁con ven", + "▁conv en", + "▁conve n", + "▁S pe", + "▁Sp e", + "▁ Spe", + "Am er", + "A mer", + "▁c ette", + "▁cet te", + "Con d", + "Co nd", + "C ond", + "de p", + "d ep", + "O wn", + "▁h ook", + "▁ho ok", + "▁ hook", + "▁d ict", + "▁di ct", + "▁dic t", + "▁ dict", + "▁Th ose", + "▁f ellow", + "▁fell ow", + "▁fel low", + "▁phil osoph", + "▁philos oph", + "vi n", + "v in", + "fer ences", + "ference s", + "ha v", + "h av", + "▁ad ding", + "▁add ing", + "▁ adding", + "ivers e", + "iver se", + "i verse", + "ga me", + "g ame", + "▁Bl ue", + "▁ Blue", + "▁c lin", + "▁cl in", + "not e", + "no te", + "n ote", + "▁R am", + "▁Ra m", + "ме р", + "м ер", + "co very", + "cover y", + "cov ery", + "c overy", + "ñ a", + "▁б и", + "▁ би", + "▁f ashion", + "▁b roke", + "▁br oke", + "▁bro ke", + "▁' \\", + "▁ '\\", + "▁re ader", + "▁read er", + "▁ reader", + "но е", + "но сти", + "ност и", + "▁pay ment", + "▁ payment", + "▁L ic", + "▁Li c", + "▁l ips", + "▁li ps", + "▁lip s", + "▁ac adem", + "▁M ot", + "▁Mo t", + "el ls", + "ell s", + "C HECK", + "▁р у", + "▁ ру", + "▁M S", + "▁ MS", + "Ed itor", + "Edit or", + "▁z one", + "▁zo ne", + "▁ zone", + "it ure", + "itu re", + "▁I T", + "▁ IT", + "run time", + "▁pro ceed", + "▁proc eed", + "ло в", + "л ов", + "▁M aria", + "▁Mar ia", + "▁Ma ria", + "ol ver", + "olve r", + "olv er", + "▁Th anks", + "▁Thank s", + "▁ Thanks", + "▁should n", + "▁J oh", + "▁Jo h", + "▁Mod el", + "▁Mo del", + "▁Mode l", + "▁ Model", + "▁S ov", + "▁So v", + "! '", + "D i", + "▁c ancer", + "▁can cer", + "Id ent", + "▁ex change", + "il ler", + "ill er", + "ille r", + "in f", + "i nf", + "LE N", + "L EN", + "() {", + "( ){", + "ag a", + "a ga", + "\"] ,", + "\" ],", + "u h", + "▁K en", + "▁Ke n", + "▁ph otos", + "▁phot os", + "▁photo s", + "▁t iny", + "▁ti ny", + "▁tin y", + "▁ tiny", + "▁g ent", + "▁gen t", + "▁ge nt", + "▁ gent", + "ü l", + "▁T ake", + "▁Ta ke", + "▁Tak e", + "▁ Take", + "id el", + "ide l", + "i del", + "ou ting", + "out ing", + "In ternal", + "Inter nal", + "Intern al", + "▁c ells", + "▁cell s", + "▁cel ls", + "ни м", + "н им", + "ha rd", + "har d", + "h ard", + "▁T own", + "▁To wn", + "▁Tow n", + "ob e", + "o be", + "pl ex", + "ple x", + "p lex", + "те р", + "т ер", + "to ns", + "ton s", + "t ons", + "▁conc entr", + "▁concent r", + "mo ck", + "m ock", + "v c", + "á z", + "▁Ch ampionship", + "▁Champion ship", + "▁Champions hip", + "▁б е", + "▁ бе", + "? ?", + "ér i", + "é ri", + "al y", + "a ly", + "▁ Ц", + "ier te", + "iert e", + "▁tot ally", + "▁total ly", + "▁A uf", + "▁Au f", + "▁our selves", + "▁S elf", + "▁Sel f", + "▁ Self", + "Form s", + "For ms", + "ight er", + "igh ter", + "▁is land", + "fm t", + "f mt", + "▁r c", + "▁ rc", + "▁t ells", + "▁tell s", + "▁tel ls", + "B B", + "di t", + "d it", + "▁vari ables", + "▁variable s", + "▁ variables", + "▁int ended", + "▁intend ed", + "iz ont", + "izon t", + "izo nt", + "▁pl ays", + "▁play s", + "da m", + "d am", + "se q", + "s eq", + "▁S up", + "▁Su p", + "▁ Sup", + "▁c ultural", + "▁cult ural", + "▁sc ream", + "__ ,", + "_ _,", + "ci pl", + "cip l", + "Time out", + "▁ ж", + "or te", + "ort e", + "▁repl aced", + "▁replace d", + "E M", + "▁ab andon", + "▁Spec ial", + "▁Spe cial", + "▁ Special", + "el len", + "ell en", + "elle n", + "▁B ru", + "▁Br u", + "ir med", + "irm ed", + "T e", + "ol t", + "o lt", + "j u", + "Arg ument", + "▁ne ut", + "▁neu t", + "▁ neut", + "sc ape", + "▁R ay", + "▁Ra y", + "▁ Ray", + "▁Pol it", + "▁Po lit", + "▁crow d", + "▁cro wd", + "▁Window s", + "▁Wind ows", + "▁ Windows", + "ie go", + "ieg o", + "i ego", + "▁e scape", + "▁esc ape", + "▁ escape", + "▁Ap ache", + "sy nc", + "syn c", + "s ync", + "eb en", + "e ben", + "if ies", + "ifi es", + "et her", + "eth er", + "ethe r", + "e ther", + "Met a", + "Me ta", + "M eta", + "▁big gest", + "Ga me", + "G ame", + "▁trans action", + "▁ transaction", + "En v", + "E nv", + "▁М о", + "▁pl enty", + "▁m el", + "▁me l", + "▁ mel", + "п ре", + "▁mot iv", + "▁о р", + "▁ ор", + "or gan", + "org an", + "▁m ock", + "▁mo ck", + "▁ mock", + "▁$ _", + "▁ $_", + "ен е", + "е не", + "▁N umber", + "▁Num ber", + "▁Nu mber", + "▁ Number", + "ck now", + "c know", + "▁Up date", + "▁ Update", + "ze ro", + "zer o", + "z ero", + "▁sur prise", + "▁surpr ise", + "ce an", + "pd f", + "p df", + "Gl obal", + "▁att end", + "▁f ond", + "▁fo nd", + "▁fon d", + "▁under stood", + "Na v", + "N av", + "▁M ic", + "▁Mi c", + "▁ Mic", + "= $", + "ok ing", + "oki ng", + "o king", + "▁Stad ium", + "Cl ose", + "▁compet ition", + "▁sold iers", + "▁soldier s", + "▁O P", + "▁ OP", + "ag ne", + "agn e", + "▁An ton", + "▁Ant on", + "Ma in", + "M ain", + "á k", + "▁# [", + "▁ #[", + "▁Com mit", + "▁Comm it", + "▁ Commit", + "py x", + "▁e ast", + "▁eas t", + "▁ east", + "▁Or der", + "▁Ord er", + "▁ Order", + "F loat", + "▁accept ed", + "▁mon itor", + "▁ monitor", + "▁p ad", + "▁pa d", + "▁ pad", + "on ic", + "oni c", + "o nic", + "▁p ushed", + "▁push ed", + "▁re place", + "▁rep lace", + "▁repl ace", + "▁ replace", + "CR E", + "C RE", + "▁r ide", + "▁ri de", + "▁rid e", + "▁ ride", + "fo und", + "f ound", + "= %", + "во й", + "▁mat ches", + "▁match es", + "▁ matches", + "▁L ie", + "▁Li e", + "▁exper iences", + "▁experience s", + "▁experi ences", + "Po ol", + "P ool", + "up s", + "u ps", + "A V", + "▁ex istence", + "▁exist ence", + "▁t hin", + "▁th in", + "▁m agn", + "▁mag n", + "▁ma gn", + "CO MP", + "COM P", + "ho me", + "hom e", + "h ome", + "▁n i", + "▁ ni", + "▁wur den", + "▁wurde n", + "ла в", + "▁te eth", + "▁S tan", + "▁St an", + "▁Sta n", + "ap pro", + "app ro", + "an ny", + "ann y", + "if ts", + "ift s", + "▁un known", + "▁ unknown", + "▁h omes", + "▁home s", + "▁hom es", + "▁ho mes", + "▁ent ity", + "▁ entity", + "ci e", + "c ie", + "ле ние", + "ia r", + "i ar", + "▁compl iance", + "▁focus ed", + "uz z", + "u zz", + "=\\ \"", + "= \\\"", + "com ponents", + "component s", + "Att r", + "At tr", + "all ery", + "alle ry", + "aller y", + "▁ident ify", + "O k", + "pi e", + "p ie", + "▁St ill", + "▁off ering", + "▁offer ing", + "▁bu sy", + "▁bus y", + "ct l", + "c tl", + "it ors", + "itor s", + "ito rs", + "▁concern ed", + "▁concer ned", + "▁b rown", + "▁br own", + "▁bro wn", + "▁brow n", + "cl k", + "Se lected", + "Select ed", + "▁B lock", + "▁Bl ock", + "▁Blo ck", + "▁ Block", + "▁e gy", + "▁eg y", + "▁ egy", + "ic ing", + "ici ng", + "i cing", + "▁U RL", + "▁ URL", + "▁t opic", + "▁to pic", + "▁top ic", + "▁ topic", + "▁Pro duct", + "▁Produ ct", + "▁ Product", + "▁ч и", + "▁ чи", + "▁t rial", + "▁tr ial", + "▁tri al", + "▁week end", + "l u", + "▁I V", + "▁ IV", + "▁E gy", + "▁Eg y", + "x C", + "▁n ove", + "▁no ve", + "▁nov e", + "▁l ett", + "▁le tt", + "▁let t", + "▁ lett", + "en ne", + "enn e", + "() ).", + "()) .", + "( )).", + ".* *", + ". **", + "▁p romise", + "▁prom ise", + "el ection", + "ele ction", + "elect ion", + "e lection", + "Aut h", + "A uth", + "r v", + "ri l", + "r il", + "▁con duct", + "▁cond uct", + "▁condu ct", + "▁ conduct", + "▁main tain", + "▁maint ain", + "▁bo at", + "▁ boat", + "▁op posite", + "▁oppos ite", + "sp in", + "spi n", + "s pin", + "web pack", + "an ta", + "ant a", + "▁o rient", + "▁or ient", + "▁ orient", + "▁s uc", + "▁su c", + "▁ex ercise", + "▁exerc ise", + "▁eff icient", + "▁ efficient", + "▁trad ition", + "▁z w", + "▁ zw", + "▁S ud", + "▁Su d", + "go ing", + "▁P ier", + "▁Pi er", + "in v", + "i nv", + "ip es", + "ipe s", + "i pes", + "ensure math", + "▁con ver", + "▁conv er", + "▁conve r", + "cre en", + "cr een", + "c reen", + "▁t error", + "▁ter ror", + "▁terr or", + "▁D ou", + "▁Do u", + "▁in valid", + "▁ invalid", + "ce ived", + "ceive d", + "▁A rab", + "▁Ar ab", + "▁w ire", + "▁wir e", + "▁ wire", + "ap plication", + "sh ift", + "Gener ic", + "▁P lan", + "▁Pl an", + "▁ Plan", + "▁W all", + "▁Wal l", + "▁Wa ll", + "▁ Wall", + "▁direct ory", + "▁director y", + "▁ directory", + "▁e gg", + "▁eg g", + "▁we alth", + "▁ wealth", + "ran dom", + "rand om", + "r andom", + "att ribute", + "▁h ide", + "▁hi de", + "▁hid e", + "▁ hide", + "Se rial", + "Ser ial", + "S erial", + "ca m", + "c am", + "▁it al", + "▁i tal", + "▁ ital", + "▁L ine", + "▁Lin e", + "▁Li ne", + "▁ Line", + "▁C HECK", + "▁ CHECK", + "ploy ment", + "▁mass ive", + "▁ex tract", + "▁ext ract", + "▁extra ct", + "▁extr act", + "▁ extract", + "ch ain", + "cha in", + "Res t", + "Re st", + "R est", + "▁L as", + "▁La s", + "▁b ear", + "▁be ar", + "▁ bear", + "▁l inks", + "▁link s", + "▁lin ks", + "▁ links", + "▁new sp", + "▁news p", + "▁F C", + "▁ FC", + "Car d", + "C ard", + "ak s", + "a ks", + "▁v isible", + "▁vis ible", + "▁ visible", + "▁M arc", + "▁Mar c", + "▁Ma rc", + "▁B oston", + "▁Bo ston", + "▁Bos ton", + "▁res erved", + "▁reserv ed", + "▁reserve d", + "▁ro of", + "lic enses", + "license s", + "d c", + "▁In formation", + "▁ Information", + "▁w itness", + "S k", + "*) ,", + "* ),", + "Sc ope", + "S cope", + "'] ;", + "' ];", + "▁M ir", + "▁Mi r", + "▁ Mir", + "ud ing", + "udi ng", + "u ding", + "▁t rend", + "▁tr end", + "▁tre nd", + "▁tren d", + "re p", + "r ep", + "▁mus ical", + "▁music al", + "▁ne ither", + "▁nei ther", + "▁C reat", + "▁Cre at", + "▁ Creat", + "▁pos itions", + "▁position s", + "▁posit ions", + "L C", + "rid ge", + "r idge", + "▁offic ers", + "▁office rs", + "▁officer s", + "▁vi olence", + "▁viol ence", + "▁T em", + "▁Te m", + "▁S us", + "▁Su s", + "▁W ay", + "▁Wa y", + "Af ter", + "A fter", + "ac ket", + "ack et", + "▁S ou", + "▁So u", + "ac er", + "ace r", + "a cer", + "| |", + "▁re mark", + "▁r emark", + "▁rem ark", + "▁ remark", + "wa ter", + "w ater", + "n ě", + "▁С а", + "▁s ed", + "▁se d", + "▁ sed", + "E ach", + "▁phot ograph", + "▁photo graph", + "▁let ters", + "▁letter s", + "▁lett ers", + "▁in vent", + "▁inv ent", + "▁M as", + "▁Ma s", + "▁s ongs", + "▁son gs", + "▁song s", + "ó l", + "ki nd", + "kin d", + "k ind", + "▁N on", + "▁No n", + "▁ Non", + "▁d ust", + "▁du st", + "** :", + "* *:", + "nab la", + ".\" ,", + ". \",", + "Loc k", + "Lo ck", + "L ock", + "▁Д о", + "▁cl uster", + "▁ cluster", + "lo ss", + "los s", + "l oss", + "▁ASS ERT", + "▁ ASSERT", + "fa ll", + "f all", + "▁re ject", + "▁ reject", + "▁Sp ring", + "▁Spr ing", + "▁ Spring", + "▁wed ding", + "▁g rav", + "▁gr av", + "▁gra v", + "▁ grav", + "ress ion", + "r ession", + "li mit", + "lim it", + "l imit", + "RE S", + "R ES", + "] }", + "▁l isted", + "▁li sted", + "▁list ed", + "▁ listed", + "▁T ele", + "▁Te le", + "▁Tel e", + "▁ Tele", + "hl ine", + "h line", + "▁ch ief", + "▁chi ef", + "ME M", + "M EM", + "да р", + "д ар", + "▁exp ensive", + "tr ace", + "tra ce", + "▁R og", + "▁Ro g", + "▁C oll", + "▁Col l", + "▁Co ll", + "▁ Coll", + "▁Aut hor", + "▁Auth or", + "▁ Author", + "▁B oard", + "▁Bo ard", + "▁ Board", + "▁C apt", + "▁Cap t", + "▁Ca pt", + "▁ Capt", + "TE XT", + "T EXT", + "▁re con", + "▁rec on", + "es ta", + "est a", + "e sta", + "▁proper ly", + "▁& \\", + "▁ &\\", + "le ton", + "let on", + "l eton", + "ik er", + "ike r", + "i ker", + "G u", + "▁K om", + "▁Ko m", + "oc o", + "o co", + "▁any more", + "▁t aste", + "▁ta ste", + "▁tast e", + "▁S anta", + "▁San ta", + "▁Sant a", + "ge x", + "g ex", + "▁Se cret", + "▁Sec ret", + "▁ Secret", + "▁tal ent", + "▁tale nt", + "▁mom ents", + "▁moment s", + "▁mo ments", + "▁B a", + "▁ex tr", + "▁ext r", + "▁ extr", + "▁Com mission", + "▁Comm ission", + "▁mod ify", + "▁Fig ure", + "▁ Figure", + "▁d omin", + "▁do min", + "▁dom in", + "▁ domin", + "▁p lot", + "▁pl ot", + "▁ plot", + "en ger", + "eng er", + "enge r", + "ut ch", + "▁c ities", + "▁cit ies", + "▁ci ties", + "▁n ut", + "▁nu t", + "▁ nut", + "pro file", + "prof ile", + "▁S tat", + "▁St at", + "▁Sta t", + "▁ Stat", + "▁n odes", + "▁no des", + "▁node s", + "▁nod es", + "▁ nodes", + "▁n s", + "▁ ns", + "ess ages", + "essage s", + "essa ges", + "im pl", + "imp l", + "ic ker", + "ick er", + "i cker", + "▁ex amples", + "▁example s", + "▁exam ples", + "ab eth", + "abe th", + "abet h", + "▁st ated", + "▁stat ed", + "▁state d", + "▁sta ted", + "fi re", + "f ire", + "bu l", + "b ul", + "▁danger ous", + "▁P ay", + "▁Pa y", + "▁ Pay", + "▁G re", + "▁Gr e", + "▁ Gre", + "▁Mon day", + "▁Mond ay", + "es ome", + "eso me", + "e some", + "ig an", + "iga n", + "i gan", + "ru nd", + "run d", + "r und", + "pr ise", + "p rise", + "fa il", + "f ail", + "▁N ever", + "▁Ne ver", + "▁Nev er", + "▁ Never", + "A v", + "▁line ar", + "▁lin ear", + "▁ linear", + "▁u l", + "▁ ul", + "WA R", + "W AR", + "ре н", + "р ен", + "▁A T", + "▁ AT", + "▁d op", + "▁do p", + "▁n ou", + "▁no u", + "Des t", + "De st", + "D est", + "▁claim s", + "en da", + "end a", + "▁c razy", + "▁cr azy", + "ge l", + "g el", + "og gle", + "ogg le", + "▁rep resentation", + "▁represent ation", + "in en", + "ine n", + "i nen", + "▁altern ative", + "▁alter native", + "D M", + "AB ILITY", + "face s", + "fa ces", + "fac es", + "f aces", + "▁do ors", + "▁door s", + "▁ doors", + "at iv", + "ati v", + "Lo ok", + "L ook", + "▁J SON", + "▁JS ON", + "▁ JSON", + "▁appe arance", + "▁appear ance", + "б ря", + "S QL", + "▁sil ence", + "ud o", + "u do", + "▁Direct or", + "▁Dire ctor", + "▁Dir ector", + "State ment", + "Stat ement", + "se lected", + "select ed", + "hi gh", + "h igh", + "pr ime", + "prim e", + "▁ign ore", + "▁ignor e", + "▁ ignore", + "▁col ors", + "▁color s", + "▁ colors", + "us hing", + "ush ing", + "▁v irt", + "▁vi rt", + "▁vir t", + "▁ virt", + "man ager", + "▁rem ote", + "▁remot e", + "▁ remote", + "ł o", + "sm all", + "▁cr ime", + "▁crim e", + "▁cri me", + "r b", + "▁c reation", + "▁cre ation", + "▁creat ion", + "▁f light", + "▁fl ight", + "▁S ign", + "▁Si gn", + "▁Sig n", + "▁ Sign", + "IL E", + "I LE", + "▁D O", + "▁ DO", + "com ment", + "comm ent", + "▁C ost", + "▁Co st", + "▁Cos t", + "▁ Cost", + "._ _", + ". __", + "▁C op", + "▁Co p", + "▁ Cop", + "▁v om", + "▁vo m", + "▁Sc ience", + "▁Sci ence", + "ле ния", + "oo p", + "o op", + "inter face", + "▁WARRAN TIES", + "▁P age", + "▁Pa ge", + "▁ Page", + "** ****", + "**** **", + "*** ***", + "ско м", + "с ком", + "TR UE", + "▁re peated", + "▁repe ated", + "▁repeat ed", + "▁е го", + "ш о", + "▁r oz", + "▁ro z", + "▁ roz", + "P e", + "▁IS BN", + "ir ts", + "irt s", + "pos es", + "po ses", + "pose s", + "p oses", + "}) $", + "} )$", + "▁ І", + "child ren", + "ble s", + "bl es", + "b les", + "EC T", + "E CT", + "▁i z", + "▁ iz", + "▁b uilder", + "▁build er", + "▁ builder", + "▁M edia", + "▁Med ia", + "▁ Media", + "ia t", + "i at", + "▁contr ast", + "▁contra st", + "” ,", + "▁L ink", + "▁Lin k", + "▁ Link", + "▁Educ ation", + "▁j oint", + "▁join t", + "▁jo int", + "▁ joint", + "▁ex ternal", + "▁extern al", + "▁ external", + "▁ро з", + "▁b its", + "▁bit s", + "▁bi ts", + "▁ bits", + "FO RM", + "FOR M", + "F ORM", + "er man", + "erm an", + "w p", + "▁M ike", + "▁Mi ke", + "▁Mik e", + "▁M aster", + "▁Ma ster", + "▁Mas ter", + "▁ Master", + "▁sen ior", + "▁N av", + "▁Na v", + "▁ Nav", + "▁record ed", + "el ing", + "eli ng", + "elin g", + "e ling", + "es h", + "e sh", + "f x", + "ка н", + "к ан", + "▁t all", + "▁tal l", + "▁ta ll", + "▁John son", + "▁s ono", + "▁so no", + "▁son o", + "▁an che", + "▁anc he", + "▁anch e", + "▁ anche", + "ic ken", + "ick en", + "i cken", + "lo op", + "l oop", + "ici ency", + "empor ary", + "▁D oes", + "▁Do es", + "▁ Does", + "▁re lation", + "▁rel ation", + "▁ relation", + "м ы", + "wa s", + "w as", + "lo w", + "l ow", + "ich te", + "icht e", + "i chte", + "▁J ones", + "▁Jo nes", + "▁Jon es", + "▁bed room", + "DI S", + "D IS", + "▁mag net", + "▁magn et", + "▁Eng ine", + "▁ Engine", + "▁feel ings", + "▁feeling s", + "▁fee lings", + "G C", + "▁t orn", + "▁to rn", + "▁tor n", + "▁relationship s", + "▁relation ships", + "▁Р е", + "▁p roud", + "▁pro ud", + "▁pr oud", + "▁t we", + "▁tw e", + "ov al", + "ova l", + "o val", + "▁w aste", + "▁was te", + "▁wa ste", + "▁red uced", + "▁redu ced", + "▁reduce d", + "il ton", + "ilt on", + "B P", + "▁for got", + "▁forg ot", + "▁bod ies", + "▁H aw", + "▁Ha w", + "la g", + "l ag", + "▁w ww", + "▁ www", + "do or", + "d oor", + "▁s ufficient", + "▁suff icient", + "▁doll ars", + "▁dollar s", + "Le n", + "L en", + "▁talk ed", + "▁tal ked", + "▁b ond", + "▁bo nd", + "▁bon d", + "▁B or", + "▁Bo r", + "}} {", + "} }{", + "ro d", + "r od", + "Pass word", + "qu are", + "▁l ights", + "▁light s", + "▁ lights", + "er en", + "ere n", + "e ren", + "▁th irty", + "N C", + "▁T ODO", + "▁TO DO", + "▁res pond", + "▁respon d", + "▁resp ond", + "▁ respond", + "ки х", + "dir ect", + "di rect", + "dire ct", + "d irect", + "a ção", + "▁he av", + "Med ia", + "M edia", + "ex it", + "e xit", + "L icense", + "` .", + "▁m ixed", + "▁mix ed", + "▁d esk", + "▁de sk", + "▁des k", + "▁te aching", + "▁teach ing", + "▁tea ching", + "▁m aj", + "▁ma j", + "▁n erv", + "▁ne rv", + "▁ner v", + "in ations", + "ination s", + "type of", + "▁co ast", + "▁ж е", + "▁ же", + "▁be side", + "▁bes ide", + "um my", + "umm y", + "Do c", + "D oc", + "▁sche dule", + "▁schedul e", + "▁sched ule", + "▁ schedule", + "▁re cover", + "▁rec over", + "▁Fur ther", + "▁ste el", + "bo ot", + "b oot", + "▁Per haps", + "▁с ъ", + "▁O s", + "▁ Os", + "ri ck", + "ric k", + "r ick", + "▁В и", + "Supp ort", + "Sup port", + "S upport", + "▁( _", + "▁ (_", + "ni l", + "n il", + "pi s", + "p is", + "x pected", + "▁process ing", + "▁proces sing", + "▁ processing", + "Bu ild", + "B uild", + "ar ian", + "ari an", + "aria n", + "a rian", + "▁i con", + "▁ic on", + "▁ icon", + "▁C A", + "▁ CA", + "wi ck", + "w ick", + "= (", + "▁al gorithm", + "▁ algorithm", + "▁You ng", + "▁Man agement", + "▁ Management", + "▁anc ient", + "▁anci ent", + "но сть", + "ност ь", + "ot i", + "o ti", + "▁comb ination", + "wor ld", + "w orld", + "n n", + "▁d ram", + "▁dr am", + "en abled", + "ena bled", + "enable d", + "A c", + "C CESS", + "ar ation", + "▁bl ocks", + "▁block s", + "▁blo cks", + "▁ blocks", + "▁Ang eles", + "▁Angel es", + "▁Q ual", + "▁Qu al", + "▁ Qual", + "▁suc ceed", + "▁succ eed", + "net work", + "▁ob lig", + "spring framework", + "▁T re", + "▁Tr e", + "ok es", + "oke s", + "o kes", + "mu n", + "m un", + "▁Net work", + "▁ Network", + "De l", + "D el", + "▁e state", + "▁est ate", + "▁esta te", + "▁l iqu", + "▁li qu", + "▁p ob", + "▁po b", + "▁d ad", + "▁da d", + "▁dist inct", + "▁T it", + "▁Ti t", + "▁L ear", + "▁Le ar", + "fer red", + "and roid", + "andro id", + "▁sub sequ", + "▁subs equ", + "▁Flor ida", + "sub set", + "▁whis per", + "Vo l", + "V ol", + "ul ous", + "ulo us", + "▁c rew", + "▁cre w", + "▁cr ew", + "▁l ug", + "▁lu g", + "pi d", + "p id", + "oc ity", + "oci ty", + "o city", + "sk b", + "s kb", + "▁t ea", + "▁te a", + "у н", + "▁hon or", + "▁ho nor", + "▁I ns", + "▁In s", + "▁ Ins", + "▁g ew", + "▁ge w", + "▁ gew", + "Det ails", + "Detail s", + "ene ath", + "e neath", + "at ar", + "ata r", + "a tar", + "▁_ {", + "▁ _{", + "am en", + "ame n", + "a men", + "▁set up", + "▁ setup", + "Trans action", + "▁bl ank", + "▁ blank", + "Fail ed", + "F ailed", + "jo b", + "j ob", + "▁p ret", + "▁pre t", + "▁pr et", + "▁ pret", + "ß e", + "lo or", + "l oor", + "ř í", + "nc ia", + "n cia", + "▁any where", + "▁L ight", + "▁Li ght", + "▁ Light", + "▁A k", + "B D", + "▁exc ited", + "▁excit ed", + "ag ers", + "age rs", + "ager s", + "a gers", + "▁w arning", + "▁war ning", + "▁warn ing", + "▁ warning", + "▁process es", + "▁proces ses", + "h u", + "▁y outh", + "▁you th", + "▁yo uth", + "▁d ogs", + "▁do gs", + "▁dog s", + "▁o ct", + "▁oc t", + "▁ oct", + "▁n ine", + "▁ni ne", + "▁nin e", + "Write r", + "Wr iter", + "Writ er", + "W riter", + "gr id", + "g rid", + "▁import ance", + "est ic", + "▁care fully", + "▁careful ly", + "ma ster", + "mas ter", + "m aster", + "▁dec isions", + "▁decision s", + "▁decis ions", + "▁p in", + "▁pi n", + "▁ pin", + "▁cr ack", + "TE ST", + "TES T", + "T EST", + "▁L ocal", + "▁Loc al", + "▁Lo cal", + "▁ Local", + "▁R ight", + "▁ Right", + "▁v ast", + "▁va st", + "▁vas t", + "▁f aster", + "▁fa ster", + "▁fast er", + "▁inst itut", + "▁ann ual", + "LA N", + "L AN", + "▁e pisode", + "▁epis ode", + "▁X V", + "▁del ivery", + "▁deliver y", + "t l", + "F P", + "ci rc", + "cir c", + "▁typ ically", + "▁typical ly", + "ig o", + "i go", + "▁int el", + "▁inte l", + "▁ intel", + "na t", + "n at", + "x b", + "ст ро", + "с тро", + ") -", + "▁B al", + "▁Ba l", + "▁ Bal", + "▁J os", + "▁Jo s", + "▁g onna", + "▁R est", + "▁Re st", + "▁Res t", + "▁ Rest", + "jo r", + "j or", + "on ia", + "oni a", + "o nia", + "or ship", + "ors hip", + "ov ery", + "ove ry", + "over y", + "o very", + "LI NE", + "LIN E", + "L INE", + "] :", + "Que ue", + "▁com pare", + "▁comp are", + "▁compar e", + "▁ compare", + "▁ap artment", + "▁apart ment", + "▁r ul", + "▁ru l", + "D r", + "gen cy", + "g ency", + "▁ob viously", + "▁obvious ly", + "zi e", + "z ie", + "yc l", + "y cl", + "fort unately", + "fortun ately", + "fortunate ly", + "▁ste pped", + "▁step ped", + "▁S eg", + "▁Se g", + "▁ Seg", + "▁Wh ich", + "▁ Which", + "▁P C", + "▁ PC", + "▁a st", + "▁as t", + "▁ ast", + "end or", + "endo r", + "▁per mission", + "▁perm ission", + "▁ permission", + "CO L", + "C OL", + "▁T EST", + "▁TE ST", + "▁ TEST", + "P ay", + "ère s", + "è res", + "▁stud ied", + "▁accom pl", + "▁accomp l", + "ro le", + "rol e", + "r ole", + "Wh ere", + "Whe re", + "W here", + "proto buf", + "met adata", + "meta data", + "Jo b", + "J ob", + "▁F our", + "▁Fou r", + "▁Fo ur", + "pl ements", + "ple ments", + "plement s", + "dis able", + "▁l oud", + "▁lo ud", + "▁lou d", + "▁happ ening", + "▁happen ing", + "▁U sing", + "▁Us ing", + "▁ Using", + "ro g", + "r og", + "▁depend s", + "▁dep ends", + "í m", + "' \\", + "▁t aught", + "sh ared", + "sha red", + "share d", + "▁att ributes", + "▁attribute s", + "▁attribut es", + "▁ attributes", + "▁A ction", + "▁Act ion", + "▁ Action", + "▁d ess", + "▁de ss", + "▁des s", + "▁ dess", + "▁h ouses", + "▁house s", + "▁hous es", + "▁ho uses", + "▁re set", + "▁res et", + "▁ reset", + "▁b ien", + "▁bi en", + "▁ex plicit", + "▁expl icit", + "LO W", + "-> _", + "▁P M", + "▁ PM", + "C ategory", + "oi ce", + "o ice", + "in to", + "int o", + "▁m ail", + "▁ma il", + "▁mai l", + "▁ mail", + "▁author ity", + "▁un able", + "▁una ble", + "file name", + "fil ename", + "é k", + "ле й", + "л ей", + "▁s ector", + "▁se ctor", + "▁sec tor", + "▁sect or", + "ap point", + "app oint", + "▁h ang", + "▁ha ng", + "▁han g", + "▁ hang", + "▁c el", + "▁ce l", + "▁ cel", + "rel ated", + "it ate", + "ita te", + "itat e", + "▁' <", + "am ber", + "amb er", + "a mber", + "▁c heap", + "▁che ap", + "▁en abled", + "▁enable d", + "▁ enabled", + "▁di vision", + "▁div ision", + "▁divis ion", + "An y", + "A ny", + "▁h ier", + "▁hi er", + "▁H ead", + "▁He ad", + "▁ Head", + "nt ax", + "n tax", + "ud a", + "u da", + "▁lim itations", + "▁limit ations", + "▁limitation s", + "▁st udio", + "▁stud io", + "med ia", + "medi a", + "m edia", + "▁cir cle", + "▁circ le", + "▁ circle", + "но ва", + "нов а", + "▁l aug", + "▁la ug", + "ac ts", + "act s", + "▁В о", + "ó d", + "pl ed", + "ple d", + "p led", + "LO C", + "L OC", + "Ex pr", + "Exp r", + "> :", + "▁pr és", + "▁pré s", + "▁ prés", + "▁laugh ed", + "▁laug hed", + "▁Th ree", + "▁ Three", + "л ы", + "▁en ds", + "▁end s", + "▁ ends", + "▁fund ament", + "▁in her", + "▁ inher", + "▁l iv", + "▁li v", + "▁ liv", + "bi d", + "b id", + "▁respons ibility", + "▁check ed", + "▁ checked", + "▁P ac", + "▁Pa c", + "▁f ault", + "▁fa ult", + "▁y ellow", + "▁s alt", + "▁sa lt", + "▁sal t", + "▁Franc isco", + "▁Francis co", + "▁ ^", + "▁O N", + "▁ ON", + "▁beaut y", + "y g", + "▁A ff", + "▁Af f", + "▁ Aff", + "▁E q", + "▁ Eq", + "▁mag ic", + "▁hand ler", + "▁handle r", + "▁ handler", + "x E", + "▁numer ous", + "▁numero us", + "▁h ole", + "▁hol e", + "▁ho le", + "▁ hole", + "▁ro oms", + "▁room s", + "▁ rooms", + "cc ión", + "cció n", + "c ción", + "▁A rm", + "▁Ar m", + "▁ Arm", + "per son", + "pers on", + "p erson", + "▁build ings", + "▁building s", + "▁p late", + "▁pl ate", + "▁plat e", + "ble d", + "bl ed", + "b led", + "er rors", + "err ors", + "error s", + "▁A gain", + "▁Ag ain", + "▁Def ault", + "▁ Default", + "▁H ard", + "▁Har d", + "▁Ha rd", + "▁ Hard", + "t ó", + "hu s", + "h us", + "▁dim ension", + "ial e", + "ia le", + "i ale", + "▁M ult", + "▁Mu lt", + "▁Mul t", + "▁ Mult", + "▁Govern ment", + "Fun c", + "F unc", + "▁b low", + "▁bl ow", + "▁blo w", + "▁re ct", + "▁r ect", + "▁rec t", + "▁ rect", + "er ra", + "err a", + "conne ction", + "connect ion", + "conn ection", + "▁pass ing", + "▁pas sing", + "ße n", + "ß en", + "ph as", + "pha s", + "p has", + "ens ional", + "ension al", + "re cord", + "rec ord", + "co hol", + "▁H arry", + "▁Har ry", + "▁Harr y", + "izont al", + "izon tal", + "▁f inger", + "▁fin ger", + "▁fing er", + "▁young er", + "▁S C", + "▁ SC", + "oper ation", + "B Y", + "he im", + "▁B ad", + "▁Ba d", + "▁ Bad", + "▁st orm", + "▁stor m", + "▁sto rm", + "▁ storm", + "▁N at", + "▁Na t", + "▁bu ying", + "▁buy ing", + "▁S ometimes", + "▁Some times", + "▁С та", + "es sed", + "ess ed", + "esse d", + "▁da mn", + "▁dam n", + "▁m eg", + "▁me g", + "um es", + "ume s", + "u mes", + "ün d", + "ü nd", + "т ра", + "▁sil ver", + "w d", + "hid den", + "h idden", + "ar do", + "ard o", + "▁commun ities", + "▁d iet", + "▁di et", + "▁die t", + "ot ted", + "ott ed", + "otte d", + "▁b at", + "▁ba t", + "▁ bat", + "an cer", + "ance r", + "anc er", + "▁f mt", + "▁ fmt", + "▁P en", + "▁Pe n", + "▁ Pen", + "▁t il", + "▁ti l", + "▁ til", + "En um", + "E num", + "PA TH", + "P ATH", + "▁mat ters", + "▁matter s", + "▁matt ers", + "time out", + "-- ----------", + "---- --------", + "-------- ----", + "--- ---------", + "----- -------", + "---------- --", + "------ ------", + "--------- ---", + "------- -----", + "----------- -", + "- -----------", + "ka n", + "k an", + "▁Cor por", + "=\" ../../", + "=\"../ ../", + "▁A le", + "▁Al e", + "hent ication", + "hentic ation", + "▁com plic", + "▁comp lic", + "▁compl ic", + "▁Se curity", + "▁Sec urity", + "▁ Security", + "OF F", + "O FF", + "R ad", + "ap se", + "aps e", + "a pse", + "▁d ance", + "▁dan ce", + "▁perm issions", + "▁permission s", + "▁war rant", + "▁l ad", + "▁la d", + "▁ lad", + "▁is ol", + "▁i sol", + "d l", + "▁A u", + "ye s", + "y es", + "▁t v", + "▁ tv", + "▁pro vider", + "▁prov ider", + "▁provide r", + "▁ provider", + "▁ter rible", + "▁terr ible", + "▁dep artment", + "▁depart ment", + "er al", + "era l", + "e ral", + "▁implement ation", + "S R", + "▁h earing", + "▁he aring", + "▁hear ing", + "▁K n", + "F R", + "t v", + "▁d iss", + "▁dis s", + "▁di ss", + "F UN", + "▁dur ante", + "▁durant e", + "os is", + "osi s", + "o sis", + "▁task s", + "▁ tasks", + "▁B lo", + "▁Bl o", + "▁ Blo", + "во д", + "▁br anch", + "▁ branch", + "▁polit ics", + "▁E lle", + "▁El le", + "▁Ell e", + "▁lead ership", + "▁leader ship", + "▁leaders hip", + "ex pr", + "exp r", + "▁techn iques", + "▁technique s", + "pr ec", + "pre c", + "p rec", + "Sig ma", + "S igma", + "im ately", + "imate ly", + "imat ely", + "t k", + "ach ment", + "▁En ter", + "▁Ent er", + "▁ Enter", + "▁cre ative", + "▁creat ive", + "▁з на", + "▁ зна", + "ap py", + "app y", + "un ched", + "unch ed", + "unc hed", + "▁' ',", + "▁'' ,", + "on der", + "ond er", + "onde r", + "o nder", + "{ -", + "NU M", + "N UM", + "▁n arr", + "▁na rr", + "▁nar r", + "Mem ory", + "▁win ning", + "▁ winning", + "▁F ollow", + "▁Fol low", + "▁ Follow", + "*/ \r", + "vis ion", + "v ision", + "res ents", + "resent s", + "zi one", + "z ione", + "▁l atter", + "▁lat ter", + "▁requ ests", + "▁request s", + "▁ requests", + "▁m argin", + "▁mar gin", + "▁marg in", + "▁ margin", + "▁{ \"", + "▁ {\"", + "v ideo", + "c n", + "▁Im age", + "▁ Image", + "T im", + "CON FIG", + "CONF IG", + "▁all owing", + "▁allow ing", + "▁comb ined", + "▁combine d", + "PU T", + "P UT", + "▁instance of", + "ig in", + "igi n", + "i gin", + "▁p ero", + "▁per o", + "▁pe ro", + "▁' '", + "▁ ''", + "▁conf idence", + "▁equ ivalent", + "▁equival ent", + "pa d", + "p ad", + "ef fect", + "eff ect", + "e ffect", + "R X", + "▁l ang", + "▁la ng", + "▁lan g", + "▁ lang", + "str ong", + "▁b ridge", + "▁br idge", + "▁ bridge", + "ay a", + "a ya", + "▁t reated", + "▁tre ated", + "▁treat ed", + "▁f orth", + "▁for th", + "▁fort h", + "S W", + "▁account s", + "▁P O", + "▁ PO", + "▁list ening", + "▁listen ing", + "Ro ute", + "R oute", + "() ))", + "()) )", + "( )))", + "cp y", + "c py", + "▁re form", + "▁ref orm", + "▁g ate", + "▁ga te", + "▁ gate", + "▁W alk", + "▁Wal k", + "▁ Walk", + "▁some how", + "t f", + "▁l ayout", + "▁la yout", + "▁lay out", + "▁ layout", + "um in", + "umi n", + "u min", + "▁consider ing", + "▁consid ering", + "▁pre mi", + "▁pr emi", + "▁prem i", + "▁M om", + "▁Mo m", + "at han", + "ath an", + "a than", + "Ge n", + "G en", + "▁plan et", + "▁plane t", + "am ples", + "amp les", + "ample s", + "▁M O", + "▁ MO", + "sh op", + "s hop", + "▁prem ier", + "▁premi er", + "▁s impl", + "▁sim pl", + "▁s egu", + "▁se gu", + "▁seg u", + "L Y", + "Su m", + "S um", + "▁t ables", + "▁table s", + "▁tab les", + "▁ta bles", + "▁ tables", + "sk a", + "s ka", + "▁ ž", + "p d", + "▁s ous", + "▁so us", + "▁sou s", + "▁con ference", + "▁confer ence", + "▁D at", + "▁Da t", + "▁ Dat", + "Sc roll", + "▁stand ards", + "▁standard s", + "▁г ру", + "es se", + "ess e", + "▁citiz ens", + "▁citizen s", + "▁occur red", + "▁dem ocr", + "▁demo cr", + "▁e lev", + "▁el ev", + "▁ele v", + "▁S em", + "▁Se m", + "▁ Sem", + "ens us", + "he aders", + "head ers", + "header s", + "▁Ch ris", + "im ento", + "iment o", + "imen to", + "ko m", + "k om", + "Co r", + "C or", + "MI N", + "M IN", + "us her", + "ush er", + "Data base", + "Dat abase", + "▁f ormal", + "▁for mal", + "▁form al", + "▁forma l", + "ig ne", + "ign e", + "▁organ izations", + "▁organiz ations", + "▁organization s", + "▁I re", + "▁Ir e", + "X ml", + "и з", + "▁p ray", + "▁pr ay", + "▁pra y", + "▁b omb", + "▁bo mb", + "▁bom b", + "▁m and", + "▁man d", + "▁ma nd", + "▁ mand", + "er ts", + "ert s", + "▁c lock", + "▁cl ock", + "▁clo ck", + "▁ clock", + "▁b uck", + "▁bu ck", + "ва ли", + "вал и", + "в али", + "en sch", + "ens ch", + "▁v olt", + "▁vo lt", + "▁vol t", + "▁ volt", + "▁fil ms", + "▁film s", + "▁pl ants", + "▁plan ts", + "▁plant s", + "in ode", + "ino de", + "i node", + "Bo olean", + "▁restaur ant", + "ía n", + "í an", + "▁de but", + "▁deb ut", + "page s", + "pa ges", + "pag es", + "p ages", + "▁wor dt", + "▁word t", + "▁Б а", + "▁great est", + "(\" /", + "▁c opyright", + "▁copy right", + "▁ copyright", + "▁r it", + "▁ri t", + "▁ rit", + "size of", + "Tr ace", + "Tra ce", + "ue nt", + "uen t", + "u ent", + "ту р", + "т ур", + "▁k o", + "▁ ko", + ": \\", + "▁b igger", + "▁big ger", + "▁perfect ly", + "ten ance", + "MA SK", + "M ASK", + "r é", + "▁e tt", + "▁et t", + "▁ ett", + "▁n ose", + "▁no se", + "▁nos e", + "▁c raft", + "▁cr aft", + "▁ craft", + "it eral", + "ite ral", + "iter al", + "▁discuss ed", + "▁Jew ish", + "C ap", + "▁Un less", + "▁Jack son", + "Att ributes", + "Attribute s", + "Attrib utes", + "▁l unch", + "▁lun ch", + "ö l", + "at r", + "a tr", + "▁pay ing", + "▁pa ying", + "Par se", + "Pars e", + "P arse", + "() \r", + "( )\r", + "la d", + "l ad", + "▁r are", + "▁ra re", + "▁[ ];", + "▁[] ;", + "▁ [];", + "st one", + "ston e", + "sto ne", + "▁u nc", + "▁un c", + "▁ unc", + "▁def ense", + "▁defens e", + "} +", + "▁Gl obal", + "▁ Global", + "▁Sov iet", + "▁Austral ian", + "▁Australia n", + "▁g li", + "▁gl i", + "var iant", + "vari ant", + "▁R on", + "▁Ro n", + "▁lo an", + "St ep", + "Ste p", + "me mber", + "mem ber", + "m ember", + "Sc h", + "S ch", + "▁Commit tee", + "▁s pending", + "▁sp ending", + "▁spend ing", + "▁T ri", + "▁Tr i", + "▁ Tri", + "▁J ournal", + "▁Jour nal", + "▁ Journal", + "▁s ugar", + "▁su gar", + "▁sug ar", + "el ly", + "ell y", + "HT ML", + "▁ad vent", + "▁adv ent", + "win g", + "wi ng", + "w ing", + "▁Wh ether", + "▁Whe ther", + "or ation", + "▁N E", + "▁ NE", + "iv eness", + "ive ness", + "iven ess", + "▁h av", + "▁ha v", + "▁ hav", + "▁con scious", + "▁ conscious", + "ee n", + "e en", + "Sym bol", + "S ymbol", + "▁к у", + "▁ ку", + "Log ger", + "▁L ittle", + "▁Lit tle", + "wide t", + "wi det", + "wid et", + "oc ation", + "pi n", + "p in", + "▁sym met", + "▁A D", + "▁ AD", + "▁pos ts", + "▁po sts", + "▁post s", + "▁ posts", + "sh al", + "sha l", + "s hal", + "▁Con f", + "▁Co nf", + "▁ Conf", + "▁ch ose", + "▁cho se", + "ma l", + "m al", + "ul o", + "u lo", + "▁M ethod", + "▁ Method", + "▁miss ed", + "▁mis sed", + "Re move", + "Rem ove", + "Aut o", + "A uto", + "VAL UE", + "th let", + "▁For ce", + "▁ Force", + "p f", + "▁ Я", + "la te", + "lat e", + "l ate", + "▁p ul", + "▁pu l", + "▁ pul", + "Po p", + "P op", + "▁adv anced", + "▁advance d", + "air es", + "ai res", + "aire s", + "a ires", + "res sed", + "ress ed", + "resse d", + "r essed", + "AM E", + "A ME", + "be ll", + "bel l", + "b ell", + "ac hing", + "ach ing", + "achi ng", + "a ching", + "i ć", + "ec ho", + "ech o", + "e cho", + "H S", + "▁fun ny", + "ри и", + "▁e er", + "▁ve get", + "▁four th", + "c f", + "trans form", + "▁g rown", + "▁gr own", + "▁grow n", + "▁gro wn", + "▁Mc C", + "si te", + "s ite", + "▁b eneath", + "▁be neath", + "▁s hell", + "▁sh ell", + "▁she ll", + "▁shel l", + "▁ shell", + "x d", + "Pl ay", + "P lay", + "sh ort", + "Ro le", + "R ole", + "▁relig ion", + "in ator", + "ina tor", + "} <", + "\" ><", + "as p", + "a sp", + "aj o", + "a jo", + "ex ports", + "exp orts", + "export s", + "▁N ode", + "▁No de", + "▁ Node", + "▁j ako", + "▁ja ko", + "▁jak o", + "▁y a", + "▁ ya", + "▁success fully", + "▁successful ly", + "▁friend ly", + "▁ friendly", + "buf f", + "bu ff", + "b uff", + "DE FAULT", + "▁pre gn", + "▁preg n", + "Requ ired", + "Require d", + "▁b inary", + "▁bin ary", + "▁ binary", + "is ting", + "ist ing", + "isti ng", + "▁st ared", + "▁star ed", + "▁stare d", + "▁sta red", + "▁circum stances", + "▁х о", + "▁ хо", + "re i", + "r ei", + "▁Г о", + "Trans form", + "cn t", + "c nt", + "▁E xt", + "▁Ex t", + "▁ Ext", + "re port", + "rep ort", + "repo rt", + "VER SION", + "▁an aly", + "▁anal y", + "▁ analy", + "▁M arg", + "▁Mar g", + "▁Ma rg", + "▁al leg", + "▁all eg", + "▁alle g", + "build er", + "b uilder", + "To String", + "La yer", + "L ayer", + "ís t", + "í st", + "Pro p", + "Pr op", + "P rop", + "▁E mp", + "▁Em p", + "▁ Emp", + "} ]", + "▁s elling", + "▁sell ing", + "▁sel ling", + "▁ selling", + "▁que ue", + "▁ queue", + "▁ser iously", + "▁serious ly", + "▁L ead", + "▁Le ad", + "▁ Lead", + "text it", + "tex tit", + "test ing", + "tes ting", + "▁П ре", + "se curity", + "sec urity", + "ia ł", + "i ał", + "ú n", + "ch ip", + "chi p", + "c hip", + "▁c andidate", + "▁candid ate", + "▁min ister", + "▁mini ster", + "▁minist er", + "▁ minister", + "er ia", + "eri a", + "e ria", + "▁H et", + "▁He t", + "ди н", + "д ин", + "▁Brit ain", + "▁b arely", + "▁bar ely", + "▁bare ly", + "▁s ty", + "▁st y", + "▁ sty", + "▁Span ish", + "▁V en", + "▁Ve n", + "time r", + "ti mer", + "tim er", + "t imer", + "кі в", + "к ів", + "▁document s", + "▁doc uments", + "(' .", + "( '.", + "▁d ebug", + "▁de bug", + "▁deb ug", + "▁ debug", + "▁cont ro", + "▁contr o", + "сто я", + "▁j oy", + "▁jo y", + "▁ joy", + "S n", + "In v", + "I nv", + "▁pro tocol", + "▁proto col", + "▁prot ocol", + "▁ protocol", + "▁f aces", + "▁face s", + "▁fac es", + "▁fa ces", + "▁ faces", + "▁Des pite", + "se d", + "s ed", + "Con f", + "Co nf", + "AR G", + "A RG", + "▁e volution", + "▁ev olution", + "▁t od", + "▁to d", + "▁P romise", + "▁Prom ise", + "▁ Promise", + "▁pos ted", + "▁po sted", + "▁post ed", + "Per m", + "Pe rm", + "P erm", + "be t", + "b et", + "An g", + "A ng", + "J ust", + "▁r um", + "▁ru m", + "▁ rum", + "la yer", + "lay er", + "l ayer", + "▁beh avi", + "▁behav i", + "ip ping", + "ipp ing", + "ippi ng", + "i pping", + "▁d ynam", + "▁dy nam", + "▁dyn am", + "▁sch eme", + "▁sche me", + "▁ scheme", + "▁pro to", + "▁pr oto", + "▁prot o", + "▁ proto", + ") /", + "Col lections", + "Collection s", + "Collect ions", + "ri ev", + "rie v", + "r iev", + "▁C lick", + "▁Cl ick", + "▁ Click", + "▁u ns", + "▁un s", + "▁ uns", + "wide tilde", + "widet ilde", + "▁remember ed", + "г і", + "in ates", + "ina tes", + "inate s", + "▁incor por", + "▁De scription", + "▁Des cription", + "▁ Description", + "▁pre pare", + "▁prep are", + "▁prepar e", + "▁ prepare", + "▁F inal", + "▁Fin al", + "▁Fi nal", + "▁ Final", + "u ation", + "▁Qu een", + "▁Que en", + "> ;", + "▁autom atically", + "▁automatic ally", + "▁sh arp", + "▁shar p", + "▁sha rp", + "▁me at", + "at eur", + "ate ur", + "as tern", + "ast ern", + "aster n", + "aste rn", + "▁st uck", + "ASS ERT", + "▁pl anned", + "▁plan ned", + "do ts", + "dot s", + "d ots", + "ook ie", + "oo kie", + "▁His tor", + "▁Hist or", + "▁re views", + "▁review s", + "IM P", + "I MP", + "▁answ ered", + "▁answer ed", + "To tal", + "T otal", + "▁s au", + "▁sa u", + "▁Me xico", + "▁Mex ico", + "contin ue", + "▁App le", + "▁Ap ple", + "like ly", + "lik ely", + "з ва", + "us ers", + "use rs", + "user s", + "▁ident ified", + "▁L ev", + "▁Le v", + "▁m ol", + "▁mo l", + "▁Is lam", + "▁com mitted", + "▁comm itted", + "▁commit ted", + "wr it", + "w rit", + "бе р", + "б ер", + "ri ft", + "rif t", + "r ift", + "▁inter rupt", + "▁ interrupt", + "▁read only", + "sch ema", + "sche ma", + "s chema", + "S m", + "D ouble", + "az a", + "a za", + "▁H al", + "▁Ha l", + "▁ Hal", + "Mo ve", + "M ove", + "▁S eries", + "▁Se ries", + "▁Ser ies", + "▁Serie s", + "▁ Series", + "in line", + "▁кото ры", + "so c", + "s oc", + "▁t ent", + "▁te nt", + "▁ten t", + "▁a mer", + "▁am er", + "▁ amer", + "ak i", + "a ki", + "▁l ady", + "▁la dy", + "▁lad y", + "▁t ired", + "▁ti red", + "▁tire d", + "▁tir ed", + "if i", + "i fi", + "▁m ême", + "▁ même", + "ou ver", + "▁a side", + "▁as ide", + "Di d", + "D id", + "', \r", + "' ,\r", + "▁br inging", + "▁bring ing", + "Draw ing", + "ar o", + "a ro", + "▁R h", + "▁N az", + "▁Na z", + "es so", + "ess o", + "▁re action", + "▁react ion", + "mit ted", + "mitt ed", + "m itted", + "▁abs olute", + "▁absolut e", + "▁ absolute", + "ha ust", + "haus t", + "(( )", + "( ()", + "▁T ask", + "▁Ta sk", + "▁ Task", + "ER S", + "E RS", + "▁^ {", + "▁ ^{", + "V D", + "▁t one", + "▁to ne", + "▁ton e", + "dis t", + "di st", + "d ist", + "v s", + "▁whe el", + "▁ wheel", + "▁administr ation", + "▁admin istration", + "▁inter ests", + "▁interest s", + "▁point er", + "▁po inter", + "▁ pointer", + "▁en counter", + "▁enc ounter", + "ave r", + "av er", + "a ver", + "▁n ord", + "▁no rd", + "▁nor d", + "ke t", + "k et", + "▁b each", + "▁be ach", + "▁enjoy ed", + "cont ains", + "▁app end", + "▁ap pend", + "▁appe nd", + "▁ append", + "W ait", + "▁s quad", + "▁squ ad", + "ze l", + "z el", + "▁med ium", + "▁medi um", + "▁ medium", + "▁s ending", + "▁send ing", + "▁sen ding", + "▁L ady", + "▁La dy", + "▁Lad y", + "ç ões", + "▁dest ination", + "▁destin ation", + "▁ destination", + "ny ch", + "n ych", + "▁conf lict", + "▁conflic t", + "▁L y", + "▁v ul", + "▁vu l", + "▁bas ically", + "▁basic ally", + "re ated", + "reat ed", + "reate d", + "rea ted", + "bl ack", + "ug ins", + "ugin s", + "▁cal m", + "▁ca lm", + "ér ie", + "éri e", + "é rie", + "ha r", + "h ar", + "ла н", + "л ан", + "▁С е", + "w atch", + "▁P ut", + "▁Pu t", + "▁ Put", + "▁d ump", + "▁du mp", + "▁ dump", + "ac her", + "ach er", + "ache r", + "a cher", + "sc roll", + "scr oll", + "▁cl aimed", + "▁claim ed", + "▁ claimed", + "▁Cont rol", + "▁ Control", + "▁bl ind", + "en ti", + "ent i", + "▁Ke ep", + "▁ Keep", + "▁Develop ment", + "im ages", + "image s", + "ima ges", + "imag es", + "▁t ough", + "▁to ugh", + "▁tou gh", + "ge bra", + "geb ra", + "▁se pt", + "▁sep t", + "he w", + "h ew", + "▁s kill", + "▁sk ill", + "▁ski ll", + "▁ skill", + "▁T ay", + "▁Ta y", + "▁k tó", + "ow ner", + "own er", + "par e", + "pa re", + "p are", + "▁f ee", + "▁fe e", + "▁ fee", + "▁contin ues", + "▁continue s", + "▁continu es", + "▁k an", + "▁ka n", + "▁ kan", + "be s", + "b es", + "▁c ha", + "▁ch a", + "▁ cha", + "ov o", + "o vo", + "▁N ight", + "▁Ni ght", + "ict ure", + "sh ire", + "s hire", + "▁es say", + "▁ess ay", + "▁sup pose", + "▁supp ose", + "et ic", + "eti c", + "Ar t", + "A rt", + "ac on", + "aco n", + "a con", + "ll a", + "l la", + "word s", + "wor ds", + "w ords", + "▁compar ison", + "▁B E", + "▁ BE", + "▁challeng es", + "▁challenge s", + "▁o l", + "▁ ol", + "cite p", + "cit ep", + "▁F oot", + "▁Fo ot", + "▁ Foot", + "▁S uch", + "▁Su ch", + "▁ Such", + "▁p apers", + "▁paper s", + "▁pa pers", + "▁pap ers", + "act iv", + "qu er", + "que r", + "q uer", + "т я", + "▁Т о", + "сь кий", + "th ur", + "do ne", + "don e", + "d one", + "▁sh ock", + "▁ded icated", + "▁dedic ated", + "▁cor respond", + "▁correspon d", + "Se cond", + "Sec ond", + "▁b ull", + "▁bu ll", + "▁bul l", + "li fe", + "lif e", + "l ife", + "ind ent", + "inde nt", + "inden t", + "▁fig ures", + "▁figure s", + "▁And rew", + "▁Andre w", + "▁Andr ew", + "is p", + "i sp", + "▁fav our", + "зд а", + "з да", + "▁E lect", + "▁El ect", + "▁Ele ct", + "F ull", + "▁near by", + "▁Reg ister", + "▁ Register", + "Sc ale", + "Scal e", + "ic ations", + "ication s", + "и н", + "▁A M", + "▁ AM", + "pa ir", + "p air", + "▁pers pective", + "▁n os", + "▁no s", + "▁ nos", + "ap a", + "a pa", + "ost ał", + "osta ł", + "▁P ers", + "▁Per s", + "▁Pe rs", + "▁ Pers", + "ic er", + "ice r", + "i cer", + "▁pl astic", + "до в", + "д ов", + "ci ples", + "cipl es", + "cip les", + "z ą", + "cl os", + "c los", + "▁у ча", + "▁ Á", + "pl ugin", + "plug in", + "▁an gle", + "▁ang le", + "▁angl e", + "▁ angle", + "▁com mission", + "▁comm ission", + "▁fun ds", + "▁fund s", + "▁in du", + "▁ind u", + "▁d rawn", + "▁dr awn", + "▁draw n", + "á m", + "▁develop ing", + "▁seg ment", + "▁ segment", + "is me", + "ism e", + "sc r", + "s cr", + "▁l ies", + "▁li es", + "▁lie s", + "▁I L", + "▁ IL", + "▁a pi", + "▁ap i", + "▁ api", + "Ext ension", + "▁s cal", + "▁sc al", + "▁ scal", + "inst all", + "▁We ek", + "▁ Week", + "▁gen tle", + "▁gent le", + "▁Canad ian", + "▁d ialog", + "▁dial og", + "▁dia log", + "▁ dialog", + "▁art icles", + "▁article s", + "▁artic les", + "The me", + "Th eme", + "S M", + "▁B ul", + "▁Bu l", + "▁ Bul", + "▁l eur", + "▁le ur", + "▁s tom", + "▁st om", + "▁sto m", + "Pl ugin", + "▁по сле", + "▁пос ле", + "▁st ead", + "▁ste ad", + "▁ stead", + "▁ ś", + "ip her", + "iph er", + "i pher", + "▁pr ze", + "▁prz e", + "▁d raft", + "▁dr aft", + "▁ draft", + "bot tom", + "b ottom", + "▁{ };", + "▁{} ;", + "▁stay ed", + "fe ature", + "feat ure", + "▁v ot", + "▁vo t", + "▁fab ric", + "ç a", + "(' #", + "re a", + "r ea", + "▁re put", + "▁rep ut", + "▁C ir", + "▁Ci r", + "▁ Cir", + "▁A L", + "▁ AL", + "▁assert Equals", + "▁ assertEquals", + "result s", + "▁C ross", + "▁Cr oss", + "▁Cro ss", + "▁ Cross", + "urs day", + "▁a udio", + "▁aud io", + "▁ audio", + "▁g ap", + "▁ga p", + "▁stre ets", + "▁street s", + "▁scient ific", + "pl atform", + "▁a uss", + "▁au ss", + "▁aus s", + "▁C ro", + "▁Cr o", + "▁part ial", + "▁parti al", + "▁ partial", + "un c", + "u nc", + "▁cho ices", + "▁choice s", + "▁и ли", + "pr ed", + "pre d", + "p red", + "▁he ads", + "▁head s", + "▁ heads", + "ter day", + "▁N ick", + "▁Nic k", + "▁Ni ck", + "▁we ird", + "as ant", + "asa nt", + "▁represent ed", + "▁п и", + "▁ пи", + "D P", + "or ders", + "ord ers", + "order s", + "cl ock", + "c lock", + "▁H o", + "ar ters", + "art ers", + "arter s", + "arte rs", + "C md", + "og a", + "o ga", + "Key s", + "Ke ys", + "Re port", + "Rep ort", + "Repo rt", + "▁V ill", + "▁Vi ll", + "▁Vil l", + "▁M u", + "▁ Mu", + "▁own ed", + "▁ owned", + "SU CCESS", + "▁type of", + "▁ typeof", + "hd r", + "h dr", + "ua ble", + "u able", + "▁neighbor hood", + "▁A P", + "▁ AP", + "▁result ing", + "▁sh adow", + "▁ shadow", + "STR ING", + "▁video s", + "▁vide os", + "ле ння", + "лен ня", + "ex pect", + "exp ect", + "▁Val ley", + "▁Vall ey", + "▁g oto", + "▁go to", + "▁got o", + "▁ goto", + "▁S her", + "▁She r", + "▁Sh er", + "fr astr", + "▁oper ating", + "▁opera ting", + "▁э то", + "▁License d", + "▁Lic ensed", + "Var iable", + "Vari able", + "▁P R", + "▁ PR", + "▁H ans", + "▁Ha ns", + "▁Han s", + "cl one", + "▁G esch", + "▁Ge sch", + "▁Ges ch", + "▁B and", + "▁Ba nd", + "▁Ban d", + "▁ Band", + "... .....", + ".... ....", + "..... ...", + "ui ng", + "u ing", + "▁hundred s", + "▁о к", + "▁emot ional", + "▁emotion al", + "▁Ind ust", + ") +", + "▁Egy pt", + "▁fr anç", + "▁ š", + "▁f asc", + "▁fa sc", + "on to", + "ont o", + "▁A dam", + "▁Ad am", + "▁l aid", + "▁la id", + "▁r ig", + "▁ri g", + "▁ rig", + "▁det ailed", + "▁detail ed", + "▁im plements", + "▁implement s", + "▁impl ements", + "▁univers ity", + "▁H y", + "▁ Hy", + "▁g rid", + "▁gr id", + "▁gri d", + "▁ grid", + "▁reg ions", + "▁region s", + "St op", + "S top", + "▁s lot", + "▁sl ot", + "▁ slot", + "▁ang ry", + "▁- =", + "▁wait ed", + "▁wa ited", + "Ver t", + "V ert", + "\": \"", + "\" :\"", + "▁e lem", + "▁el em", + "▁ele m", + "▁ elem", + "▁r ég", + "▁ré g", + "ow ed", + "owe d", + "o wed", + "Mem ber", + "Me mber", + "M ember", + "▁r atio", + "▁rat io", + "▁ ratio", + "is en", + "ise n", + "i sen", + "▁L em", + "▁Le m", + "ge ry", + "ger y", + "g ery", + "▁c ream", + "▁cre am", + "▁ét ait", + "▁ était", + "▁g eb", + "▁ge b", + "▁ geb", + "un ique", + "uni que", + "▁D eb", + "▁De b", + "▁f actory", + "▁fact ory", + "▁factor y", + "▁ factory", + "ż e", + "d ialog", + "▁Con fig", + "▁Conf ig", + "▁ Config", + "Sy nc", + "S ync", + "an gers", + "ang ers", + "ange rs", + "anger s", + "▁gover ning", + "▁govern ing", + "▁H un", + "▁Hu n", + "Sp ace", + "S pace", + "▁j est", + "▁je st", + "ic ious", + "ici ous", + "icio us", + "▁em phas", + "▁emp has", + "um ps", + "ump s", + "▁E sp", + "▁Es p", + "▁ Esp", + "▁s ul", + "▁su l", + "▁histor ical", + "▁historic al", + "ij a", + "i ja", + "▁l ying", + "▁ly ing", + "▁ lying", + "▁St eve", + "▁Ste ve", + "▁me asures", + "▁measure s", + "▁meas ures", + "os to", + "ost o", + "o sto", + "? ”", + "▁p ocket", + "▁poc ket", + "▁S at", + "▁Sa t", + "▁p itch", + "▁pit ch", + "▁n atur", + "▁nat ur", + "▁hum ans", + "▁human s", + "▁Sim on", + "▁Si mon", + "ad ores", + "ado res", + "ador es", + "(\" \\", + "( \"\\", + "in king", + "ink ing", + "▁ex pos", + "▁exp os", + "mat erial", + "mate rial", + "m aterial", + "▁app arently", + "▁apparent ly", + "▁appar ently", + "▁C amb", + "▁Cam b", + "▁Ca mb", + "▁B ox", + "▁Bo x", + "▁ Box", + "▁s paces", + "▁sp aces", + "▁space s", + "ex ists", + "exist s", + "▁act ing", + "▁ac ting", + "OR Y", + "зо ва", + "Go od", + "G ood", + "ien ne", + "i enne", + "▁William s", + "▁f ruit", + "▁fr uit", + "▁fru it", + "ie ra", + "ier a", + "i era", + "▁L im", + "▁Li m", + "▁ Lim", + "▁t rait", + "▁tr ait", + "▁tra it", + "▁ trait", + "▁art ists", + "▁artist s", + "▁ab sor", + "▁abs or", + "ra it", + "rai t", + "r ait", + "LO AD", + "▁mov ies", + "▁movie s", + "▁d ynamic", + "▁dynam ic", + "▁dyn amic", + "▁ dynamic", + "as ts", + "ast s", + "a sts", + "▁In teger", + "▁ Integer", + "▁sm oke", + "п і", + "an gel", + "ang el", + "ange l", + ">( \"", + "> (\"", + "▁in strument", + "▁instr ument", + "▁f uel", + "▁fue l", + "▁fu el", + "но ї", + "atal ogue", + "atalog ue", + "▁s erial", + "▁se rial", + "▁ser ial", + "▁ serial", + "File s", + "Fil es", + "Fi les", + "F iles", + "▁bath room", + "il o", + "i lo", + "es to", + "est o", + "e sto", + "▁p m", + "▁ pm", + "ent ials", + "ential s", + "enti als", + "▁On line", + "wh ite", + "▁t ips", + "▁tip s", + "▁ti ps", + "▁cap able", + "Fi g", + "F ig", + "T V", + "▁о н", + "▁ он", + "k é", + "bit r", + "bi tr", + "b itr", + "Map ping", + "Ma pping", + "M apping", + "▁t ak", + "▁ta k", + "ю щи", + "в ля", + ")\" ,", + ") \",", + "▁K arl", + "▁Kar l", + "▁Ka rl", + "▁H uman", + "▁Hu man", + "▁Hum an", + "▁P ot", + "▁Po t", + "▁rep resents", + "▁represent s", + "▁cons istent", + "▁consist ent", + "_ (", + "we n", + "w en", + "▁R ose", + "▁Ro se", + "▁Ros e", + "la w", + "l aw", + "▁F ROM", + "▁FR OM", + "▁ FROM", + "▁beg ins", + "▁begin s", + "▁e dit", + "▁ed it", + "▁ edit", + "▁mount ain", + "▁ch apter", + "▁chap ter", + "▁wonder ed", + "▁indust rial", + "▁M ajor", + "▁Ma jor", + "▁Maj or", + "▁g es", + "▁ge s", + "▁ ges", + "▁direct ed", + "▁dire cted", + "er os", + "ero s", + "e ros", + "▁W ild", + "▁Wil d", + "▁Wi ld", + "li ament", + "lia ment", + "Bo ok", + "B ook", + "user name", + "ho t", + "h ot", + "▁n am", + "▁na m", + "▁ nam", + "▁le ague", + "br a", + "b ra", + "ко н", + "к он", + "▁T al", + "▁Ta l", + "▁В а", + "▁ex ports", + "▁exp orts", + "▁export s", + "▁ exports", + "( @", + "▁sh aring", + "▁shar ing", + "▁sha ring", + "▁T ro", + "▁Tr o", + "ś ć", + "ues day", + "yl v", + "y lv", + "▁gu itar", + "el en", + "ele n", + "e len", + "Se lection", + "Select ion", + "S election", + "▁conf ident", + "ry pto", + "rypt o", + "▁h ors", + "▁hor s", + "▁ho rs", + "ed itor", + "edit or", + "edi tor", + "▁should ers", + "▁shoulder s", + "get Name", + "en cing", + "enc ing", + "enci ng", + "SE LECT", + "SEL ECT", + "в ши", + "▁kind s", + "▁kin ds", + "▁W el", + "▁We l", + "▁pur poses", + "▁purpose s", + "Mat rix", + "in valid", + "▁own ers", + "▁owner s", + "▁ owners", + "▁Rec ords", + "▁Record s", + "▁ Records", + "▁Pro cess", + "▁ Process", + "▁c hat", + "▁ch at", + "▁cha t", + "▁ chat", + "▁D or", + "▁Do r", + "▁b in", + "▁bi n", + "▁ bin", + "re dit", + "red it", + "r edit", + "oi re", + "oir e", + "o ire", + "▁T otal", + "▁To tal", + "▁Tot al", + "▁ Total", + "▁F amily", + "▁Famil y", + "▁ Family", + "AR Y", + "▁b read", + "▁br ead", + "▁bre ad", + "▁ bread", + "▁com pre", + "▁comp re", + "▁compr e", + "▁sh oes", + "▁shoe s", + "▁r az", + "▁ra z", + "▁ raz", + "▁tr ace", + "▁tra ce", + "▁ trace", + "ne j", + "n ej", + "or ted", + "ort ed", + "orte d", + "h n", + "▁pro cedure", + "▁proced ure", + "pro perties", + "pl ier", + "▁h ero", + "▁he ro", + "▁her o", + "▁ hero", + "pan el", + "pa nel", + "p anel", + "▁mark ed", + "▁mar ked", + "▁wor ried", + "\\ |", + "pt s", + "p ts", + "▁S upport", + "▁Sup port", + "▁Supp ort", + "▁ Support", + "▁ser ving", + "▁serv ing", + "F ail", + "▁dis appoint", + "▁Sc ot", + "▁ple asure", + "▁j udge", + "▁jud ge", + "▁judg e", + "ze ich", + "▁for ever", + "▁fore ver", + "▁Ze it", + "uo us", + "u ous", + "in ent", + "ine nt", + "inen t", + "i nent", + "▁d w", + "▁ dw", + "▁w aren", + "▁war en", + "▁wa ren", + "▁ware n", + "▁fl ash", + "▁ flash", + "▁tro ops", + "▁dr ugs", + "▁dru gs", + "▁drug s", + "▁d iam", + "▁di am", + "▁dia m", + ". ~", + "im p", + "i mp", + "in ned", + "inn ed", + "▁E V", + "▁ EV", + "St ruct", + "Str uct", + "▁just ice", + "▁offic ials", + "▁official s", + "ff ff", + "fff f", + "f fff", + "▁Com mon", + "▁Comm on", + "▁ Common", + "▁C at", + "▁Ca t", + "▁ Cat", + "▁tom orrow", + "▁é l", + "▁ él", + "Text ure", + "Te xture", + "qp oint", + "q point", + "▁F ried", + "▁Fr ied", + "▁T erm", + "▁Te rm", + "▁Ter m", + "▁ Term", + "pgf qpoint", + "▁n em", + "▁ne m", + "▁ nem", + "no rm", + "nor m", + "n orm", + "▁hard ly", + "od a", + "o da", + "ze ta", + "zet a", + "z eta", + "em ic", + "emi c", + "e mic", + "▁по лу", + "▁пол у", + "▁lo aded", + "▁load ed", + "▁ loaded", + "ke s", + "k es", + "ci ó", + "c ió", + "▁f ool", + "▁fo ol", + "▁foo l", + "▁t rick", + "▁tr ick", + "▁tri ck", + "▁d st", + "▁ds t", + "▁ dst", + "Fin d", + "Fi nd", + "F ind", + "▁в се", + "}} ,", + "} },", + "▁frame work", + "▁ framework", + "▁mer ely", + "▁mere ly", + "▁un ion", + "▁ union", + "▁Ed ward", + "ri f", + "r if", + "Fl ag", + "F lag", + "▁cris is", + "▁cri sis", + "▁fin ite", + "▁ finite", + "▁l ol", + "▁lo l", + "▁K im", + "▁Ki m", + "на та", + "sin ce", + "s ince", + "▁com pat", + "▁comp at", + "▁ compat", + "▁p ert", + "▁per t", + "▁pe rt", + "▁ pert", + "ib ilities", + "ibil ities", + "▁tamb ién", + "ib li", + "▁t een", + "▁te en", + "▁ teen", + "▁sym pt", + "or al", + "ora l", + "o ral", + "de rs", + "der s", + "d ers", + "ot te", + "ott e", + "п ри", + "▁J ane", + "▁Jan e", + "▁Ja ne", + "▁original ly", + "▁origin ally", + "▁thro at", + "ma g", + "m ag", + "su p", + "s up", + "un i", + "u ni", + "$ $", + "▁L ibrary", + "▁ Library", + "▁att acks", + "▁attack s", + "in gen", + "ing en", + "inge n", + "(' /", + "▁h es", + "▁he s", + "▁ hes", + "co in", + "c oin", + "oun ce", + "▁Academ y", + "MOD ULE", + "is ms", + "ism s", + "▁A dv", + "▁Ad v", + "▁ Adv", + "▁B ol", + "▁Bo l", + "▁inc ident", + ")^ {", + ") ^{", + "▁b ij", + "▁bi j", + "▁R ome", + "▁Rom e", + "▁Ro me", + "▁It aly", + "▁Ital y", + "ev ents", + "event s", + "even ts", + "▁F ern", + "▁Fe rn", + "▁Fer n", + "▁b er", + "▁be r", + "▁ ber", + "▁sil ent", + "▁p ier", + "▁pie r", + "▁pi er", + "▁Y O", + "▁pl ain", + "▁ plain", + "B as", + "▁p ill", + "▁pi ll", + "▁pil l", + "ra se", + "ras e", + "r ase", + "▁car rying", + "▁carry ing", + "▁re sp", + "▁r esp", + "▁res p", + "▁ resp", + "ну ю", + "▁typ ical", + "Wrap per", + "W rapper", + "▁g au", + "▁ga u", + "▁chem ical", + "▁h al", + "▁ha l", + "▁ hal", + "th row", + "Cl uster", + "▁G ab", + "▁Ga b", + "▁G irl", + "▁Gi rl", + "▁Gir l", + "qu ir", + "▁A rg", + "▁Ar g", + "▁ Arg", + "▁rel ief", + "▁relie f", + "▁reli ef", + "▁В е", + "d m", + "▁fr ustr", + "▁fru str", + "\\ %", + "▁st ores", + "▁store s", + "▁stor es", + "▁sto res", + "▁bott le", + "▁bot tle", + "▁L ew", + "▁Le w", + "tw o", + "t wo", + "st ad", + "sta d", + "▁che ek", + "▁concern s", + "▁concer ns", + "▁help ful", + "▁co verage", + "▁cover age", + "is i", + "i si", + "AD D", + "A DD", + "as ync", + "asy nc", + "a sync", + "▁approxim ately", + "▁approx imately", + "▁approximate ly", + "if fer", + "iff er", + "iffe r", + "ho ok", + "h ook", + "▁e num", + "▁en um", + "▁ enum", + "ov á", + "o vá", + "▁e vil", + "▁ev il", + "▁const antly", + "▁constant ly", + "ap ply", + "app ly", + "▁si è", + "▁pract ices", + "▁practice s", + "▁te achers", + "▁teach ers", + "▁teacher s", + "▁S n", + "▁ Sn", + "▁A wards", + "▁Award s", + "▁Aw ards", + "▁sub stant", + "▁subst ant", + "▁$ .", + "▁ $.", + "d k", + "▁m ob", + "▁mo b", + "▁ mob", + "▁ing red", + "ve re", + "ver e", + "v ere", + "Mult i", + "пе р", + "п ер", + "st al", + "sta l", + "s tal", + "ya rd", + "yar d", + "y ard", + "requ ired", + "require d", + "ve ment", + "v ement", + "▁int elligence", + "▁intellig ence", + "▁th inks", + "▁think s", + "▁thin ks", + "▁person ally", + "▁personal ly", + "▁tr ained", + "▁tra ined", + "▁train ed", + "▁ trained", + "or ney", + "orn ey", + "orne y", + ") \\", + "an al", + "ana l", + "a nal", + "Se ction", + "S ection", + "pl us", + "ü t", + "▁em bed", + "▁emb ed", + "▁ embed", + "▁st rings", + "▁str ings", + "▁string s", + "▁ strings", + "Be fore", + "B efore", + "pro c", + "pr oc", + "p roc", + "▁с по", + "▁сп о", + "▁ спо", + "tr l", + "t rl", + "v r", + "Back ground", + "log ger", + "ag raph", + "agr aph", + "agra ph", + "a graph", + "ie st", + "ies t", + "i est", + "▁good s", + "bat ch", + "b atch", + "▁opt ional", + "▁option al", + "▁ optional", + "▁Tay lor", + "▁recogn ize", + "wal k", + "w alk", + "▁H it", + "▁Hi t", + "▁ Hit", + "▁Eliz abeth", + "} :", + "▁care ful", + "кра ї", + "▁loc ations", + "▁location s", + "▁struct ures", + "▁structure s", + "▁d isk", + "▁dis k", + "▁di sk", + "▁ disk", + "▁sh ips", + "▁ship s", + "▁ ships", + "▁s uo", + "▁su o", + "▁s owie", + "▁so wie", + "▁sow ie", + "▁E ss", + "▁Es s", + "▁H ash", + "▁Ha sh", + "▁Has h", + "▁ Hash", + "▁reason able", + "▁More over", + "▁form ula", + "▁C entre", + "▁Cent re", + "▁res idents", + "▁resident s", + "▁resid ents", + "R S", + "Id s", + "I ds", + "▁K now", + "▁Kn ow", + "▁t rib", + "▁tr ib", + "▁tri b", + "▁r és", + "▁ré s", + "▁s table", + "▁st able", + "▁sta ble", + "▁stab le", + "▁ stable", + "▁W ould", + "▁Wo uld", + "▁ Would", + "▁break ing", + "▁bre aking", + "▁ breaking", + "▁me al", + "▁p hen", + "▁ph en", + "▁f el", + "▁fe l", + "▁ fel", + "▁F red", + "▁Fr ed", + "▁Fre d", + "Aut hor", + "Auth or", + "▁c apture", + "▁capt ure", + "▁ capture", + "op ts", + "opt s", + "o pts", + "▁every where", + "▁s que", + "▁squ e", + "▁sq ue", + "▁m oder", + "▁mod er", + "▁mo der", + "▁mode r", + "set up", + "▁S upp", + "▁Su pp", + "▁Sup p", + "▁ Supp", + "▁when ever", + "▁whe never", + "{ (", + "wa rt", + "war t", + "w art", + "▁t oe", + "▁to e", + "Pre fix", + "Pref ix", + "P refix", + "ho u", + "h ou", + "ga ge", + "g age", + "> \"", + "▁f rag", + "▁fr ag", + "▁fra g", + "▁ frag", + "▁The orem", + "mem ory", + "▁cont ents", + "▁content s", + "▁conten ts", + "▁ contents", + "do cs", + "doc s", + "} '", + "▁Ir ish", + "The n", + "Th en", + "T hen", + "aa ts", + "aat s", + "a ats", + "Sa ve", + "S ave", + "▁a gency", + "▁ag ency", + "▁и ме", + "▁им е", + "до ва", + "дов а", + "▁F unction", + "▁Fun ction", + "▁ Function", + "N N", + "dest roy", + "▁M essage", + "▁Mess age", + "▁ Message", + "▁c ancel", + "▁can cel", + "▁ cancel", + "▁super ior", + "▁e c", + "▁ ec", + "▁liter ature", + "▁P ART", + "▁PA RT", + "▁PAR T", + "▁ PART", + "I l", + "▁C ab", + "▁Ca b", + "eng ine", + "▁b asket", + "▁bas ket", + "wor th", + "wort h", + "w orth", + "▁S el", + "▁Se l", + "f etch", + "▁St adt", + "▁Stad t", + "▁Sta dt", + "▁К и", + "▁con j", + "▁se iner", + "▁sein er", + "▁seine r", + "▁sei ner", + "▁conf irmed", + "▁confirm ed", + "▁Ar gent", + "▁Arg ent", + "am ar", + "ama r", + "a mar", + "pgf path", + "▁strugg le", + "Pat tern", + "▁M iddle", + "it an", + "ita n", + "i tan", + "▁m oon", + "▁mo on", + "or ough", + "oro ugh", + "o rough", + "▁Cath olic", + "▁str uck", + "▁stru ck", + "] ->", + "▁we apon", + "▁weap on", + "▁su bst", + "▁sub st", + "▁subs t", + "▁inst ructions", + "▁instruct ions", + "▁instruction s", + "▁occ as", + "▁oc cas", + "prote cted", + "▁L ess", + "▁Le ss", + "▁Les s", + "▁ Less", + "▁b atch", + "▁bat ch", + "▁ batch", + "▁con tra", + "▁cont ra", + "▁contr a", + "▁de ck", + "▁dec k", + "▁ deck", + "▁ign ored", + "▁ignore d", + "▁ignor ed", + "▁ref used", + "▁refuse d", + "tr igger", + "▁crim inal", + "G A", + "ol ly", + "oll y", + "▁B ell", + "▁Be ll", + "▁Bel l", + "▁ Ю", + "for ward", + "▁p refix", + "▁pre fix", + "▁pref ix", + "▁ prefix", + "▁im mediate", + "▁immedi ate", + "▁as signed", + "▁ass igned", + "▁assign ed", + "▁e lected", + "▁elect ed", + "▁ele cted", + "▁to night", + "▁ton ight", + "▁D ies", + "▁Die s", + "▁Di es", + "▁B each", + "▁Be ach", + "▁pre ced", + "▁prec ed", + "ow ał", + "owa ł", + "▁gal ax", + "▁log ic", + "en za", + "enz a", + "▁Cap tain", + "▁Capt ain", + "▁H ay", + "▁Ha y", + "▁f acts", + "▁fact s", + "▁fac ts", + "▁н и", + "▁ ни", + "t é", + "▁s b", + "▁ sb", + "op ed", + "ope d", + "o ped", + "▁com bat", + "▁comb at", + "▁expl ore", + "▁explo re", + "▁( -", + "▁ (-", + "Load er", + "Lo ader", + "▁Wil son", + "▁l ocked", + "▁loc ked", + "▁lock ed", + "▁ locked", + ": )", + "▁qu el", + "▁que l", + "▁q uel", + "▁ quel", + "▁Г а", + "T y", + "▁tem ps", + "▁temp s", + "▁g host", + "Mat erial", + "M aterial", + "ER CHANT", + "point er", + "po inter", + "ж да", + "ah a", + "a ha", + "ul f", + "▁sup plement", + "▁supp lement", + "▁d ismiss", + "▁dis miss", + "▁cl osing", + "▁clos ing", + "▁clo sing", + "▁vul ner", + "▁ap rès", + "▁apr ès", + "▁over whel", + "ско е", + "▁dis ag", + "ac ia", + "aci a", + "a cia", + "ou red", + "our ed", + "o ured", + "ru ption", + "rupt ion", + "▁P S", + "▁ PS", + "End point", + "Re al", + "▁T ag", + "▁Ta g", + "▁ Tag", + "▁st airs", + "▁sta irs", + "▁stair s", + "▁ stairs", + "ly n", + "l yn", + "▁e leg", + "▁el eg", + "▁ele g", + "▁v eter", + "▁ve ter", + "▁vet er", + "factor y", + "fact ory", + "f actory", + "an ne", + "ann e", + "▁B at", + "▁Ba t", + "▁fr anc", + "▁fra nc", + "lu ng", + "l ung", + "▁\" '", + ".' ,", + ". ',", + "▁C ountry", + "▁Count ry", + "▁Coun try", + "▁ Country", + "^{ [", + "▁y ours", + "▁you rs", + "▁your s", + "▁yo urs", + "ail ability", + "Cl ear", + "C lear", + "ät t", + "ä tt", + "пи с", + "п ис", + "▁j oke", + "▁jo ke", + "▁ann oy", + "▁r ag", + "▁ra g", + "▁ rag", + "var i", + "va ri", + "v ari", + "ле кс", + "лек с", + "▁P sy", + "il ty", + "ilt y", + "mo unt", + "m ount", + "▁c ual", + "▁cu al", + "▁s olar", + "▁so lar", + "▁sol ar", + "}^ {(", + "}^{ (", + "} ^{(", + "Sh ort", + "▁tax es", + "App end", + "Ap pend", + "Appe nd", + "W in", + "est yle", + "esty le", + "e style", + "▁fac il", + "▁fa cil", + "в ро", + "▁s ought", + "▁sou ght", + "▁b are", + "▁bar e", + "▁ba re", + "▁re act", + "▁ react", + "ja r", + "j ar", + "MA C", + "M AC", + "lo v", + "l ov", + "wa rn", + "war n", + "w arn", + "▁cru cial", + "▁m useum", + "ни ц", + "▁K ent", + "▁Ke nt", + "▁Ken t", + "May be", + "▁b ike", + "▁bi ke", + "▁Add ress", + "▁ Address", + "X ML", + "▁ad mitted", + "▁adm itted", + "▁admit ted", + "▁$ (\\", + "▁$( \\", + "▁sp ell", + "▁spe ll", + "▁spel l", + "▁ spell", + "▁v ic", + "▁vi c", + "gr e", + "g re", + "▁p roc", + "▁pro c", + "▁pr oc", + "▁ proc", + "th eless", + "the less", + "▁N om", + "▁No m", + "▁R ail", + "▁Ra il", + "▁acc eler", + "▁con vin", + "▁conv in", + "▁Pro perty", + "▁ Property", + "▁D A", + "▁ DA", + "▁cl ip", + "▁ clip", + "▁pl ugin", + "▁plug in", + "▁ plugin", + "Lim it", + "Li mit", + "L imit", + "view s", + "br u", + "b ru", + "▁p ra", + "▁pr a", + "▁a k", + "▁ ak", + "▁e j", + "▁ ej", + "▁o pts", + "▁op ts", + "▁opt s", + "▁ opts", + "▁sl ip", + "▁g ang", + "▁gan g", + "▁ga ng", + "▁ gang", + "as ted", + "ast ed", + "aste d", + "a sted", + "ual s", + "ua ls", + "u als", + "▁d ying", + "▁dy ing", + "Col l", + "Co ll", + "C oll", + "am men", + "amm en", + "▁Pol icy", + "▁ Policy", + "ERCHANT ABILITY", + "▁Col lection", + "▁Coll ection", + "▁Collect ion", + "▁ Collection", + "▁v ec", + "▁ve c", + "▁ vec", + "▁D ick", + "▁Di ck", + "st ud", + "▁la yers", + "▁lay ers", + "▁layer s", + "▁ layers", + "▁t ied", + "▁tie d", + "▁ti ed", + "}\\ \\", + "} \\\\", + "▁al ors", + "▁j ou", + "▁jo u", + "▁ch icken", + "▁chi cken", + "▁chick en", + "▁perman ent", + "▁Every thing", + "▁L ow", + "▁Lo w", + "▁ Low", + "▁C ook", + "▁Co ok", + "▁pe ak", + "▁PARTIC ULAR", + "▁d ear", + "▁de ar", + "i č", + "▁introdu ce", + "▁caus ing", + "▁ca using", + "пи са", + "пис а", + "Bo und", + "B ound", + "hu nd", + "h und", + "mult i", + "mul ti", + "▁p are", + "▁par e", + "▁pa re", + "▁ pare", + "an nt", + "ann t", + "▁b reat", + "▁bre at", + "▁commit ment", + "▁increasing ly", + "ко й", + "▁F riend", + "▁ Friend", + "▁stat istics", + "▁statist ics", + "▁Man ager", + "▁ Manager", + "pl icate", + "plic ate", + "plica te", + "Cl oud", + "ac i", + "a ci", + "▁Con ference", + "Sp an", + "S pan", + "▁C EO", + "▁CE O", + "▁W ait", + "▁Wa it", + "▁ Wait", + "▁O ber", + "▁Ob er", + "if ting", + "ift ing", + "im iento", + "imi ento", + "get Element", + "▁g le", + "▁gl e", + "▁ gle", + "ли я", + "▁w ieder", + "▁wie der", + "▁inst ruction", + "▁instr uction", + "▁instruct ion", + "gl y", + "g ly", + "▁bl ame", + "▁list ade", + "▁lista de", + "▁a apt", + "▁Lew is", + "Fr agment", + "▁g ear", + "▁ge ar", + "mi ll", + "mil l", + "m ill", + "pro d", + "pr od", + "p rod", + "▁bur ning", + "▁burn ing", + "є ться", + "▁m é", + "▁ mé", + "è ne", + "▁com plicated", + "▁compl icated", + "▁complic ated", + "b h", + "▁Just ice", + "▁t ested", + "▁te sted", + "▁test ed", + "▁st aring", + "▁star ing", + "▁sta ring", + "▁surv ive", + "▁surviv e", + "▁c ous", + "▁co us", + "▁cou s", + "▁r ib", + "▁ri b", + "▁ rib", + "am l", + "a ml", + "▁T rust", + "▁Tr ust", + "▁Tru st", + "▁c ad", + "▁ca d", + "▁T err", + "▁Te rr", + "▁Ter r", + "▁m apping", + "▁map ping", + "▁ma pping", + "▁ mapping", + "▁tw elve", + "▁g rant", + "▁gr ant", + "▁gran t", + "▁gra nt", + "▁th orough", + "▁ Ü", + "▁fol ks", + "▁folk s", + "▁Cont ent", + "▁ Content", + "▁child hood", + "ck er", + "cke r", + "c ker", + "с но", + "RE CT", + "REC T", + "R ECT", + "▁f inale", + "▁fin ale", + "▁final e", + "▁sh ower", + "▁show er", + "ér ic", + "éri c", + "é ric", + "▁s pat", + "▁sp at", + "od ge", + "р ь", + "▁p es", + "▁pe s", + "▁ pes", + "ed a", + "e da", + "D b", + "▁Ant onio", + "▁Anton io", + "▁eng aged", + "▁engage d", + "▁v ess", + "▁ve ss", + "val s", + "va ls", + "v als", + "▁elect ronic", + "▁electron ic", + "▁electro nic", + "le mma", + "lem ma", + "▁W y", + "ma d", + "m ad", + "mer ge", + "ap on", + "a pon", + "▁priv ile", + "▁nov embre", + "▁nove mbre", + "▁S ports", + "▁Sp orts", + "▁Sport s", + "wi ll", + "w ill", + "▁control s", + "▁contr ols", + "▁contro ls", + "▁ controls", + "▁c ategories", + "▁categ ories", + "▁categor ies", + "▁ categories", + "▁Georg ia", + "ip edia", + "▁A V", + "▁ AV", + "at ori", + "ator i", + "ato ri", + "▁_ __", + "▁__ _", + "▁ ___", + "▁ À", + "▁R yan", + "▁Ry an", + "▁Char lie", + "▁Charl ie", + "▁и сто", + "▁ис то", + "▁em otion", + "▁emot ion", + "▁co oking", + "▁cook ing", + "▁attempt s", + "▁FIT NESS", + "ät er", + "ä ter", + "En able", + "D T", + "▁Ch ange", + "▁ Change", + "Asp Net", + "▁г а", + "▁ га", + "▁ord inary", + "▁ordin ary", + "▁S QL", + "▁ SQL", + "pl ane", + "plan e", + "p lane", + "% .", + "▁Sum mer", + "▁av ait", + "up p", + "u pp", + "▁ill ness", + "UI NT", + "U INT", + "> {", + "▁zw ischen", + "▁hard ware", + "▁sound ed", + "equ iv", + "▁p iano", + "▁pi ano", + "▁pian o", + "us et", + "use t", + "u set", + "k n", + "TR Y", + "▁b ab", + "▁ba b", + "не н", + "н ен", + "▁rel iable", + "▁reli able", + "▁Bron nen", + "▁St ore", + "▁Sto re", + "▁ Store", + "A z", + "▁» ,", + "▁ »,", + "St atic", + "Stat ic", + "d w", + "gr een", + "gre en", + "g reen", + "▁' ';", + "▁'' ;", + "li j", + "l ij", + "ev a", + "e va", + "ні й", + "▁S yd", + "▁Sy d", + "in ois", + "ino is", + "con vert", + "conv ert", + "▁decl are", + "▁declar e", + "br es", + "bre s", + "b res", + "IN K", + "it led", + "itle d", + "▁acc ord", + "▁ac cord", + "▁m ars", + "▁mar s", + "▁ma rs", + "Sequ ence", + "zi p", + "z ip", + "▁Braz il", + "▁meet ings", + "▁meeting s", + "▁accur acy", + "▁M achine", + "▁Mach ine", + "▁ Machine", + "▁aut or", + "▁au tor", + "▁auto r", + "▁ autor", + "▁a insi", + "▁ain si", + "Sim ple", + "Res ources", + "Re sources", + "Resource s", + "ка за", + "каз а", + "▁M P", + "▁ MP", + "th ey", + "the y", + "▁B ang", + "▁Ba ng", + "▁Ban g", + "▁e ing", + "▁ein g", + "▁ eing", + "ate ful", + "▁Some thing", + "▁Som ething", + "▁ Something", + "▁up set", + "Hist ory", + "Hi story", + "dim ensional", + "▁explan ation", + "▁c iv", + "▁ci v", + "▁c once", + "▁con ce", + "▁conc e", + "▁kö z", + "▁prom ised", + "▁promise d", + "ж ду", + "we d", + "w ed", + "For e", + "F ore", + "Am ount", + "A mount", + "ab b", + "a bb", + "▁cl othing", + "▁cloth ing", + "▁clo thing", + "ли сь", + "oe n", + "o en", + "▁Pr int", + "▁Pri nt", + "▁Prin t", + "▁ Print", + "▁s izes", + "▁size s", + "▁si zes", + "▁b anks", + "▁bank s", + "▁ban ks", + "ri bed", + "rib ed", + "ribe d", + "▁' ../", + "▁'. ./", + "FI X", + "F IX", + "▁H ug", + "▁Hu g", + "▁z n", + "▁ zn", + "▁I NT", + "▁IN T", + "▁ INT", + "▁in stances", + "▁inst ances", + "▁instance s", + "▁along side", + "Name space", + "Names pace", + "▁re new", + "▁ren ew", + "▁a sc", + "▁as c", + "▁ asc", + "▁w aves", + "▁wa ves", + "▁wave s", + "▁p om", + "▁po m", + "D uration", + "day s", + "da ys", + "d ays", + "$ (", + "▁grab bed", + "▁sur gery", + "▁surge ry", + "▁surg ery", + "▁re store", + "▁rest ore", + "▁ restore", + "Norm al", + "N ormal", + "▁L eb", + "▁Le b", + "▁anal yt", + "▁analy t", + "Lite ral", + "L iteral", + "H A", + "▁sh ares", + "▁share s", + "▁shar es", + "▁sha res", + "il let", + "ill et", + "ille t", + "ol s", + "o ls", + "▁D og", + "▁Do g", + "or no", + "orn o", + "▁man ip", + "ja v", + "j av", + "▁ess entially", + "▁essential ly", + "▁cas ual", + "op l", + "o pl", + "▁ р", + "▁S U", + "▁ SU", + "▁engine ering", + "▁engineer ing", + "▁Pr ime", + "▁Pri me", + "▁Prim e", + "▁S W", + "▁ SW", + "▁re aching", + "▁reach ing", + "▁в ла", + "▁Ро сси", + "▁K re", + "▁Kr e", + "er ry", + "err y", + "▁op pon", + "▁opp on", + "pro gram", + "pr ogram", + "em per", + "emp er", + "is Empty", + "▁U nit", + "▁Un it", + "▁ Unit", + "IN TER", + "INT ER", + "INTE R", + "et he", + "eth e", + "e the", + "z d", + "CU R", + "C UR", + "▁v m", + "▁ vm", + "con v", + "co nv", + "ro pol", + "rop ol", + "r opol", + "▁Co ast", + "▁S elect", + "▁Se lect", + "▁Sel ect", + "▁ Select", + "▁бы ла", + "▁был а", + "▁V e", + "ow y", + "o wy", + "▁my th", + "ce ptions", + "ception s", + "cept ions", + "class es", + "▁w orden", + "▁wor den", + "▁word en", + "▁ass ault", + "▁d ual", + "▁du al", + "OR K", + "▁in ches", + "▁inc hes", + "▁inch es", + "▁F A", + "▁ FA", + "▁St ation", + "▁Stat ion", + "▁ Station", + "▁person ality", + "▁personal ity", + "▁s car", + "▁sc ar", + "▁ scar", + "▁reg ime", + "▁not en", + "▁no ten", + "▁note n", + "▁r ural", + "▁ru ral", + "iz a", + "i za", + "Aud io", + "A udio", + "▁dis put", + "▁disp ut", + "▁a ver", + "▁av er", + "▁ave r", + "▁ aver", + "▁o bst", + "▁ob st", + "▁obs t", + "▁Reg ion", + "▁ Region", + "ut f", + "u tf", + "▁C ass", + "▁Cas s", + "▁Ca ss", + "hs pace", + "h space", + "▁sh ipping", + "▁ship ping", + "ik o", + "i ko", + "ic ked", + "ick ed", + "num er", + "nu mer", + "n umer", + "д на", + "ri el", + "rie l", + "r iel", + "dis abled", + "disable d", + "op ol", + "o pol", + "lo oking", + "look ing", + "▁class ical", + "▁classic al", + "▁construct ed", + "▁constru cted", + "▁refer enties", + "] +", + "▁capt ured", + "▁capture d", + "▁min imal", + "▁minim al", + "▁mini mal", + "▁s ock", + "▁so ck", + "▁soc k", + "▁ sock", + "fa ther", + "f ather", + "is ión", + "isi ón", + "▁equ ally", + "▁equal ly", + "▁eq ually", + "▁red uction", + "▁redu ction", + "An t", + "A nt", + "ais on", + "ai son", + "a ison", + "▁ar gue", + "▁arg ue", + "cir cle", + "circ le", + "▁t oler", + "▁to ler", + "}\" ,", + "} \",", + "▁prim arily", + "us al", + "usa l", + "u sal", + "▁al gebra", + "▁gather ed", + "▁Re member", + "▁Rem ember", + "_) ;", + "_ );", + "UT E", + "U TE", + "▁K it", + "▁Ki t", + "▁ Kit", + "S y", + "HE AD", + "▁re cipe", + "▁rec ipe", + "▁recip e", + "▁sc enario", + "▁scen ario", + "▁Follow ing", + "VA R", + "V AR", + "▁y ard", + "▁ya rd", + "▁ yard", + "▁st ad", + "▁sta d", + "▁ stad", + "* (", + "▁valid ate", + "▁ validate", + "DE X", + "D EX", + "▁commit tee", + "▁t emporary", + "▁tempor ary", + "▁consequ ences", + "▁consequence s", + "▁égal ement", + "кти в", + "к тив", + "▁r a", + "▁ ra", + "▁dis pl", + "▁di spl", + "▁disp l", + "▁app s", + "▁ap ps", + "▁ apps", + "▁Te il", + "▁» .", + "▁ ».", + "▁adopt ed", + "ten sor", + "t ensor", + "▁fe min", + "▁fem in", + "▁м ар", + "▁ма р", + "ло ги", + "te ch", + "t ech", + "▁R ot", + "▁Ro t", + "▁ Rot", + "▁kn ees", + "▁kne es", + "▁knee s", + "ph ys", + "phy s", + "ow ej", + "owe j", + "▁Ox ford", + "ан д", + "а нд", + "he ll", + "hel l", + "h ell", + "ograf ia", + "▁ex posed", + "▁exp osed", + "▁expos ed", + "▁expose d", + "kt op", + "k top", + "ob y", + "o by", + "lo wer", + "low er", + "l ower", + "▁Se nate", + "▁Sen ate", + "▁s word", + "▁sw ord", + "▁swo rd", + "Fl ow", + "F low", + "▁Un fortunately", + "▁box es", + "▁ boxes", + "▁cu ando", + "▁pi lot", + "▁pil ot", + "▁Al bum", + "▁Alb um", + "B al", + "So rt", + "S ort", + "FI ELD", + "▁de sert", + "▁des ert", + "CO MM", + "COM M", + "ro ns", + "ron s", + "r ons", + "ad ows", + "ado ws", + "adow s", + "▁l oyal", + "▁lo yal", + "▁as set", + "▁ass et", + "▁ asset", + "▁m ud", + "▁mu d", + "ф а", + "▁second ary", + "▁ secondary", + "▁А р", + "▁c ul", + "▁cu l", + "▁ cul", + "▁As ian", + "▁Asia n", + "▁stay ing", + "▁sta ying", + "▁data set", + "▁dat aset", + "▁ dataset", + "▁U SE", + "▁US E", + "▁ USE", + "▁l oves", + "▁lo ves", + "▁love s", + "▁lov es", + "▁vel ocity", + "▁veloc ity", + "á v", + "▁purch ased", + "▁purchase d", + "SO C", + "S OC", + "▁compet itive", + "▁Foot ball", + "is ka", + "isk a", + "i ska", + "▁kn ock", + "st airs", + "sta irs", + "az y", + "a zy", + "▁v end", + "▁ve nd", + "▁ven d", + "▁ar ts", + "▁art s", + "▁ arts", + "▁B ras", + "▁Br as", + "▁Bra s", + "ue la", + "uel a", + "u ela", + "кт о", + "к то", + "tr im", + "tri m", + "t rim", + "▁d irty", + "▁dir ty", + "▁dirt y", + "▁ dirty", + "▁webs ites", + "▁website s", + "▁In dep", + "▁Ind ep", + "▁с тра", + "▁ст ра", + "▁ стра", + "s r", + "▁t icket", + "▁tick et", + "at ile", + "ati le", + "a tile", + "▁implement ed", + "▁вре мя", + "▁bo wl", + "▁bow l", + "DA TE", + "DAT E", + "D ATE", + "▁al ter", + "▁alt er", + "▁ alter", + "▁S pace", + "▁Sp ace", + "▁ Space", + "▁accom pan", + "▁accomp an", + "or don", + "ord on", + "▁do ctors", + "▁doctor s", + "ist as", + "ista s", + "C ast", + "до м", + "CT L", + "C TL", + "ur ers", + "ure rs", + "urer s", + "▁ingred ients", + "▁calcul ated", + "▁calculate d", + "▁calc ulated", + "▁le ather", + "▁s ensitive", + "▁sens itive", + "▁sus pic", + "▁susp ic", + "st an", + "sta n", + "s tan", + "▁an ni", + "▁ann i", + "▁ anni", + "aw ait", + "awa it", + "a wait", + "▁Fr anç", + "▁Fran ç", + "▁ab ort", + "▁ abort", + "▁Sp irit", + "▁W alter", + "▁Wal ter", + "▁Walt er", + "un kt", + "unk t", + "▁vert ical", + "▁ vertical", + "OR S", + "O RS", + "be st", + "bes t", + "b est", + "▁Cl ient", + "▁ Client", + "it ated", + "ita ted", + "itate d", + "itat ed", + "▁в а", + "▁ ва", + "▁ Č", + "▁v ille", + "▁vi lle", + "▁vill e", + "▁vil le", + "▁ ville", + "▁di plom", + "or ne", + "orn e", + "▁b ars", + "▁bar s", + "▁ba rs", + "▁ bars", + "U ri", + "AP TER", + "pon s", + "po ns", + "p ons", + "ut z", + "u tz", + "Pro to", + "Pr oto", + "▁st ir", + "▁ц е", + "▁ це", + "▁pr imer", + "▁prim er", + "▁pri mer", + "▁prime r", + "ig ible", + "igi ble", + "ex tra", + "ext ra", + "extr a", + "▁Bo oks", + "▁Book s", + "▁B os", + "▁Bo s", + "▁E t", + "▁W elt", + "▁We lt", + "▁Wel t", + "▁Kore a", + "▁Ko rea", + "▁Kor ea", + "ри то", + "р ито", + "▁v ibr", + "▁vi br", + "S elf", + "line ar", + "lin ear", + "о б", + "▁L ang", + "▁La ng", + "▁Lan g", + "▁ Lang", + "▁de eper", + "▁deep er", + "▁term in", + "▁ter min", + "▁ termin", + "en schaft", + "ens chaft", + "ensch aft", + "▁ро ці", + "am med", + "amm ed", + "vis ible", + "v isible", + "▁IO Exception", + "▁ IOException", + "▁W ind", + "▁Win d", + "▁Wi nd", + "us qu", + "▁S top", + "▁St op", + "▁Sto p", + "▁ Stop", + "▁ор га", + "IN VALID", + "INVAL ID", + "▁c ub", + "▁cu b", + "▁j ew", + "▁je w", + "▁cap tain", + "▁capt ain", + "з і", + "ch unk", + "apt ure", + "ash board", + "▁div ided", + "▁divid ed", + "▁divide d", + "▁ext ensive", + "▁extens ive", + "▁s uffer", + "▁suff er", + "▁he ading", + "▁head ing", + "▁ heading", + "cre ated", + "create d", + "creat ed", + "c reated", + "▁quiet ly", + "▁n y", + "▁ ny", + "▁по л", + "▁ пол", + "\" +", + "ik an", + "ika n", + "i kan", + "▁design s", + "z u", + "}+ \\", + "} +\\", + "Oper ator", + "▁Le mma", + "▁Lem ma", + "▁на у", + "ac ji", + "ло ве", + "лов е", + "Serv let", + "▁K evin", + "▁Ke vin", + "st age", + "sta ge", + "b n", + "text width", + "fa iled", + "fail ed", + "f ailed", + "▁St aff", + "▁Sta ff", + "▁e nem", + "▁en em", + "un de", + "und e", + "u nde", + "ен ь", + "е нь", + "Pack et", + "P acket", + "▁A ls", + "▁Al s", + "ka r", + "k ar", + "][ '", + "] ['", + "ke d", + "k ed", + "Per s", + "Pe rs", + "P ers", + ">: :", + "> ::", + "▁a rc", + "▁ar c", + "▁ arc", + "▁sy nt", + "▁syn t", + "SP E", + "S PE", + "▁Д а", + "▁M i", + "▁M oh", + "▁Mo h", + "▁De ath", + "b rowser", + "▁D ave", + "▁Dav e", + "▁Da ve", + "▁s ucc", + "▁su cc", + "▁suc c", + "t oggle", + "▁t ack", + "▁ta ck", + "Com ment", + "Comm ent", + "er on", + "ero n", + "e ron", + "▁aware ness", + "▁h ug", + "▁cont emporary", + "▁contempor ary", + "ul ating", + "ula ting", + "▁T itle", + "▁Tit le", + "▁Ti tle", + "▁ Title", + "▁TH IS", + "hav ior", + "ran k", + "r ank", + "▁do zen", + "▁che ese", + "co ln", + "col n", + "▁rad ius", + "▁radi us", + "▁ radius", + "▁dim ensions", + "▁dimension s", + "rodu ction", + "rod uction", + "▁ad ds", + "▁add s", + "▁house hold", + "▁D avis", + "▁Dav is", + "▁Da vis", + "pk g", + "p kg", + "{ $", + "▁cas ino", + "▁P ierre", + "▁Pier re", + "▁Pi erre", + "▁object ive", + "tr ain", + "tra in", + "▁Mich igan", + "pay load", + "▁r ug", + "▁ru g", + "▁ rug", + "▁se vere", + "▁sever e", + "me an", + "▁t oss", + "▁to ss", + "▁embar rass", + "▁V ery", + "▁Ver y", + "▁Ve ry", + "▁ Very", + "▁appe al", + "▁Com put", + "▁Comp ut", + "▁ Comput", + "▁forgot ten", + "▁k ernel", + "▁ker nel", + "▁ kernel", + "▁car bon", + "▁carb on", + "f w", + "▁С у", + "▁Emp ire", + "▁qu ote", + "▁quot e", + "▁ quote", + "et z", + "e tz", + "▁m ini", + "▁min i", + "▁mi ni", + "▁p ipe", + "▁pi pe", + "▁pip e", + "▁ pipe", + "▁n ous", + "▁no us", + "▁nou s", + "▁M ove", + "▁Mo ve", + "▁Mov e", + "▁ Move", + "▁д у", + "▁ ду", + "▁nerv ous", + "▁М ар", + "▁Ма р", + "* \r", + "▁B ush", + "▁Bus h", + "▁Bu sh", + "▁pe er", + "▁ peer", + "▁W rit", + "▁Wr it", + "▁ Writ", + "▁satisf ied", + "▁pull ing", + "▁pul ling", + "▁P ur", + "▁Pu r", + "▁M iller", + "▁Mil ler", + "▁Mill er", + "▁F L", + "▁ FL", + "am az", + "ama z", + "a maz", + "▁m ile", + "▁mil e", + "▁mi le", + "▁ mile", + "▁N eed", + "▁Ne ed", + "▁ Need", + "▁sup plies", + "▁a ño", + "▁p ace", + "▁pa ce", + "▁pac e", + "▁ pace", + "▁Vict oria", + "▁Victor ia", + "▁ou ght", + "▁ ought", + "▁P layer", + "▁Pl ayer", + "▁Play er", + "▁ Player", + "agnost ic", + "▁v iv", + "▁vi v", + "▁ viv", + "▁Pat rick", + "▁Patri ck", + "▁ Š", + "▁St ory", + "▁Sto ry", + "ac a", + "a ca", + "▁mount ains", + "▁mountain s", + "CL ASS", + "▁fr agment", + "▁frag ment", + "▁ fragment", + "▁sett lement", + "▁settle ment", + "▁Further more", + "▁dr ivers", + "▁dri vers", + "▁driv ers", + "▁drive rs", + "▁driver s", + "▁J u", + "▁бы ли", + "▁был и", + "Row s", + "Ro ws", + "R ows", + "▁im pression", + "▁imp ression", + "▁impress ion", + "▁in fer", + "▁inf er", + "▁Ex pl", + "▁Exp l", + "ol ute", + "olut e", + "olu te", + "ov an", + "ova n", + "o van", + "ar ance", + "aran ce", + "CA P", + "C AP", + "▁en force", + "▁B urn", + "▁Bur n", + "▁Bu rn", + "Res et", + "Re set", + "mo ther", + "mot her", + "m other", + "▁B attle", + "▁Bat tle", + "▁Batt le", + "▁ Battle", + "pad ding", + "p adding", + "ia te", + "iat e", + "i ate", + "▁c ried", + "▁cr ied", + "▁cri ed", + "A K", + "un s", + "u ns", + "▁siè cle", + "▁Cont in", + "▁ Contin", + "ban k", + "b ank", + "ju nit", + "j unit", + "object s", + "Ro t", + "R ot", + "is sa", + "iss a", + "▁be gun", + "▁beg un", + "* -", + "▁vis iting", + "▁visit ing", + "ж де", + "▁target s", + "▁tar gets", + "▁L atin", + "▁Lat in", + "у т", + "▁E sc", + "▁Es c", + "* ;", + "ån g", + "å ng", + "▁( {", + "▁ ({", + "▁di agram", + "▁dia gram", + "Mod els", + "Model s", + "Mode ls", + "▁part nership", + "▁partner ship", + "▁partners hip", + "▁fr ån", + "ul ty", + "ult y", + "Po d", + "P od", + "CA LL", + "CAL L", + "C ALL", + "mod al", + "mo dal", + "si g", + "s ig", + "it zer", + "itz er", + "it el", + "ite l", + "▁convin ced", + "▁convince d", + "ab l", + "a bl", + "ст ве", + "ств е", + "▁c ot", + "▁co t", + "▁re peat", + "▁repe at", + "▁ repeat", + "▁l ists", + "▁li sts", + "▁list s", + "▁ lists", + "so und", + "s ound", + "▁r oyal", + "▁ro yal", + "▁gr ace", + "▁gra ce", + "▁o raz", + "▁or az", + "Not ification", + "pr ite", + "prit e", + "p rite", + "▁arriv al", + "▁arr ival", + "an cell", + "ance ll", + "anc ell", + "ancel l", + "hent ic", + "de code", + "dec ode", + "▁fant astic", + "pro gress", + "pro xy", + "pr oxy", + "z ő", + "ke l", + "k el", + "▁conven ient", + "aqu e", + "a que", + "ri et", + "rie t", + "r iet", + "▁Dig ital", + "io rs", + "ior s", + "i ors", + "▁B udd", + "▁Bud d", + "▁Bu dd", + "and ra", + "ad dy", + "add y", + "▁o vers", + "▁over s", + "▁ov ers", + "▁consum ers", + "▁consumer s", + "▁consume rs", + "p n", + "mo use", + "m ouse", + "▁B C", + "▁ BC", + "de g", + "d eg", + "pe rm", + "per m", + "p erm", + "it és", + "ité s", + "▁и спо", + "▁ис по", + "he ast", + "h east", + "ho ur", + "hou r", + "h our", + "PAR AM", + "con scious", + "▁w ing", + "▁win g", + "▁ wing", + "▁atmos phere", + "▁g ig", + "▁gi g", + "▁con tre", + "▁cont re", + "▁contr e", + "▁dr ama", + "▁dram a", + "я т", + "▁Fr ont", + "▁Fro nt", + "▁ Front", + "▁philosoph y", + "▁H art", + "▁Har t", + "▁Ha rt", + "▁n urs", + "▁nu rs", + "▁nur s", + "ur as", + "ura s", + "u ras", + "▁T ru", + "▁Tr u", + "▁s ud", + "▁su d", + "▁per forming", + "▁perform ing", + "п ы", + "▁conf used", + "▁che cks", + "▁check s", + "am t", + "a mt", + "Ma ke", + "M ake", + "▁R O", + "▁ RO", + "▁d f", + "▁ df", + "iz ations", + "ization s", + "▁deg li", + "▁architect ure", + "Render er", + "▁Л а", + "▁p tr", + "▁pt r", + "▁ ptr", + "▁die ser", + "▁dies er", + "▁diese r", + "sub mit", + "▁top ics", + "▁topic s", + "▁princip les", + "▁prin ciples", + "▁principle s", + "var s", + "va rs", + "v ars", + "so ck", + "soc k", + "s ock", + "▁ton gue", + "▁tong ue", + "▁percent age", + "▁S S", + "▁ SS", + "▁d ol", + "▁do l", + "▁r ice", + "▁ri ce", + "▁ric e", + "▁ rice", + "í o", + "▁E astern", + "▁East ern", + "▁Easter n", + "▁recogn ition", + "▁E rn", + "▁Er n", + "▁U t", + "▁ Ut", + "▁c aut", + "▁ca ut", + "▁Cl oud", + "▁ Cloud", + "▁con version", + "▁conv ersion", + "▁convers ion", + "▁Oh io", + "▁M E", + "▁ ME", + "▁sur ely", + "▁sure ly", + "▁g ard", + "▁gar d", + "▁ga rd", + "pu is", + "p uis", + "▁u rg", + "▁ur g", + "▁ urg", + "im i", + "i mi", + "▁abs ence", + "▁w inner", + "▁win ner", + "L anguage", + "▁HT TP", + "▁ HTTP", + "w t", + "▁trans lation", + "▁transl ation", + "▁ translation", + "с с", + "▁K ind", + "▁Ki nd", + "▁Kin d", + "▁ Kind", + "Tw o", + "T wo", + "▁Re volution", + "▁Rev olution", + "In sert", + "Ins ert", + "Ev ery", + "E very", + "or ient", + "ori ent", + "orie nt", + "o rient", + "▁т ра", + "▁ тра", + "▁emot ions", + "▁emotion s", + "det ails", + "detail s", + "▁f lu", + "▁fl u", + "▁ flu", + "▁oper ate", + "▁opera te", + "A g", + "un ning", + "unn ing", + "▁part ie", + "▁parti e", + "tr i", + "t ri", + "▁gold en", + "▁gol den", + "▁Б и", + "▁found ation", + "is ten", + "ist en", + "iste n", + "i sten", + "▁Car los", + "▁Carl os", + "▁Carlo s", + "Child ren", + "▁neigh b", + "▁C art", + "▁Car t", + "▁Ca rt", + "▁ Cart", + "Be gin", + "B egin", + "г да", + "▁s cheduled", + "▁schedule d", + "▁schedul ed", + "' >", + "▁observ ations", + "▁observation s", + "▁produ cer", + "▁produce r", + "ath ers", + "ather s", + "a thers", + "но му", + "ном у", + "▁expect ations", + "▁expectation s", + "os o", + "o so", + "z h", + "mu table", + "mut able", + "▁wr ites", + "▁writ es", + "▁write s", + "▁p ushing", + "▁push ing", + "▁se ats", + "▁sea ts", + "▁seat s", + "▁br east", + "▁bre ast", + "ap ing", + "api ng", + "a ping", + "▁Sim ple", + "▁ Simple", + "▁s ocket", + "▁soc ket", + "▁sock et", + "▁ socket", + "▁sl ave", + "▁sla ve", + "▁ slave", + "il ey", + "ile y", + "i ley", + "▁ass istant", + "▁assist ant", + "▁t rim", + "▁tr im", + "▁tri m", + "▁ trim", + "▁land scape", + "▁landsc ape", + "▁associ ation", + "qu ant", + "▁Pal est", + "▁swe at", + "en gers", + "eng ers", + "enge rs", + "enger s", + "? _", + "é p", + "> .", + "▁c urious", + "▁cur ious", + "▁Com ponent", + "▁ Component", + "▁re placement", + "▁repl acement", + "▁replace ment", + "ра ль", + "рал ь", + "р аль", + "▁Tr ack", + "▁Tra ck", + "▁ Track", + "▁Re move", + "▁Rem ove", + "▁ Remove", + "▁S ize", + "▁Si ze", + "▁ Size", + "pe ror", + "per or", + "▁cal culate", + "▁calcul ate", + "▁calc ulate", + "▁s essions", + "▁session s", + "▁type d", + "▁typ ed", + "▁ty ped", + "▁sub mit", + "▁subm it", + "▁ submit", + "!! !", + "! !!", + "▁part ition", + "▁ partition", + "ed ing", + "edi ng", + "e ding", + "-- ---", + "---- -", + "--- --", + "- ----", + "az ioni", + "azi oni", + "lie ß", + "on al", + "ona l", + "o nal", + "▁sh ru", + "▁shr u", + "▁RE G", + "▁ REG", + "▁F ac", + "▁Fa c", + "▁ Fac", + "config uration", + "▁бы ло", + "▁был о", + "▁A mong", + "▁Am ong", + "__ );", + "__) ;", + "_ _);", + "▁Ser ver", + "▁Serv er", + "▁ Server", + "▁L OG", + "▁LO G", + "▁ LOG", + "▁c and", + "▁can d", + "▁ca nd", + "'] );", + "']) ;", + "' ]);", + "go v", + "g ov", + "▁S ix", + "▁Si x", + "un defined", + "und efined", + "undef ined", + "▁t y", + "▁ ty", + "as a", + "a sa", + "▁part icles", + "▁partic les", + "▁particle s", + "▁parti cles", + "▁ф ор", + "▁фо р", + "▁ фор", + "` `", + "T ube", + "el and", + "ela nd", + "e land", + "fo ld", + "fol d", + "f old", + "og o", + "o go", + "▁appro aches", + "▁approach es", + "on da", + "ond a", + "ag r", + "a gr", + ", $", + "▁{ {", + "▁ {{", + "▁Mod ern", + "▁Mo dern", + "▁Mode rn", + "▁W inter", + "▁Win ter", + "av ailable", + "▁L ud", + "▁Lu d", + "▁c asa", + "▁cas a", + "▁ca sa", + "▁C ould", + "▁Co uld", + "▁Cou ld", + "▁ Could", + "▁fif teen", + "▁pot entially", + "▁potential ly", + "^ ^", + "▁se it", + "▁sei t", + "An imation", + "Anim ation", + "ко го", + "к ого", + "Z one", + "el if", + "eli f", + "e lif", + "▁acknow led", + "▁own ership", + "▁owner ship", + "▁owners hip", + "▁describ es", + "▁describe s", + "▁re verse", + "▁revers e", + "▁rever se", + "▁ reverse", + "▁con test", + "▁cont est", + "▁sc ored", + "▁score d", + "▁op posed", + "▁opp osed", + "▁oppos ed", + "fl ex", + "f lex", + "kr e", + "k re", + "▁mer ge", + "▁ merge", + "▁cover ing", + "▁cov ering", + "▁hon estly", + "▁honest ly", + "▁M ess", + "▁Me ss", + "▁Mes s", + "▁r arely", + "▁rare ly", + "▁incred ible", + "it age", + "ita ge", + "▁vict ims", + "▁victim s", + "ны ми", + "ным и", + "w l", + "iz za", + "izz a", + "i zza", + "d n", + "on de", + "ond e", + "o nde", + "▁pr zy", + "▁prz y", + "▁HT ML", + "▁ HTML", + "▁pay load", + "▁ payload", + "Bu s", + "B us", + "us b", + "u sb", + "F n", + "▁display ed", + "▁o cean", + "▁A venue", + "▁Av enue", + "ac ion", + "aci on", + "acio n", + "gh an", + "g han", + "met ric", + "m etric", + "ie ties", + "iet ies", + "▁attract ive", + "▁attr active", + "▁f ö", + "▁ fö", + "C reat", + "ver ter", + "vert er", + "▁Al ice", + "▁Ali ce", + "по л", + "▁f raction", + "▁fr action", + "▁fra ction", + "▁fract ion", + "▁behav iour", + "▁behavi our", + "▁Jer sey", + "▁re venue", + "▁rev enue", + "▁reven ue", + "▁t res", + "▁tr es", + "▁tre s", + "▁ tres", + "IL D", + "I LD", + "▁É t", + "▁s ync", + "▁sy nc", + "▁syn c", + "▁ sync", + "wi ch", + "w ich", + "▁anc est", + "ъ т", + "om o", + "o mo", + "▁I de", + "▁Id e", + "▁g ained", + "▁gain ed", + "▁ga ined", + "▁moment um", + "▁K o", + "ie u", + "i eu", + "ie lt", + "iel t", + "i elt", + "▁bon us", + "▁te xture", + "▁text ure", + "▁ texture", + "Mod al", + "Mo dal", + "NE XT", + "N EXT", + "▁годи не", + "▁l anguages", + "▁language s", + "v t", + "▁represent ing", + "▁D ream", + "▁Dre am", + "cur r", + "cu rr", + "qu al", + "q ual", + "▁j s", + "▁ js", + "bu rn", + "bur n", + "b urn", + "▁contribut ions", + "▁contribution s", + "▁r ic", + "▁ri c", + "▁ ric", + "}- \\", + "} -\\", + "={ {", + "= {{", + "ca rt", + "car t", + "c art", + "F B", + "ju d", + "j ud", + "es p", + "e sp", + "▁elect ron", + "▁electro n", + "▁e ll", + "▁el l", + "▁ ell", + "▁Run time", + "▁ Runtime", + "ac hel", + "ach el", + "ache l", + "a chel", + "\\ _", + "we ek", + "pack et", + "p acket", + "▁Secret ary", + "▁Jahr hund", + "▁th reshold", + "▁ threshold", + "ba ge", + "bag e", + "b age", + "▁con cer", + "▁conc er", + "▁conce r", + "▁b one", + "▁bo ne", + "▁bon e", + "▁ bone", + "▁Holly wood", + "Cur sor", + "C ursor", + "▁aw arded", + "▁award ed", + "▁sum mary", + "▁summar y", + "▁ summary", + "ag gio", + "agg io", + "aggi o", + "▁st ell", + "▁ste ll", + "▁ stell", + "▁f lesh", + "▁fl esh", + "▁fle sh", + "P air", + "▁A ge", + "▁Ag e", + "ing ton", + "▁' .", + "▁ '.", + "as er", + "ase r", + "a ser", + "ко ва", + "ков а", + "▁qu art", + "▁q uart", + "▁quar t", + "ry ption", + "rypt ion", + "All oc", + "Al loc", + "ft en", + "fte n", + "f ten", + "Oper and", + "▁ind icated", + "▁indic ated", + "▁indicate d", + "($ _", + "( $_", + "get String", + "▁list ener", + "▁listen er", + "▁ listener", + "sp ir", + "spi r", + ") _", + "ve ns", + "ven s", + "v ens", + "▁food s", + "▁foo ds", + "an za", + "anz a", + "te il", + "DE SC", + "▁n otion", + "▁not ion", + "▁em ployment", + "▁employ ment", + "▁s wing", + "▁sw ing", + "▁ swing", + "nb sp", + "▁p ounds", + "▁pound s", + "to ols", + "tool s", + "too ls", + "t ools", + "▁particip ate", + "▁T ax", + "▁Ta x", + "▁ Tax", + "▁с кла", + "ap ol", + "a pol", + "▁f ost", + "▁fo st", + "▁fos t", + "com pat", + "comp at", + "▁public ation", + "▁rapid ly", + "▁W is", + "▁Wi s", + "Event Listener", + "▁prem ière", + "▁premi ère", + "us o", + "u so", + "ext end", + "▁M ERCHANTABILITY", + "UT F", + "U TF", + "▁exper iments", + "▁experi ments", + "▁experiment s", + "sin gle", + "sing le", + "s ingle", + "z k", + "▁n aj", + "▁na j", + "}} }", + "} }}", + "Li n", + "L in", + "▁inter act", + "▁inte ract", + "▁c ms", + "▁cm s", + "▁Ro ger", + "▁Rog er", + "▁Р у", + "> '", + "com mit", + "comm it", + "ло сь", + "▁out come", + "▁h its", + "▁hit s", + "▁hi ts", + "▁и м", + "▁ им", + "▁s park", + "▁sp ark", + "con sole", + "cons ole", + "▁ver w", + "▁ve rw", + "▁ка то", + "agnost ics", + "agnostic s", + "▁s oci", + "▁so ci", + "▁soc i", + "▁d ining", + "▁di ning", + "▁din ing", + "▁t ech", + "▁te ch", + "▁ tech", + "š t", + "fo lio", + "fol io", + "ult ane", + "ultan e", + "кт ор", + "кто р", + "к тор", + "▁B rand", + "▁Br and", + "▁Bra nd", + "Jo in", + "J oin", + "▁и ю", + "▁p ros", + "▁pro s", + "▁pr os", + "▁pos it", + "Pub lic", + "P ublic", + "AspNet Core", + "▁S hop", + "▁Sh op", + "▁ Shop", + "▁co inc", + "▁coin c", + "ни ем", + "ние м", + "▁re ferences", + "▁refer ences", + "▁reference s", + "ab out", + "name space", + "names pace", + "D L", + "▁I R", + "▁ IR", + "▁c ada", + "▁ca da", + "▁cad a", + "▁Jord an", + "▁g ep", + "▁ge p", + "▁b ron", + "▁br on", + "▁bro n", + "andid ate", + "EX PECT", + "EXP ECT", + "am o", + "a mo", + "▁De utsch", + "au c", + "a uc", + "▁ра йо", + "▁рай о", + "▁L abor", + "▁La bor", + "▁Lab or", + "▁surround ed", + "т ро", + "▁n ome", + "▁no me", + "▁nom e", + "▁under lying", + "▁educ ational", + "▁education al", + "R IGHT", + "CO UNT", + "in ch", + "inc h", + "Ty p", + "T yp", + "um ph", + "ump h", + "fo ur", + "f our", + "Control s", + "▁c p", + "▁ cp", + "co st", + "cos t", + "c ost", + "▁mechan ism", + "en ess", + "ene ss", + "enes s", + "e ness", + "é qu", + "▁acqu ired", + "▁acquire d", + "▁f alls", + "▁fall s", + "▁fal ls", + "▁ falls", + "▁H ou", + "▁Ho u", + "▁L E", + "▁ LE", + "for Each", + "▁ver tex", + "▁vert ex", + "▁ vertex", + "▁I F", + "▁ IF", + "cur s", + "cu rs", + "c urs", + "' =>", + "те ри", + "тер и", + "▁S A", + "▁ SA", + "ri ers", + "rie rs", + "rier s", + "r iers", + "▁u w", + "▁ uw", + "▁m arks", + "▁mark s", + "▁mar ks", + "▁ marks", + "▁en erg", + "▁ener g", + "ho f", + "h of", + "ylv ania", + "▁Al len", + "▁All en", + "um py", + "ump y", + "о го", + "ст ву", + "ств у", + "vo ice", + "v oice", + "▁en gage", + "▁eng age", + "▁m ant", + "▁man t", + "▁ma nt", + "or se", + "ors e", + "== =", + "= ==", + "▁impro vement", + "▁improve ment", + "Op t", + "O pt", + "▁arr ested", + "▁arrest ed", + "ти я", + "▁с ле", + "▁ сле", + "it ched", + "itch ed", + "soc ket", + "sock et", + "s ocket", + "▁c ycl", + "▁cy cl", + "▁ cycl", + "▁S M", + "▁ SM", + "▁S ex", + "▁Se x", + "▁neut ral", + "▁neutr al", + "ва в", + "▁J ess", + "▁Je ss", + "▁Jes s", + "▁d ip", + "▁di p", + "▁op position", + "▁oppos ition", + "▁b orrow", + "▁bor row", + "с пе", + "▁av ant", + "ко ла", + "▁t a", + "▁ ta", + "An im", + "A nim", + "▁G all", + "▁Gal l", + "▁Ga ll", + "rg b", + "r gb", + "▁gu ilty", + "▁guilt y", + "▁bu ried", + "▁bur ied", + "▁g y", + "▁ gy", + "Init ial", + "▁acc omp", + "▁ac comp", + "▁accom p", + "▁breath ing", + "▁breat hing", + "ber ry", + "b erry", + "GR O", + "G RO", + "▁subsequ ent", + "rou pe", + "roup e", + "ul pt", + "ulp t", + "t b", + "▁ ä", + "P i", + "arg v", + "▁M ust", + "▁Mus t", + "▁Mu st", + "▁ Must", + ": '", + "sv g", + "ou p", + "o up", + "▁prec isely", + "▁precise ly", + "▁T a", + "re na", + "ren a", + "r ena", + "▁f older", + "▁fol der", + "▁fold er", + "▁ folder", + "▁Ch annel", + "▁ Channel", + "▁re vol", + "▁rev ol", + "M iss", + "ло м", + "red dit", + "adel ph", + "▁dis crim", + "▁disc rim", + "▁a ve", + "▁av e", + "▁ ave", + "pl eted", + "ple ted", + "plete d", + "plet ed", + "p leted", + "▁g ently", + "▁gent ly", + "FF FF", + "ro py", + "rop y", + "r opy", + "▁d ial", + "▁di al", + "▁dia l", + "Not Found", + "▁\" [", + "Hom e", + "H ome", + "on te", + "ont e", + "o nte", + "▁re lie", + "▁rel ie", + "▁reli e", + "▁Con text", + "▁Cont ext", + "▁ Context", + "▁st ats", + "▁stat s", + "▁sta ts", + "▁ stats", + "▁E nergy", + "oun ced", + "ounce d", + "▁gr ave", + "▁grav e", + "▁gra ve", + "▁re cip", + "▁rec ip", + "ли н", + "л ин", + "bl og", + "blo g", + "b log", + "▁na am", + "▁w o", + "▁ wo", + "▁direct ions", + "▁dire ctions", + "▁direction s", + "▁Lin coln", + "! )", + "un ci", + "unc i", + "ne q", + "n eq", + "Tag s", + "T ags", + "▁t um", + "▁tu m", + "▁s aving", + "▁sa ving", + "▁sav ing", + "ail le", + "ai lle", + "a ille", + "item ize", + "▁F amil", + "▁Fa mil", + "ms m", + "m sm", + "ne ws", + "new s", + "FF ER", + "F FER", + "▁D ead", + "▁De ad", + "▁ Dead", + "▁terr itory", + "▁territor y", + "▁territo ry", + "▁K at", + "▁Ka t", + "oc ker", + "ock er", + "o cker", + "in teger", + "inte ger", + "▁s ne", + "▁sn e", + "▁f ails", + "▁fa ils", + "▁fail s", + "▁franç ais", + "▁int roduction", + "▁introdu ction", + "▁G rant", + "▁Gr ant", + "▁Gran t", + "▁Gra nt", + "ycl e", + "yc le", + "y cle", + "'] .", + "' ].", + "▁v ier", + "▁vi er", + "▁vie r", + "▁ vier", + "nat ive", + "n ative", + "▁K le", + "▁Kl e", + "qu ote", + "quot e", + "User s", + "Us ers", + "Use rs", + "▁ad vis", + "▁adv is", + "▁g ym", + "▁gy m", + "▁prote in", + "ا ل", + "▁M ai", + "▁Ma i", + "▁prov iders", + "▁provide rs", + "▁provider s", + "▁so il", + "gu i", + "g ui", + "▁N ation", + "▁Nat ion", + "re ation", + "reat ion", + "▁T ab", + "▁Ta b", + "▁ Tab", + "en sis", + "ens is", + "in as", + "ina s", + "i nas", + "▁Scot land", + "▁dis patch", + "▁disp atch", + "▁ dispatch", + "un ion", + "uni on", + "▁b ere", + "▁be re", + "▁ber e", + "▁ bere", + "▁P ow", + "▁Po w", + "▁H ig", + "▁Hi g", + "▁stud ying", + "▁study ing", + "RE F", + "R EF", + "SS L", + "S SL", + "▁f right", + "▁fr ight", + "▁S ORT", + "▁SO RT", + "▁com pr", + "▁comp r", + "▁Mad rid", + "row ned", + "rown ed", + "r owned", + "op es", + "ope s", + "o pes", + "pd ev", + "p dev", + "▁w ash", + "▁was h", + "▁wa sh", + "▁' ../../", + "▁'../ ../", + "}} _", + "} }_", + "▁acc um", + "rol ling", + "roll ing", + "▁N C", + "▁ NC", + "▁f iction", + "▁fi ction", + "▁fict ion", + "ip t", + "i pt", + "conne cted", + "connect ed", + "lim its", + "limit s", + "▁l ap", + "▁la p", + "▁ lap", + "▁where as", + "pro m", + "pr om", + "p rom", + "▁appoint ment", + "Pro gram", + "Pr ogram", + "▁П ер", + "▁Пе р", + "na h", + "n ah", + "Valid ation", + "ic ons", + "ico ns", + "icon s", + "i cons", + "äl l", + "ä ll", + "▁rad ical", + "▁radi cal", + "▁ex clusive", + "▁excl usive", + "▁exclus ive", + "em ony", + "emon y", + "▁challeng ing", + "▁m s", + "▁ ms", + "▁P rivate", + "▁Priv ate", + "▁ Private", + "▁v ida", + "▁vi da", + "▁vid a", + "▁дру ги", + "▁camp us", + "▁cam pus", + "form s", + "for ms", + "д но", + "pl aat", + "bs t", + "b st", + "AT ED", + "ATE D", + "▁Ab stract", + "▁Abs tract", + "▁ Abstract", + "▁int ense", + "▁intens e", + "▁L td", + "▁contro vers", + "ó g", + "▁s ă", + "▁land ing", + "▁lan ding", + "! =", + "▁sc enes", + "▁scene s", + "▁scen es", + "▁Ch ap", + "▁Cha p", + "▁sp oken", + "▁spoke n", + "▁spo ken", + "cre d", + "cr ed", + "c red", + "▁p ride", + "▁pr ide", + "▁pri de", + "qu et", + "que t", + "▁m eter", + "▁me ter", + "▁met er", + "▁de utsch", + "uu m", + "u um", + "▁b less", + "▁bl ess", + "▁ble ss", + "▁H ann", + "▁Ha nn", + "▁Han n", + "▁input s", + "▁ inputs", + "▁R ow", + "▁Ro w", + "▁ Row", + "▁with draw", + "▁withd raw", + "P al", + "ac les", + "acle s", + "acl es", + "a cles", + "as sets", + "ass ets", + "asse ts", + "asset s", + "▁v l", + "▁ vl", + "ве де", + "вед е", + "▁G ot", + "▁Go t", + "▁air port", + "win d", + "wi nd", + "w ind", + "▁Columb ia", + "▁ch ocolate", + "▁h ö", + "▁ hö", + "▁al arm", + "FT WARE", + "▁J ay", + "▁Ja y", + "▁s ake", + "▁sa ke", + "▁reg istration", + "▁registr ation", + "vi d", + "v id", + "▁l ake", + "▁la ke", + "▁user name", + "▁ username", + "▁h ack", + "▁ha ck", + "index Of", + "c x", + "▁f estival", + "▁fest ival", + "▁club s", + "case s", + "ca ses", + "cas es", + "c ases", + "CT RL", + "]; \r", + "] ;\r", + "▁A ud", + "▁Au d", + "▁ Aud", + "▁prim era", + "▁prime ra", + "▁primer a", + "ва т", + "в ат", + "▁brill iant", + "ut her", + "uth er", + "u ther", + "▁difficult y", + "it als", + "ital s", + "ita ls", + "▁sc ores", + "▁score s", + "▁pol ít", + "data base", + "dat abase", + "as ka", + "ask a", + "a ska", + "▁## ####", + "▁### ###", + "▁#### ##", + "▁##### #", + "▁a cid", + "▁ac id", + "at on", + "ato n", + "a ton", + "at omic", + "ato mic", + "atom ic", + "fr eq", + "fre q", + "f req", + "▁WARRAN TY", + "▁report ing", + ".) ,", + ". ),", + "▁n ights", + "▁night s", + "▁program me", + ")} {", + ") }{", + "xi c", + "x ic", + "▁s po", + "▁sp o", + "line d", + "li ned", + "lin ed", + "l ined", + "qu arters", + "er ee", + "ere e", + "e ree", + "mer s", + "me rs", + "m ers", + "▁s erves", + "▁ser ves", + "▁serv es", + "▁serve s", + "co w", + "c ow", + "ль ко", + "en so", + "ens o", + "▁env iron", + "▁ environ", + "Li ke", + "L ike", + "an che", + "anc he", + "anch e", + "▁cr ash", + "▁K ap", + "▁Ka p", + "no indent", + "Con n", + "Co nn", + "▁ав то", + "▁in frastructure", + "IM E", + "I ME", + "▁R oom", + "▁Ro om", + "▁ Room", + "ne ed", + "n eed", + "or er", + "ore r", + "o rer", + "▁D est", + "▁De st", + "▁Des t", + "▁ Dest", + "▁D omin", + "▁Do min", + "▁Dom in", + "ather ine", + "▁Syd ney", + "▁g auge", + "▁gau ge", + "▁ga uge", + "▁j et", + "▁je t", + "▁ jet", + "b ably", + "▁comm only", + "▁common ly", + "▁st ations", + "▁stat ions", + "▁station s", + "ia h", + "i ah", + "n l", + "ж у", + "et en", + "ete n", + "e ten", + "_ )", + "ia c", + "i ac", + "am os", + "amo s", + "a mos", + "ne ment", + "nem ent", + "n ement", + "ko n", + "k on", + "Inter val", + "▁cab in", + "▁ca bin", + "▁e g", + "▁ eg", + "▁sh ots", + "▁shot s", + "▁ shots", + "▁A rea", + "▁Ar ea", + "▁Are a", + "▁ Area", + "sm ith", + "param eter", + "' }", + "▁h em", + "▁he m", + "▁ hem", + "▁s inging", + "▁sing ing", + "▁sin ging", + "▁access ible", + "▁P rin", + "▁Pr in", + "▁Pri n", + "opt ional", + "option al", + "an cial", + "anc ial", + "ancia l", + "sh ips", + "ship s", + "▁can vas", + "▁ canvas", + "sp e", + "s pe", + "▁address es", + "▁x ml", + "▁ xml", + "▁' \"", + "▁ '\"", + "▁k ar", + "▁ka r", + "▁ kar", + "ö ff", + "▁a ges", + "▁ag es", + "▁age s", + "▁ ages", + "ё р", + "zi ng", + "zin g", + "z ing", + "▁ö ver", + "▁C lean", + "▁Cle an", + "▁ Clean", + "▁Sil ver", + "▁о со", + "▁ос о", + "he alth", + "Al i", + "A li", + "▁t s", + "▁ ts", + "at ern", + "ate rn", + "ater n", + "a tern", + "▁cho osing", + "▁bur ned", + "▁burn ed", + "br id", + "b rid", + "ro oms", + "room s", + "öt t", + "ö tt", + "K ERN", + "▁d ish", + "▁dis h", + "▁di sh", + "S a", + "De tail", + "Det ail", + "▁H ind", + "▁Hi nd", + "▁D ans", + "▁Dan s", + "▁Da ns", + "i ę", + "▁J ahren", + "▁Jah ren", + "▁Jahr en", + "▁Jahre n", + "▁Ja hren", + "ext ension", + "al las", + "all as", + "alla s", + "▁B illy", + "▁Bill y", + "▁Bil ly", + "us ammen", + "it ud", + "itu d", + "ge on", + "geo n", + "Te mp", + "T emp", + "Le g", + "L eg", + "itt el", + "itte l", + "add le", + "▁mus cle", + "▁sc ared", + "▁scar ed", + "ss on", + "s son", + "▁de note", + "▁den ote", + "ie urs", + "ieu rs", + "ieur s", + "i eurs", + "▁o range", + "▁or ange", + "▁h ub", + "▁ hub", + "▁re b", + "▁r eb", + "▁ reb", + "ed i", + "e di", + "▁vo ices", + "▁voice s", + "F older", + "▁s uspend", + "▁sus pend", + "▁susp end", + "▁ suspend", + "▁He art", + "▁sc rap", + "▁scr ap", + "▁a ggreg", + "▁ag greg", + "▁Gu ide", + "trans action", + "▁r iding", + "▁ri ding", + "▁rid ing", + "▁v á", + "▁ vá", + "▁b reed", + "▁br eed", + "▁bre ed", + "▁bree d", + "▁con cert", + "▁conc ert", + "▁conce rt", + "▁concer t", + "appro x", + "▁ch ances", + "▁chance s", + "To k", + "T ok", + "E q", + "par ts", + "part s", + "p arts", + "▁sch olar", + "▁schol ar", + "of fs", + "off s", + "fl ush", + "flu sh", + "! ”", + "▁lo gin", + "▁log in", + "▁ login", + "▁so ort", + "▁M and", + "▁Man d", + "▁Ma nd", + "▁function al", + "▁B ou", + "▁Bo u", + "▁subject s", + "my s", + "m ys", + "▁extra ord", + "▁Build ing", + "ik t", + "i kt", + "B ad", + "ia mi", + "iam i", + "i ami", + "Dr iver", + "D river", + "êt e", + "ê te", + "▁k v", + "▁ kv", + "▁t imer", + "▁time r", + "▁tim er", + "▁ti mer", + "▁ timer", + "ition ally", + "itional ly", + "▁a thlet", + "▁ath let", + "▁\" );", + "▁\") ;", + "▁ \");", + "w y", + "CF G", + "▁he aven", + "▁heav en", + "о в", + "▁exper imental", + "▁experiment al", + "▁b ounds", + "▁bound s", + "▁ bounds", + "IC K", + "I CK", + "▁ex cit", + "▁exc it", + "▁qu it", + "▁qui t", + "▁q uit", + "▁univers al", + "д ь", + "▁S P", + "▁ SP", + "▁st ub", + "▁ stub", + "▁k le", + "▁kl e", + "▁ kle", + "▁B art", + "▁Bar t", + "▁Ba rt", + "▁\" @", + "pe l", + "p el", + "▁( !(", + "▁(! (", + "▁se lector", + "▁select or", + "▁sel ector", + "▁sele ctor", + "▁ selector", + "E B", + "▁c oc", + "▁co c", + "et ed", + "ete d", + "e ted", + "ют ь", + "ю ть", + "▁poss ess", + "▁R ick", + "▁Ric k", + "▁unus ual", + "ter min", + "term in", + "▁b ags", + "▁bag s", + "▁ba gs", + "▁lo ading", + "▁load ing", + "▁ loading", + "▁t f", + "▁ tf", + "▁) \r", + "▁ )\r", + "pro vider", + "prov ider", + "plet ion", + "▁c ursor", + "▁cur sor", + "▁ cursor", + "▁pa used", + "▁paus ed", + "▁pause d", + "и м", + "▁coun sel", + "] <", + "ze ch", + "zec h", + "z ech", + "▁t ie", + "▁ti e", + "▁M oon", + "▁Mo on", + "▁ar med", + "▁arm ed", + "▁ armed", + "▁ob serve", + "▁observ e", + "▁obs erve", + "▁per met", + "▁perm et", + "▁J ob", + "▁Jo b", + "▁ Job", + "fö r", + "f ör", + "arg ument", + "▁egg s", + "▁eg gs", + "ás t", + "á st", + "▁incred ibly", + "wer ken", + "werk en", + "iz ard", + "izar d", + "iza rd", + "▁p ainted", + "▁pain ted", + "▁pa inted", + "▁paint ed", + "▁Viet nam", + "▁vi olent", + "▁viol ent", + "Es t", + "E st", + "ier ra", + "i erra", + "re ader", + "read er", + "rea der", + "we ise", + "wei se", + "▁J osh", + "▁Jo sh", + "▁Jos h", + "▁H im", + "▁Hi m", + "as hes", + "ash es", + "or igin", + "orig in", + "ori gin", + "▁sp ir", + "▁ spir", + "▁T ree", + "▁Tr ee", + "▁Tre e", + "▁ Tree", + "▁n iet", + "▁nie t", + "▁ni et", + "WI N", + "W IN", + "mar gin", + "m argin", + "▁inv olves", + "▁invol ves", + "▁involve s", + "▁organ is", + "▁N acional", + "bar a", + "ba ra", + "b ara", + "▁de puis", + "▁dep uis", + "pi o", + "p io", + "fe atures", + "feature s", + "feat ures", + "st ru", + "str u", + "▁Dis ney", + "▁restaur ants", + "▁restaurant s", + "Mil l", + "M ill", + ")) \r", + ") )\r", + "с ла", + "rem ote", + "▁Th ird", + "▁base ball", + "▁al gun", + "▁alg un", + "] $", + "▁em ployed", + "▁employ ed", + "po t", + "p ot", + "▁Un ityEngine", + "▁ UnityEngine", + "▁integr ation", + "▁risk s", + "▁ris ks", + "▁st ro", + "▁str o", + "▁ag osto", + "▁ago sto", + "incl uding", + "▁M ind", + "▁Min d", + "▁Mi nd", + "▁st roke", + "▁str oke", + "▁stro ke", + "▁ stroke", + "▁de als", + "▁deal s", + "aj ax", + "aja x", + "a jax", + "ё т", + "▁\\ |", + "▁ \\|", + "ta r", + "t ar", + "adelph ia", + "▁s ab", + "▁sa b", + "pu r", + "p ur", + "▁sc rew", + "▁scr ew", + "▁in ev", + "▁\\ ;", + "▁Don ald", + "▁ Donald", + "ö d", + "cc a", + "c ca", + "es is", + "esi s", + "e sis", + "▁separ ated", + "▁separate d", + "DB G", + "D BG", + "ag ent", + "age nt", + "agen t", + "a gent", + "▁p acked", + "▁pack ed", + "▁pac ked", + "▁ packed", + "н ня", + "in tern", + "int ern", + "inter n", + "inte rn", + "▁M onte", + "▁Mon te", + "▁Mont e", + "▁Mo nte", + "▁prov ince", + "▁provinc e", + "▁provin ce", + "▁exp anded", + "▁expand ed", + "▁appro ached", + "▁approach ed", + "▁E p", + "CL K", + "▁o re", + "▁or e", + "▁ ore", + "B atch", + "▁impress ive", + "R M", + "▁L ocation", + "▁Loc ation", + "▁ Location", + "▁sh ame", + "▁sha me", + "wrap per", + "w rapper", + "un wrap", + "pe er", + "Bit s", + "Bi ts", + "B its", + "▁S N", + "▁ SN", + "sc ar", + "s car", + "Com e", + "Co me", + "C ome", + "▁coun cil", + "▁shout ed", + "ma king", + "m aking", + "▁M aur", + "▁Ma ur", + "▁Mau r", + "▁w is", + "LE TE", + "LET E", + "▁f s", + "▁ fs", + "▁d z", + "▁ dz", + "un que", + "ue go", + "u ego", + "R andom", + "H tml", + "ze m", + "z em", + "▁D utch", + "▁Gold en", + "▁Gol den", + "▁T ar", + "▁Ta r", + "▁H erm", + "▁He rm", + "▁Her m", + "▁str etch", + "▁stret ch", + "var d", + "va rd", + "v ard", + "▁t ries", + "▁tr ies", + "▁tri es", + "W I", + "▁disappe ared", + "▁disappear ed", + "▁cr usher", + "▁crush er", + "▁K an", + "▁Ka n", + "Ma g", + "M ag", + "ø r", + "▁Cam bridge", + "▁Camb ridge", + "▁do po", + "▁dop o", + "at ura", + "atur a", + "atu ra", + "he art", + "▁Sp iel", + "/* *\r", + "/** \r", + "Dir ection", + "Direct ion", + "Di rection", + "D irection", + "at ting", + "att ing", + "atti ng", + "wi g", + "w ig", + "▁c odes", + "▁co des", + "▁code s", + "▁cod es", + "▁ codes", + "▁pow der", + "al ert", + "ale rt", + "aler t", + "sem bl", + "semb l", + "▁y e", + "▁ ye", + "St ar", + "S tar", + "▁ro ots", + "▁root s", + "▁H oll", + "▁Hol l", + "▁Ho ll", + "Re le", + "Rel e", + "R ele", + "▁const itu", + "n c", + "“ .", + "re ference", + "refer ence", + "if icial", + "ific ial", + "ifi cial", + "clos ure", + "▁fig ured", + "▁figure d", + "▁assum ption", + "getElement ById", + "▁A G", + "▁ AG", + "os es", + "ose s", + "o ses", + "▁_ \"", + "ep per", + "ob re", + "o bre", + "en umerate", + "о графи", + "▁less ons", + "▁lesson s", + "▁qual ified", + "Per son", + "Pers on", + "P erson", + "an se", + "ans e", + "▁M ort", + "▁Mor t", + "▁Mo rt", + "s ylvania", + "▁c ré", + "▁cr é", + "Bind ing", + "Bin ding", + "B inding", + "і с", + "▁V ari", + "▁Var i", + "▁Va ri", + "▁ Vari", + "▁re minded", + "▁remind ed", + "▁members hip", + "▁member ship", + "ip er", + "ipe r", + "i per", + "zt e", + "z te", + "▁c ref", + "▁cre f", + "▁cr ef", + "▁ cref", + "▁P A", + "▁ PA", + "plaat st", + "▁Env ironment", + "▁ Environment", + "bo y", + "b oy", + "▁ph rase", + "▁phr ase", + "▁ phrase", + "riv ial", + "ra g", + "r ag", + "во ди", + "вод и", + "▁p se", + "▁ps e", + "▁ pse", + "▁tour nament", + ")} ,", + ") },", + "▁S ound", + "▁So und", + "▁Sou nd", + "▁ Sound", + "▁V el", + "▁Ve l", + "▁ Vel", + "▁B erg", + "▁Be rg", + "▁Ber g", + "el son", + "els on", + "▁ref uge", + "▁else where", + "qu ality", + "qual ity", + "▁abandon ed", + "▁F lo", + "▁Fl o", + "ib il", + "i bil", + "UA L", + "U AL", + "▁Pl atz", + "▁d elta", + "▁del ta", + "▁ delta", + "▁B uy", + "▁Bu y", + "ri ère", + "r ière", + "▁fl our", + "▁flo ur", + "▁laugh ing", + "▁laug hing", + "▁Look ing", + "▁Lo oking", + "Ag ent", + "A gent", + "▁w x", + "▁ wx", + "▁W ales", + "▁Wal es", + "▁Wa les", + "C tx", + "▁c ake", + "▁ca ke", + "▁c rate", + "▁cr ate", + "▁ crate", + "▁к ла", + "▁ кла", + "an ga", + "ang a", + "Z ero", + "▁amount s", + "Tr a", + "T ra", + "om etric", + "omet ric", + "o metric", + "▁con straints", + "▁constr aints", + "▁constraint s", + "▁tem ple", + "▁templ e", + "▁temp le", + "▁install ation", + "st roke", + "str oke", + "▁N eder", + "▁Ne der", + "▁Ned er", + "ț i", + "▁I bid", + "▁o bs", + "▁ob s", + "▁ obs", + "ent ries", + "entr ies", + "▁j usqu", + "OR M", + "O RM", + "▁S ky", + "▁Sk y", + "ik es", + "ike s", + "i kes", + "na k", + "n ak", + "▁m odes", + "▁mod es", + "▁mo des", + "▁mode s", + "▁Hit ler", + "▁b elt", + "▁be lt", + "▁bel t", + "▁point ing", + "▁B an", + "▁Ba n", + "ign ore", + "▁per su", + "▁pers u", + "▁Bes ides", + "yn om", + "y nom", + "▁leg is", + "▁C PU", + "▁CP U", + "▁ CPU", + "an ded", + "and ed", + "ande d", + "ui s", + "u is", + "bs ite", + "b site", + "▁E uro", + "▁Eu ro", + "▁ut ter", + "▁ utter", + "e clipse", + "▁ir re", + "▁irr e", + "▁D ocument", + "▁Doc ument", + "▁ Document", + "▁Mean while", + "▁famil ie", + "ver ify", + "▁J ason", + "▁Ja son", + "▁O rt", + "▁Or t", + "▁ci udad", + "▁techn ologies", + "▁ча сти", + "▁част и", + "▁час ти", + "ni ca", + "nic a", + "n ica", + "can cel", + "c ancel", + "V irtual", + "▁ev ident", + "am an", + "ama n", + "a man", + "▁Sup reme", + "at oes", + "ato es", + "▁ste ady", + "▁stead y", + "▁month ly", + "▁SO FTWARE", + "Di e", + "D ie", + "▁app lying", + "▁apply ing", + "▁appl ying", + "Di g", + "D ig", + "vi er", + "v ier", + "▁го ро", + "▁W H", + "▁ WH", + "▁min ds", + "▁mind s", + "▁k am", + "▁ka m", + "▁expert ise", + "▁not ification", + "▁ notification", + ". -", + "▁del iber", + "▁H E", + "▁ HE", + "▁res ist", + "ou tes", + "out es", + "oute s", + "o utes", + "▁How ard", + "▁Ho ward", + "spec ial", + "spe cial", + "▁p resentation", + "▁present ation", + "▁You Tube", + "mi r", + "m ir", + "▁r ust", + "▁ru st", + "▁rus t", + "▁ rust", + "▁n ations", + "▁nat ions", + "▁nation s", + "▁G ets", + "▁Ge ts", + "▁Get s", + "▁respon ses", + "▁response s", + "▁respons es", + "ar ded", + "ard ed", + "arde d", + "im mer", + "imm er", + "▁reve al", + "▁M eg", + "▁Me g", + "▁tod os", + "▁todo s", + "▁a de", + "▁ad e", + "▁ ade", + "ateg ories", + "ategor ies", + "▁pay ments", + "▁payment s", + "ô t", + "En umer", + "Enum er", + "E numer", + "▁platform s", + "▁plat forms", + "▁life time", + "▁lif etime", + "Com plete", + "Comp lete", + "Qu est", + "Que st", + "Q uest", + "en ders", + "end ers", + "ender s", + "ende rs", + "▁c um", + "▁cu m", + "pl er", + "ple r", + "p ler", + "▁app l", + "▁ap pl", + "äh rend", + "ähr end", + "з ь", + "en ez", + "ene z", + "e nez", + "over ty", + "yn chron", + "▁arg ued", + "▁argue d", + "▁K ath", + "▁Kat h", + "▁Ka th", + "▁s ynchron", + "▁syn chron", + "▁B uilder", + "▁Build er", + "▁ Builder", + "B order", + "Pl an", + "P lan", + "ri eb", + "rie b", + "r ieb", + "n m", + "FOR MAT", + "FORM AT", + "us k", + "u sk", + "▁j umped", + "▁jump ed", + "ch arg", + "char g", + "cha rg", + "▁cont ribute", + "▁contribut e", + "Me sh", + "M esh", + "Un ivers", + "re ll", + "rel l", + "r ell", + "▁p olar", + "▁pol ar", + "▁po lar", + "▁tr ois", + "▁tro is", + "ic io", + "ici o", + "i cio", + "Group s", + "G roups", + "▁( %", + "▁ (%", + "Lo op", + "L oop", + "▁g az", + "▁ga z", + "db g", + "d bg", + "LA Y", + "L AY", + "Jo hn", + "J ohn", + "bl ocks", + "block s", + "blo cks", + "▁l ung", + "▁lu ng", + "▁lun g", + "▁ lung", + "▁k ön", + "▁kö n", + "th rough", + "▁fif th", + "lish er", + "l isher", + "▁inv olving", + "▁invol ving", + "▁De ep", + "▁ Deep", + "▁обла сти", + "▁s ull", + "▁su ll", + "▁sul l", + "Ex port", + "Exp ort", + "▁K ate", + "▁Kat e", + "▁Ka te", + "per iod", + "ch arge", + "char ge", + "charg e", + "G T", + "\"> \r", + "\" >\r", + "ти н", + "т ин", + "▁O tt", + "▁Ot t", + "▁inter actions", + "▁interaction s", + "▁interact ions", + "▁Tor onto", + "TR ACE", + "TRA CE", + "▁d ifer", + "▁di fer", + "▁dif er", + "▁lib eral", + "▁liber al", + "▁p article", + "▁part icle", + "▁partic le", + "▁parti cle", + "▁sur ve", + "▁surv e", + "al ous", + "alo us", + "re ason", + "rea son", + "▁de pression", + "▁dep ression", + "▁depress ion", + "а л", + "▁f lower", + "▁fl ower", + "▁flo wer", + "▁flow er", + "▁wa ar", + "▁h ade", + "▁had e", + "▁ha de", + "▁cent uries", + "ut y", + "u ty", + "par ty", + "part y", + "▁appro val", + "gener ate", + "▁B arn", + "▁Bar n", + "▁Ba rn", + "▁m arg", + "▁mar g", + "▁ma rg", + "▁m onde", + "▁mon de", + "▁mo nde", + "▁mond e", + "▁o ok", + "▁ ook", + "▁Cl ark", + "▁Clar k", + "▁the oret", + "vious ly", + "vi ously", + "v iously", + "? )", + "▁R ud", + "▁Ru d", + "st mt", + "in ction", + "inct ion", + "▁t un", + "▁tu n", + "▁ro ads", + "▁road s", + "▁rot ation", + "▁ rotation", + "pp en", + "ppe n", + "p pen", + "sen sor", + "s ensor", + "▁K ol", + "▁Ko l", + "id elines", + "ide lines", + "idel ines", + "▁ є", + "▁com posed", + "▁comp osed", + "▁compos ed", + "▁v irus", + "▁vi rus", + "▁vir us", + "' $", + "S N", + "▁V on", + "▁Vo n", + "mon t", + "mo nt", + "m ont", + "la r", + "l ar", + "▁opin ions", + "▁opinion s", + "uct ion", + "u ction", + "ru pal", + "rup al", + "under line", + "▁hor ror", + "Mus t", + "Mu st", + "M ust", + "ot to", + "ott o", + "o tto", + "Sh ould", + "▁stat ist", + "▁g em", + "▁ge m", + "▁ gem", + "▁se cre", + "▁sec re", + "▁st rip", + "▁str ip", + "▁stri p", + "▁ strip", + "▁d irt", + "▁di rt", + "▁dir t", + "ama zon", + "amaz on", + "▁R ound", + "▁Ro und", + "▁Rou nd", + "▁ Round", + "▁dis covery", + "▁disc overy", + "▁discover y", + "▁disco very", + "▁G O", + "▁ GO", + "▁substant ial", + "ib t", + "i bt", + "▁dem ands", + "▁demand s", + "▁every day", + "▁b esch", + "▁be sch", + "▁bes ch", + "▁B ridge", + "▁Br idge", + "▁H D", + "▁ HD", + "▁D ol", + "▁Do l", + "▁t rès", + "▁tr ès", + "an ni", + "ann i", + "ro it", + "() );\r", + "()); \r", + "()) ;\r", + "( ));\r", + "fa r", + "f ar", + "tim estamp", + "▁bul k", + "Bl ack", + "▁g an", + "▁ga n", + "▁ gan", + "set ting", + "ret val", + "ва не", + "ван е", + "nu ng", + "n ung", + "▁talk s", + "▁tal ks", + "▁scient ists", + "▁scientist s", + "▁v ig", + "▁vi g", + "▁quant ity", + "▁G ard", + "▁Gar d", + "▁Ga rd", + "▁mov ements", + "▁move ments", + "▁movement s", + "äh r", + "ä hr", + "ling s", + "lin gs", + "l ings", + "▁Т е", + "te am", + "ri to", + "rit o", + "r ito", + "▁as sembly", + "▁ assembly", + "il st", + "ils t", + "i lst", + "▁happ iness", + "▁le af", + "▁ leaf", + "▁ass essment", + "▁assess ment", + "Co ord", + "ir s", + "i rs", + "sa m", + "s am", + "▁att orney", + "▁g eme", + "▁ge me", + "▁gem e", + "▁ geme", + "ID E", + "I DE", + "▁V ere", + "▁Ver e", + "▁Ve re", + "▁Anth ony", + "am iento", + "ami ento", + "▁A st", + "▁As t", + "▁cir cul", + "▁circ ul", + "▁Fr ances", + "▁Franc es", + "▁France s", + "▁Fran ces", + "▁p ent", + "▁pe nt", + "▁pen t", + "▁ pent", + "▁m ate", + "▁mat e", + "▁ma te", + "▁ mate", + "▁Trans port", + "▁ Transport", + "ow o", + "o wo", + "ч у", + "is tes", + "ist es", + "iste s", + "TR AN", + "TRA N", + "T RAN", + "IM PORT", + "IMP ORT", + "▁B reak", + "▁Bre ak", + "▁ Break", + "▁s ons", + "▁so ns", + "▁son s", + "▁invest ors", + "▁Phil ipp", + "▁Philip p", + "TH OD", + "▁pan ic", + "▁pa nic", + "▁ panic", + "▁: )", + "▁d etection", + "▁det ection", + "▁detect ion", + "▁sim ultane", + "nt e", + "n te", + "▁list ened", + "▁listen ed", + "к ре", + "▁B rig", + "▁Br ig", + "Option al", + "Opt ional", + "▁a bund", + "▁ab und", + "▁c riteria", + "▁crit eria", + "▁c hip", + "▁ch ip", + "▁chi p", + "▁ chip", + "▁ок ру", + "▁Con stant", + "▁Const ant", + "▁ Constant", + "▁m ining", + "▁min ing", + "▁mi ning", + "▁mini ng", + "та л", + "т ал", + "ma tes", + "mat es", + "mate s", + "m ates", + "▁w orship", + "▁wor ship", + "ro uter", + "rou ter", + "route r", + "r outer", + "C N", + "▁M atch", + "▁Mat ch", + "▁ Match", + "▁C ole", + "▁Col e", + "▁Co le", + "▁down t", + "▁dow nt", + "▁h oles", + "▁hol es", + "▁ho les", + "▁hole s", + "▁gr ateful", + "RES ULT", + "▁Europ a", + "▁Euro pa", + "▁con sent", + "▁cons ent", + "▁conse nt", + "l ä", + "op ter", + "opt er", + "▁colle agues", + "or ous", + "oro us", + "o rous", + "▁enem ies", + "ha ng", + "han g", + "h ang", + "act ual", + "Object s", + "▁я к", + "▁fl uid", + "▁flu id", + "fix ed", + "f ixed", + "▁G raph", + "▁Gr aph", + "▁Gra ph", + "▁ Graph", + "▁scr atch", + "ce rs", + "cer s", + "c ers", + "ri bu", + "rib u", + "▁valid ation", + "▁ validation", + "▁com pletion", + "▁complet ion", + "▁B egin", + "▁Be gin", + "▁Beg in", + "▁ Begin", + "end point", + "ri ent", + "rie nt", + "rien t", + "r ient", + "C M", + "▁S ite", + "▁Si te", + "▁ Site", + "▁expl ains", + "▁explain s", + "tr es", + "tre s", + "t res", + "▁any body", + "fo reach", + "fore ach", + "for each", + "lo n", + "l on", + "Ch ain", + "▁B uff", + "▁Bu ff", + "▁ Buff", + "oc al", + "oca l", + "o cal", + "▁M organ", + "▁Mor gan", + "▁s ang", + "▁sa ng", + "▁san g", + "▁pass es", + "▁pas ses", + "@ @", + "ij d", + "i jd", + "W ord", + "▁H ung", + "▁Hun g", + "▁Hu ng", + "▁F er", + "▁Fe r", + "▁v ý", + "ba st", + "bas t", + "b ast", + "▁enter tainment", + "▁entertain ment", + "hi n", + "h in", + "▁g rat", + "▁gr at", + "▁gra t", + "▁M ember", + "▁Me mber", + "▁Mem ber", + "▁ Member", + "▁M inn", + "▁Min n", + "▁Mi nn", + "▁pr inted", + "▁print ed", + "▁prin ted", + "▁Frank lin", + "▁I mp", + "▁Im p", + "▁ Imp", + "M achine", + "column s", + "▁de leted", + "▁del eted", + "▁delete d", + "▁delet ed", + "▁ deleted", + "▁manufact uring", + "▁re ly", + "▁r ely", + "▁rel y", + "▁con se", + "▁cons e", + "▁f ishing", + "▁fish ing", + "▁fis hing", + "bl o", + "b lo", + "- $", + "▁. \"", + "▁ .\"", + "▁clin ical", + "▁clinic al", + "▁Stud ies", + "▁Б у", + "def inition", + "▁evalu ation", + "▁eval uation", + "▁att acked", + "▁attack ed", + "▁fro zen", + "ze nt", + "zen t", + "z ent", + "▁ú lt", + "▁r ational", + "▁rat ional", + "▁ratio nal", + "ot he", + "oth e", + "o the", + "Can cel", + "C ancel", + "hi story", + "hist ory", + "set Text", + "▁a lc", + "▁al c", + "▁h ydro", + "▁hy dro", + "▁hyd ro", + "▁The atre", + "▁M aterial", + "▁Mat erial", + "▁ Material", + "IO Exception", + "**** **/", + "****** /", + "sp l", + "s pl", + "NO DE", + "att rs", + "attr s", + "▁m ie", + "▁mi e", + "▁off ices", + "▁offic es", + "▁office s", + "r ó", + "▁j am", + "▁ja m", + "▁Id ent", + "▁Ide nt", + "▁ Ident", + "v é", + "Set ting", + "▁Sever al", + "▁Sev eral", + "▁dec ay", + "And roid", + "▁S ave", + "▁Sa ve", + "▁Sav e", + "▁ Save", + "un ted", + "unt ed", + "unte d", + "▁Mount ain", + "us c", + "u sc", + "▁mar zo", + "▁a sleep", + "▁as leep", + "▁sold ier", + "▁D ouble", + "▁Dou ble", + "▁ Double", + "P K", + "▁cont rad", + "▁contr ad", + "▁contra d", + "▁w ins", + "▁win s", + "ce iver", + "ceive r", + "▁se asons", + "▁season s", + "▁seas ons", + "▁C hall", + "▁Ch all", + "▁Cha ll", + "▁health care", + "ła d", + "ł ad", + "о т", + "▁F ive", + "▁Fi ve", + "▁H ell", + "▁He ll", + "▁Hel l", + "▁world wide", + "▁' ,", + "▁ ',", + "я н", + "ma de", + "mad e", + "m ade", + "▁respon ded", + "▁respond ed", + "▁a y", + "▁ ay", + "▁proced ures", + "▁procedure s", + "те ра", + "тер а", + "▁cle ared", + "▁clear ed", + "\"] .", + "\" ].", + "▁T arget", + "▁Tar get", + "▁ Target", + "▁S ide", + "▁Si de", + "▁Sid e", + "▁ Side", + "om in", + "omi n", + "o min", + "▁de ploy", + "▁T ell", + "▁Te ll", + "▁Tel l", + "▁ Tell", + "▁on going", + "fl oor", + "f loor", + "▁b ones", + "▁bo nes", + "▁bon es", + "▁bone s", + "▁De lete", + "▁Del ete", + "▁ Delete", + "▁shru gged", + "O ur", + "De r", + "D er", + "▁init ialize", + "▁initial ize", + "▁ initialize", + "▁T ed", + "▁Te d", + "MA GE", + "MAG E", + "M AGE", + "▁h ire", + "▁hi re", + "▁ hire", + "▁tr acking", + "▁track ing", + "▁a sh", + "▁as h", + "▁ ash", + "▁ce iling", + "ка х", + "et ti", + "ett i", + "e tti", + "▁cour age", + "▁cou rage", + "ensch app", + "ют ся", + "ю тся", + "Mo re", + "M ore", + "▁fol g", + "▁fo lg", + "▁ folg", + "▁Gr ace", + "▁Gra ce", + "▁K elly", + "▁Kel ly", + "▁re ven", + "▁r even", + "▁rev en", + "▁reve n", + "▁A li", + "▁Al i", + "▁ Ali", + "▁d isp", + "▁dis p", + "▁di sp", + "▁ disp", + "▁de feat", + "▁defe at", + "▁cre ature", + "▁creat ure", + "▁Kenn edy", + "▁D iego", + "▁Die go", + "▁Di ego", + "EM P", + "E MP", + "▁s team", + "▁ste am", + "end ance", + "ri g", + "r ig", + "▁ign or", + "▁ig nor", + "em en", + "eme n", + "e men", + "▁G ru", + "▁Gr u", + "▁pro posal", + "▁propos al", + "▁we iter", + "▁weit er", + "▁ лі", + "ib les", + "ible s", + "i bles", + "▁consider ation", + "▁belie ves", + "▁believe s", + "▁S oph", + "▁So ph", + "“ ,", + "▁Matt hew", + "▁circ uit", + "▁s inger", + "▁sing er", + "▁sin ger", + "▁S quare", + "ç o", + "Ed ge", + "▁a str", + "▁as tr", + "▁ast r", + "▁ astr", + "▁represent ative", + "▁comprehens ive", + "li ga", + "lig a", + "l iga", + "▁m ere", + "▁me re", + "▁mer e", + "tb l", + "t bl", + "▁contin uing", + "▁continu ing", + "ograph er", + "ograp her", + "LE D", + "L ED", + "▁/* **/", + "▁/** */", + "▁s ear", + "▁se ar", + "▁sea r", + "▁enorm ous", + "iz i", + "i zi", + "Di t", + "D it", + "th ere", + "ther e", + "the re", + "t here", + "і н", + "си те", + "▁gu erra", + "▁end point", + "▁ endpoint", + "▁le sson", + "▁les son", + "▁less on", + "zo n", + "z on", + "var iable", + "vari able", + "и с", + "▁research ers", + "▁attempt ed", + "▁e nf", + "▁en f", + "ту ра", + "тур а", + "▁de fin", + "▁def in", + "ве ст", + "▁aw ful", + "▁lo west", + "▁low est", + "ru les", + "rule s", + "r ules", + "▁un like", + "inter val", + "▁produ cing", + "▁K am", + "▁Ka m", + "▁I MP", + "▁IM P", + "▁ IMP", + "Gener al", + "Gen eral", + "▁f aire", + "▁fa ire", + "▁fair e", + "▁max im", + "▁ma xim", + "as semb", + "ass emb", + "asse mb", + "assem b", + "ac ent", + "ace nt", + "a cent", + "? >", + "pl ica", + "plic a", + "p lica", + "▁r am", + "▁ra m", + "▁ ram", + "ma te", + "mat e", + "m ate", + "ц у", + "m n", + "▁H i", + "▁ Hi", + "▁st ages", + "▁stage s", + "▁stag es", + "▁sta ges", + "▁Ed itor", + "▁Edit or", + "▁ Editor", + "▁t ang", + "▁tan g", + "▁ta ng", + "R D", + "▁i ch", + "▁ic h", + "▁ ich", + "▁depend ent", + "▁dep endent", + "▁ dependent", + "li fer", + "life r", + "lif er", + "l ifer", + "as cript", + "asc ript", + "a script", + "▁expos ure", + "ре з", + "▁m art", + "▁mar t", + "▁ma rt", + "▁ mart", + "▁Bar cel", + "xs pace", + "x space", + "SE SSION", + "▁p rest", + "▁pre st", + "▁pr est", + "▁pres t", + "UR CE", + "- .", + "▁се ло", + "ha ve", + "hav e", + "h ave", + "▁observ ation", + "▁obs ervation", + "▁comm ands", + "▁command s", + "▁ commands", + "▁e ager", + "▁out door", + "▁DE BUG", + "▁ DEBUG", + "▁h r", + "▁ hr", + "A X", + "▁p uzz", + "▁pu zz", + "bl ank", + "бу р", + "б ур", + "▁k ennis", + "▁reg arded", + "▁regard ed", + "▁} ),", + "▁}) ,", + "▁ }),", + "vol ume", + "▁про из", + "▁Tr aining", + "▁Tra ining", + "▁Train ing", + "a ñ", + "▁f ois", + "▁foi s", + "▁fo is", + "▁т ри", + "▁ три", + "в ня", + "▁opt imal", + "▁optim al", + "▁sub scription", + "▁subs cription", + "▁ subscription", + "br idge", + "brid ge", + "b ridge", + "im ental", + "iment al", + "imen tal", + "▁Th ink", + "▁\" ;", + "▁ \";", + "▁leg isl", + "▁legis l", + "▁H op", + "▁Ho p", + "▁br anches", + "▁branch es", + "▁V eg", + "▁Ve g", + "▁s print", + "▁spr int", + "▁fl ux", + "▁flu x", + "▁Fr eder", + "▁Fre der", + "▁Fred er", + "si s", + "s is", + "not ify", + "▁Ф ран", + "so m", + "s om", + "ny m", + "n ym", + "▁R é", + "le tt", + "let t", + "l ett", + "ing ham", + "▁F arm", + "▁Far m", + "▁Fa rm", + "DO M", + "D OM", + "▁sh ield", + "He re", + "Her e", + "H ere", + "▁T reat", + "▁Tre at", + "▁Lu ke", + "▁un safe", + "an ton", + "ant on", + "anto n", + "▁Im per", + "▁Imp er", + "▁tele phone", + "▁un lock", + "▁ unlock", + "Own er", + "col lection", + "coll ection", + "collect ion", + "▁s nd", + "▁sn d", + "▁ snd", + "▁su iv", + "▁ent ering", + "▁enter ing", + "ше н", + "ш ен", + "▁L abel", + "▁La bel", + "▁Lab el", + "▁ Label", + "se lector", + "sel ector", + "select or", + "▁G ET", + "▁ GET", + "▁qu ando", + "▁quand o", + "▁f ed", + "▁fe d", + "▁ fed", + "j Query", + "Or igin", + "▁A lan", + "▁Al an", + "math scr", + "▁pregn ant", + "▁preg nant", + "Ex pect", + "Exp ect", + "re sources", + "res ources", + "resource s", + "▁er sten", + "▁erst en", + "▁ers ten", + "▁erste n", + "al ia", + "ali a", + "a lia", + "▁ret ired", + "▁retire d", + "û t", + "Cr ed", + "C red", + "▁m éd", + "▁mé d", + "▁e rh", + "▁er h", + "Frame work", + "Sl ot", + "S lot", + "d uration", + "sa l", + "s al", + "▁com position", + "▁compos ition", + "art icle", + "gp u", + "g pu", + "▁per mitted", + "▁perm itted", + "▁permit ted", + "▁F ont", + "▁Fo nt", + "▁ Font", + "▁M uch", + "▁Mu ch", + "▁p ending", + "▁pen ding", + "▁ pending", + "▁ag encies", + "Column s", + "▁k lik", + "▁kl ik", + "▁r ating", + "▁rat ing", + "▁ra ting", + "▁ rating", + "min d", + "mi nd", + "m ind", + "▁Penn sylvania", + "J ava", + "ab stract", + "abs tract", + "▁d umb", + "▁du mb", + "▁V I", + "▁ VI", + "us a", + "u sa", + "Rem ote", + "▁YO U", + "▁C reek", + "▁Cre ek", + "ма ти", + "мат и", + "Bot tom", + "B ottom", + "▁roll ing", + "▁ rolling", + "▁b undle", + "▁bund le", + "▁ bundle", + "▁g olf", + "▁gol f", + "gp io", + "g pio", + "▁Ch air", + "▁Cha ir", + "▁c ls", + "▁cl s", + "▁ cls", + "$ }", + "▁Par liament", + "f ühr", + "Man y", + "Ma ny", + "M any", + "▁S ep", + "▁Se p", + "▁ Sep", + "▁bad ly", + "ig i", + "i gi", + "▁Geme inde", + "Il l", + "I ll", + "▁А н", + "ua rt", + "uar t", + "u art", + "it empty", + "item pty", + "▁N iger", + "▁Ni ger", + "▁im migr", + "▁imm igr", + "Su per", + "Sup er", + "S uper", + "v á", + "ist ribute", + "istribut e", + "Hel pers", + "Helper s", + "Help ers", + "▁w aters", + "▁water s", + "▁wat ers", + "▁wa ters", + "▁join ing", + "▁jo ining", + "om itempty", + "▁Other wise", + "▁H ost", + "▁Ho st", + "▁ Host", + "▁re dd", + "▁red d", + "▁d y", + "▁ dy", + "▁con verted", + "▁convert ed", + "▁conver ted", + "▁pr ayer", + "▁pray er", + "▁pra yer", + "▁У краї", + "▁Укра ї", + "▁e lections", + "▁elect ions", + "▁ele ctions", + "▁election s", + "re b", + "r eb", + "er ie", + "eri e", + "e rie", + "▁с вя", + "Ab s", + "A bs", + "ie mbre", + "iem bre", + "i embre", + "hol ders", + "hold ers", + "holder s", + "▁R ol", + "▁Ro l", + "ut schen", + "uts chen", + "utsch en", + "utsche n", + "▁G h", + "ter y", + "te ry", + "t ery", + "ан г", + "а нг", + "▁narr ative", + "min us", + "m inus", + "▁I ron", + "▁Ir on", + "=\" #", + "▁w and", + "▁wa nd", + "▁ wand", + "▁w ished", + "▁wish ed", + "▁wis hed", + "ic ode", + "ico de", + "i code", + "or r", + "o rr", + "[ [", + "▁detect ed", + "▁municip al", + "▁P our", + "▁Po ur", + "▁S erv", + "▁Se rv", + "▁Ser v", + "▁ Serv", + "cite t", + "cit et", + "c itet", + "▁g rey", + "▁gr ey", + "▁gre y", + "▁R ap", + "▁Ra p", + "▁v oy", + "▁vo y", + "▁l leg", + "▁ll eg", + "▁cur rency", + "▁curr ency", + "▁ currency", + "▁S cript", + "▁Sc ript", + "▁ Script", + "str ument", + "stru ment", + "▁expect ing", + "▁t ickets", + "▁tick ets", + "▁ticket s", + "▁b ucket", + "▁buck et", + "▁ bucket", + "eg r", + "e gr", + "▁j acket", + "▁jack et", + "dr v", + "d rv", + "▁lo ans", + "▁loan s", + "▁k ann", + "▁kan n", + "▁ka nn", + "▁integr al", + "▁character istics", + "▁characteristic s", + "(\" .", + "( \".", + "▁man ual", + "▁d ynamics", + "▁dynam ics", + "▁dynamic s", + ": *", + "sh a", + "s ha", + "re ens", + "ree ns", + "reen s", + "on ical", + "oni cal", + "onic al", + "▁to ile", + "añ a", + "a ña", + "▁d istant", + "▁di stant", + "▁dist ant", + "▁hand led", + "▁handle d", + "Bo ol", + "B ool", + "▁pe nal", + "▁pen al", + "▁Th ings", + "▁prom inent", + "▁ex ped", + "▁exp ed", + "▁He lp", + "▁Hel p", + "▁ Help", + "▁a sp", + "▁as p", + "▁ asp", + "la p", + "l ap", + "▁A uth", + "▁Aut h", + "▁Au th", + "▁ Auth", + "Bas ic", + "ach uset", + "▁B ild", + "▁Bi ld", + "▁Bil d", + "▁ent itled", + "▁j ag", + "▁ja g", + "▁reject ed", + "▁m emor", + "▁me mor", + "▁mem or", + "▁memo r", + "or ts", + "ort s", + "▁ap plies", + "▁appl ies", + "▁L anguage", + "▁ Language", + "spec ific", + "achuset ts", + "HA ND", + "H AND", + "▁R oute", + "▁Ro ute", + "▁Rou te", + "▁ Route", + "mark et", + "mar ket", + "▁K y", + "▁p ose", + "▁pos e", + "▁po se", + "▁ pose", + "AC HE", + "ACH E", + "po ll", + "pol l", + "p oll", + "▁r ocks", + "▁ro cks", + "▁rock s", + "bo ne", + "bon e", + "b one", + "▁D IS", + "▁DI S", + "▁ DIS", + "W atch", + "▁sm iling", + "ри о", + "Mon th", + "Mont h", + "▁e fter", + "con struct", + "const ruct", + "▁b ands", + "▁band s", + "▁ban ds", + "▁ bands", + "▁collabor ation", + "ни ми", + "ним и", + "gl as", + "g las", + "▁v y", + "▁ vy", + "▁eng agement", + "▁engage ment", + "__ )", + "_ _)", + "▁w ings", + "▁win gs", + "▁wing s", + "ки м", + "к им", + "net je", + "at iva", + "ati va", + "ativ a", + "▁Du ke", + "ле е", + "▁With in", + "▁d ove", + "▁do ve", + "▁c b", + "▁ cb", + "ye rs", + "yer s", + "y ers", + "po w", + "p ow", + "[ (", + "▁evalu ate", + "▁eval uate", + "Point s", + "▁р і", + "▁ рі", + "od igd", + "odi gd", + "on omy", + "ono my", + "onom y", + "▁Ill inois", + "▁T yp", + "▁Ty p", + "▁ Typ", + "▁coord inates", + "▁coordinate s", + "pis ode", + "uck ed", + "uc ked", + "▁f lav", + "▁fl av", + "▁br ands", + "▁brand s", + "▁cal endar", + "▁ calendar", + "Li b", + "L ib", + "▁uit gen", + "▁t ale", + "▁tal e", + "▁ta le", + "▁brief ly", + "▁m ic", + "▁mi c", + "▁ mic", + "RE SS", + "RES S", + "▁sp äter", + "▁integr ated", + "▁integrate d", + "▁cook ies", + "▁cookie s", + "▁uitgen odigd", + "▁P riv", + "▁Pr iv", + "▁Pri v", + "▁ Priv", + "▁phen omen", + "▁vo egen", + "Su pp", + "Sup p", + "S upp", + "▁re fers", + "▁ref ers", + "▁refer s", + "па д", + "▁Cl inton", + "▁Clin ton", + "▁ass ignment", + "▁assign ment", + "in als", + "ina ls", + "inal s", + "i nals", + "▁a sym", + "▁as ym", + "cy cle", + "cycl e", + "c ycle", + "▁And erson", + "▁Anders on", + "▁b inding", + "▁bin ding", + "▁bind ing", + "▁ binding", + "ri que", + "r ique", + "hi nd", + "hin d", + "h ind", + "▁be half", + "▁beh alf", + "▁F le", + "▁Fl e", + "▁break s", + "▁bre aks", + "▁so ap", + "▁ soap", + "ва р", + "в ар", + "▁v ä", + "▁ vä", + "▁c rying", + "▁cr ying", + "▁cry ing", + "▁ →", + "▁m sm", + "▁ms m", + "▁ msm", + "▁bo ots", + "▁boot s", + "ow ing", + "owi ng", + "o wing", + "▁b ell", + "▁be ll", + "▁bel l", + "▁ bell", + "su ite", + "suit e", + "▁Bund es", + "▁Bun des", + "Y ear", + "nd ef", + "nde f", + "n def", + "O ther", + "▁go ogle", + "▁goog le", + "▁ google", + "EN CE", + "ENC E", + "WE R", + "W ER", + "Le s", + "L es", + "Sh ared", + "Share d", + "▁E D", + "▁ ED", + "IF T", + "I FT", + "▁flo ating", + "▁float ing", + "ý m", + "{} ,", + "{ },", + "Bin ary", + "B inary", + "▁ro ce", + "ra j", + "r aj", + "▁be werken", + "B F", + "▁H ur", + "▁Hu r", + "ce n", + "c en", + "▁e re", + "▁er e", + "▁ ere", + "▁c amb", + "▁cam b", + "▁ca mb", + "▁Pak istan", + "▁great ly", + "▁log ging", + "▁ logging", + "/ .", + "Ten sor", + "T ensor", + "▁op ens", + "▁open s", + "▁ opens", + "▁R io", + "▁klik ken", + "▁sc ulpt", + "ap ore", + "apor e", + "w x", + "▁N ich", + "▁Nic h", + "▁Ni ch", + "na n", + "n an", + "▁inj ured", + "com pare", + "comp are", + "compar e", + "th a", + "t ha", + "Sam ple", + "S ample", + "Sh ell", + "She ll", + "S hell", + "▁comm ander", + "▁command er", + "▁re ceiver", + "▁rece iver", + "▁receive r", + "▁h opes", + "▁hope s", + "▁hop es", + "▁ho pes", + "▁b yl", + "▁by l", + "▁pro xy", + "▁pr oxy", + "▁ proxy", + "▁g all", + "▁gal l", + "▁ga ll", + "get Id", + "▁B ab", + "▁Ba b", + "fe ld", + "fel d", + "f eld", + "▁\" _", + "▁H ab", + "▁Ha b", + "sim ple", + "▁execut ed", + "▁execute d", + "▁a te", + "▁at e", + "▁ ate", + "▁an imation", + "▁anim ation", + "▁ animation", + "▁in hab", + "▁бо ль", + "▁r outer", + "▁ro uter", + "▁rout er", + "▁route r", + "▁rou ter", + "▁ router", + "▁gl ob", + "▁glo b", + "▁ glob", + "Ge plaatst", + "▁begin netje", + "▁K ur", + "▁Ku r", + "▁Х а", + "al igned", + "align ed", + "▁cert ificate", + "▁ Å", + ".) .", + ". ).", + "▁s oll", + "▁so ll", + "▁sol l", + "▁Im port", + "▁Imp ort", + "▁ Import", + "ре ди", + "ред и", + "р еди", + "▁pand emic", + "▁n ic", + "▁ni c", + "▁ nic", + "v ä", + "▁G ree", + "▁Gr ee", + "▁Gre e", + "▁S ay", + "▁Sa y", + "▁д і", + "▁ ді", + "▁N um", + "▁Nu m", + "▁ Num", + "▁rough ly", + "▁des pués", + "▁ ​", + "▁spec ify", + "Map per", + "lic ht", + "li cht", + "lich t", + "l icht", + "▁th umb", + "▁ thumb", + "wi e", + "w ie", + "▁un likely", + "▁unlike ly", + "▁ unlikely", + "▁E dd", + "▁Ed d", + "He y", + "H ey", + "▁O pt", + "▁Op t", + "▁ Opt", + "B LOCK", + "во р", + "в ор", + "▁ ×", + "▁b a", + "▁ ba", + "▁period s", + "▁title s", + "▁tit les", + "Me d", + "M ed", + "▁f on", + "▁fo n", + "▁ fon", + "▁b ast", + "▁bas t", + "▁ba st", + "▁ bast", + "▁F orest", + "▁For est", + "▁Fore st", + "▁Fo rest", + "▁ №", + "on ds", + "ond s", + "▁f al", + "▁fa l", + "▁g esch", + "▁ge sch", + "▁ges ch", + "▁ gesch", + "dir ection", + "di rection", + "direct ion", + "dire ction", + "d irection", + "IF Y", + "▁L A", + "▁ LA", + "▁( ((", + "▁(( (", + "▁ (((", + "GT H", + "G TH", + "it udes", + "itude s", + "itu des", + "itud es", + "▁dest ruction", + "▁destruct ion", + "▁J a", + "▁s take", + "▁st ake", + "▁sta ke", + "iffer ent", + "iffe rent", + "▁ident ical", + "▁f og", + "▁fo g", + "▁R eb", + "▁Re b", + "▁ Reb", + "ски е", + "сту п", + "ja x", + "j ax", + "▁M ars", + "▁Mar s", + "▁Ma rs", + "▁hist oric", + "▁histor ic", + "▁V o", + "▁ Vo", + "▁entre pre", + "▁t ension", + "▁tens ion", + "▁W HERE", + "▁WH ERE", + "▁WHE RE", + "▁Phil adelphia", + "Count er", + "Co unter", + "C ounter", + "▁fr ames", + "▁frame s", + "▁fra mes", + "▁fram es", + "▁ frames", + "▁m uy", + "▁mu y", + "e j", + "ö t", + "e u", + "▁че лове", + "PRO C", + "PR OC", + "▁res olved", + "▁resolve d", + "▁ resolved", + "▁t ape", + "▁tap e", + "▁ta pe", + "ци он", + "▁sing ular", + "▁person nel", + "▁M un", + "▁Mu n", + "▁O cc", + "▁ Occ", + "▁scal ar", + "▁ scalar", + "de ss", + "des s", + "d ess", + "▁c able", + "▁cab le", + "▁ca ble", + "be ing", + "b eing", + "▁J enn", + "▁Je nn", + "▁Jen n", + "▁er st", + "▁ers t", + "▁ erst", + "Action s", + "Act ions", + "A ctions", + "Env ironment", + "vi a", + "v ia", + "▁strugg ling", + "▁D VD", + "wh e", + "w he", + "▁throw ing", + "▁thr owing", + "▁thro wing", + "Bound s", + "B ounds", + "▁M D", + "▁ MD", + "▁\" ../", + "▁\". ./", + "▁satisf y", + "▁Color ado", + "▁Act ive", + "▁Activ e", + "▁ Active", + "Task s", + "<>( );", + "<> ();", + "< >();", + "▁sl ipped", + "▁slip ped", + "▁po ison", + "▁poi son", + "z b", + "Dis patch", + "war ning", + "warn ing", + "w arning", + "▁ult imate", + "p icture", + "ex pression", + "exp ression", + "expr ession", + "express ion", + "▁T alk", + "▁Tal k", + "▁f lick", + "▁fl ick", + "▁rais ing", + "▁ra ising", + "▁ raising", + "▁trans actions", + "▁transaction s", + "▁gl ance", + "▁g ri", + "▁gr i", + "▁п рез", + "▁пре з", + "se lection", + "sel ection", + "select ion", + "s election", + "њ а", + "en dl", + "end l", + "▁A bb", + "▁Ab b", + "▁b old", + "▁bo ld", + "▁bol d", + "▁ bold", + "▁maint ained", + "▁maintain ed", + "Ex ists", + "▁encour aged", + "▁encourage d", + "Qu al", + "Q ual", + "▁ess ere", + "▁h ired", + "▁hi red", + "▁hire d", + "let ter", + "lett er", + "lette r", + "it ches", + "itch es", + "oth ers", + "other s", + "othe rs", + "o thers", + "▁w oj", + "▁wo j", + "▁inj uries", + "▁d il", + "▁di l", + "ex ecut", + "exec ut", + "▁Ste el", + "▁G arden", + "▁Gar den", + "▁Gard en", + "з я", + "\\, \\", + "\\ ,\\", + "▁An gel", + "▁Ang el", + "pr im", + "p rim", + ">: ]<", + "g b", + "pe at", + "in te", + "int e", + "i nte", + "▁ap olog", + "▁reg ulations", + "▁regul ations", + "▁regulation s", + "S rc", + "k h", + "Up load", + "U pload", + "ma pping", + "map ping", + "m apping", + "▁p resents", + "▁pres ents", + "▁present s", + "▁po etry", + "▁poet ry", + "▁st ops", + "▁stop s", + "▁sto ps", + "▁T ol", + "▁To l", + "▁t ower", + "▁to wer", + "▁tow er", + "▁O UT", + "▁ OUT", + "Th ank", + "Than k", + "▁organ ic", + "▁d rei", + "▁dr ei", + "▁dre i", + "▁p ound", + "▁po und", + "▁pou nd", + "cent ury", + "▁mod ules", + "▁module s", + "▁ modules", + "▁д ере", + "▁де ре", + "▁w orn", + "▁wor n", + "▁wo rn", + "▁par ad", + "▁para d", + "▁pa rad", + "▁C os", + "▁Co s", + "fi c", + "f ic", + "▁бе з", + "▁Jim my", + "▁l ands", + "▁land s", + "▁lan ds", + "▁ lands", + "▁min ist", + "▁mini st", + "vs pace", + "v space", + "▁light ing", + "▁n aked", + "▁na ked", + "▁design er", + "▁St ream", + "▁Stre am", + "▁ Stream", + "TM P", + "T MP", + "Cent er", + "C enter", + "resent ation", + "ON T", + "O NT", + "▁e rs", + "▁er s", + "▁ ers", + "▁measure ment", + "▁mus cles", + "▁muscle s", + "▁I gn", + "▁ Ign", + "▁C OM", + "▁CO M", + "▁ COM", + "▁f ru", + "▁fr u", + "▁gen re", + "▁al pha", + "▁ alpha", + "▁ret irement", + "▁retire ment", + "▁G on", + "▁Go n", + "ő l", + "cont ents", + "content s", + "▁he aling", + "▁heal ing", + "▁s ido", + "▁si do", + "▁sid o", + "incip al", + "Per mission", + "Perm ission", + "ра к", + "▁G ordon", + "▁Gor don", + "▁R ank", + "▁Ran k", + "▁ Rank", + "▁Aut om", + "▁Au tom", + "▁Auto m", + "▁ Autom", + "Con structor", + "Construct or", + "wi ki", + "wik i", + "w iki", + "▁concern ing", + "▁concer ning", + "riz ona", + "▁var iant", + "▁vari ant", + "▁ variant", + "▁arr anged", + "▁arrang ed", + "▁arrange d", + "▁S pr", + "▁Sp r", + "BP ACK", + "B PACK", + "Tim estamp", + "re store", + "rest ore", + "aw are", + "awa re", + "a ware", + "▁Ob serv", + "▁ Observ", + "▁S V", + "▁ SV", + "ip p", + "i pp", + "▁Execut ive", + "▁col leg", + "▁coll eg", + "▁colle g", + "▁explicit ly", + "wr itten", + "writ ten", + "▁K ön", + "▁Kö n", + "ir us", + "i rus", + "▁H old", + "▁Hol d", + "▁Ho ld", + "▁P ract", + "▁Pr act", + "▁Pra ct", + "Char acter", + "▁red istribute", + "uer to", + "▁Stud ent", + "▁ Student", + "▁el der", + "▁D op", + "▁Do p", + "v p", + "▁H ub", + "▁Hu b", + "▁ Hub", + "▁gr ounds", + "▁ground s", + "▁R y", + "▁sign als", + "▁sig nals", + "▁signal s", + "▁g ifts", + "▁gift s", + "▁streng then", + "▁strength en", + "▁L yn", + "▁Ly n", + "com mun", + "comm un", + "▁на й", + "▁fin ance", + "▁financ e", + "no c", + "n oc", + "he lm", + "hel m", + "h elm", + "▁c uts", + "▁cut s", + "▁cu ts", + "▁advent ure", + "▁R ic", + "▁intellect ual", + "▁Out put", + "▁ Output", + "▁aw k", + "▁ awk", + "▁concentr ation", + "▁guid ance", + "Buf f", + "Bu ff", + "B uff", + "▁f illing", + "▁fil ling", + "▁fill ing", + "▁reg ul", + "▁del icious", + "([ ]", + "( []", + "ши х", + "▁t ons", + "▁to ns", + "▁ton s", + "▁ tons", + "act ivity", + "activ ity", + "G P", + "LO B", + "L OB", + "st adt", + "sta dt", + "stad t", + "ta l", + "t al", + "▁im g", + "▁i mg", + "▁ img", + "▁r ush", + "▁ru sh", + "▁rus h", + "att ice", + "atti ce", + "▁p ok", + "▁po k", + "st eps", + "ste ps", + "step s", + "▁l id", + "▁li d", + "▁D NA", + "B rowser", + "▁lad ies", + "▁an nées", + "▁ann ées", + "▁resc ue", + "av ity", + "avi ty", + "ro ck", + "roc k", + "r ock", + "▁glass es", + "▁B ey", + "▁Be y", + ")} $", + ") }$", + "de tail", + "det ail", + "▁d és", + "▁dé s", + "ta x", + "t ax", + "▁favour ite", + "▁prec ision", + "▁con oc", + "▁co noc", + "M s", + "▁N ative", + "▁Nat ive", + "▁ Native", + "▁P il", + "▁Pi l", + "Input Stream", + "or p", + "o rp", + "▁P ap", + "▁Pa p", + "▁p icking", + "▁pick ing", + "▁pic king", + "ip h", + "i ph", + "Load ing", + "Lo ading", + "▁pr iest", + "▁pri est", + "H ook", + "▁p ist", + "▁pi st", + "▁U ne", + "▁Un e", + "▁ Une", + "% ,", + "▁b il", + "▁bi l", + "▁ bil", + "▁conserv ative", + "ev al", + "eva l", + "e val", + "ik ing", + "iki ng", + "i king", + "'} ,", + "' },", + "▁sa uce", + "▁sau ce", + "▁D ue", + "▁Du e", + "as sen", + "ass en", + "asse n", + "▁occasion ally", + "▁occasional ly", + "▁Д ж", + "un known", + "unk nown", + "DE D", + "D ED", + "▁d rum", + "▁dr um", + "▁dru m", + "▁d ub", + "▁du b", + "AT URE", + "us age", + "usa ge", + "get Type", + "re ply", + "rep ly", + "▁strateg ic", + "▁k ap", + "▁ka p", + "▁ kap", + "de sign", + "des ign", + "date time", + "dat etime", + "▁P rim", + "▁Pr im", + "▁Pri m", + "▁ Prim", + "Ma ster", + "M aster", + "▁Cor ps", + "▁consider able", + "▁T u", + "▁ ла", + "▁t ous", + "▁to us", + "▁tou s", + "▁c lar", + "▁cl ar", + "▁po em", + "al bum", + "] *", + "lo aded", + "load ed", + "▁travel ing", + "▁trav eling", + "вы е", + "▁F err", + "▁Fe rr", + "▁Fer r", + "▁p harm", + "▁ph arm", + "ab i", + "a bi", + "▁} \\", + "▁ }\\", + "col lect", + "coll ect", + "▁B our", + "▁Bo ur", + "▁Bou r", + "O C", + "▁measure ments", + "▁measurement s", + "▁Profess ional", + "▁s ensor", + "▁sens or", + "▁sen sor", + "▁ sensor", + "ut sche", + "uts che", + "utsch e", + "▁dem anded", + "▁demand ed", + "▁accompan ied", + "▁p rend", + "▁pre nd", + "▁pr end", + "▁enc oding", + "▁ encoding", + "▁Gesch ichte", + "▁m ig", + "▁mi g", + "▁G ib", + "▁Gi b", + "▁Re ich", + "▁m yster", + "▁my ster", + "▁myst er", + "▁M ock", + "▁Mo ck", + "▁ Mock", + "▁phys ically", + "▁physical ly", + "▁B au", + "▁Ba u", + "▁S ingle", + "▁Sing le", + "▁Sin gle", + "▁ Single", + "▁man aging", + "▁K il", + "▁Ki l", + "▁Tem ple", + "▁Temp le", + "▁l ev", + "▁le v", + "▁ lev", + "▁l í", + "CP U", + "C PU", + "▁Prem ier", + "▁G ive", + "▁Gi ve", + "ir i", + "i ri", + "N V", + "▁A I", + "▁ AI", + "▁f p", + "▁ fp", + "лекс анд", + "▁t ant", + "▁tan t", + "▁ta nt", + "▁f ot", + "▁fo t", + "Null able", + "▁gu ards", + "▁guard s", + "On ce", + "▁ch amber", + "▁cha mber", + "fil m", + "fi lm", + "▁b ias", + "▁bi as", + "▁ bias", + "▁T ai", + "▁Ta i", + "ins ic", + "insi c", + "▁m l", + "▁ ml", + "▁K a", + "ва л", + "в ал", + "▁S ER", + "▁SE R", + "▁ SER", + "▁Some one", + "}} _{", + "}}_ {", + "} }_{", + "Fix ed", + "F ixed", + "▁b ent", + "▁be nt", + "▁ben t", + "▁pro hib", + "▁b id", + "▁bi d", + "▁ bid", + "▁fe wer", + "▁few er", + "к ры", + "▁l ugar", + "▁lug ar", + "▁lu gar", + "▁de serve", + "▁des erve", + "ss l", + "s sl", + "▁c fg", + "▁cf g", + "▁ cfg", + "re ck", + "rec k", + "▁st ability", + "▁stabil ity", + "▁stab ility", + "re size", + "res ize", + "▁assert That", + "Tr igger", + "▁ста нов", + "▁стан ов", + "▁ станов", + "pl ugins", + "plugin s", + "plug ins", + "▁l ets", + "▁le ts", + "▁let s", + "▁ lets", + "хі д", + "х ід", + "▁La ura", + "▁Lau ra", + "не р", + "н ер", + "▁br ut", + "▁bru t", + "▁F I", + "▁ FI", + "is ons", + "ison s", + "iso ns", + "▁d yn", + "▁dy n", + "▁ dyn", + "ic her", + "ich er", + "iche r", + "i cher", + "ray ed", + "▁frequ ent", + "▁jed och", + "▁Mar ine", + "st rings", + "str ings", + "string s", + "▁U til", + "▁Ut il", + "▁ Util", + "▁b os", + "▁bo s", + "Mu s", + "M us", + "▁Portug al", + "Str ategy", + "▁по се", + "▁пос е", + "▁sl ice", + "▁slic e", + "▁ slice", + "▁ins ight", + "▁w idget", + "▁wid get", + "▁ widget", + "▁gén éral", + "message s", + "m essages", + "▁H u", + "▁requ irement", + "▁require ment", + "Si de", + "S ide", + "empl ates", + "emplate s", + "▁cer emony", + "▁ceremon y", + "▁phys ics", + "▁grad uate", + "▁gradu ate", + "▁ graduate", + "par a", + "pa ra", + "p ara", + "▁pre serv", + "▁pres erv", + "▁sh ops", + "▁shop s", + "▁ shops", + "ze k", + "z ek", + "▁u b", + "▁ ub", + "pre pare", + "▁O il", + "▁f ib", + "▁fi b", + "▁run time", + "▁ runtime", + "▁h ogy", + "▁ho gy", + "Warn ing", + "War ning", + "W arning", + "▁Con vert", + "▁ Convert", + "bour ne", + "▁emer ged", + "▁emerge d", + "▁Д и", + "ight h", + "igh th", + "gu ard", + "ka l", + "k al", + "valid ation", + "ên cia", + "ê ncia", + "▁dr inks", + "▁drink s", + "the orem", + "H R", + "ie v", + "i ev", + "ploy ee", + "Us age", + "▁с пе", + "▁сп е", + "▁ спе", + "dis patch", + "disp atch", + "▁inst antly", + "▁instant ly", + "ob i", + "o bi", + "▁just ify", + "▁N ev", + "▁Ne v", + "▁я вля", + "ag ra", + "agr a", + "a gra", + "▁trans mission", + "▁transm ission", + "fl y", + "f ly", + "; ' ;", + "> ';", + "▁cou sin", + "▁cous in", + "create Element", + "Co uld", + "C ould", + "▁cap ac", + "▁p ause", + "▁pa use", + "▁paus e", + "▁ pause", + "Array List", + "kt e", + "k te", + "ord ered", + "order ed", + "▁sh aking", + "▁sha king", + "label s", + "lab els", + "▁redu cing", + "вы х", + "US ED", + "USE D", + "U SED", + "▁v oting", + "▁vo ting", + "▁vot ing", + "▁Min istry", + "▁M ig", + "▁Mi g", + "▁C hen", + "▁Ch en", + "▁Che n", + "▁ac company", + "▁accompan y", + "▁accomp any", + "ul le", + "ull e", + "u lle", + "▁g a", + "▁ ga", + "▁equ ipped", + "▁equip ped", + "▁n un", + "▁nu n", + "Be t", + "B et", + "▁lic ensed", + "▁license d", + "AR CH", + "F N", + "▁eng ines", + "▁engine s", + "▁s ter", + "▁st er", + "▁ste r", + "▁ ster", + "▁loc ale", + "▁local e", + "▁ locale", + "▁в ъ", + "lin ks", + "link s", + "l inks", + "▁Cap ital", + "▁al ien", + "▁ali en", + "W r", + "р ъ", + "Car t", + "C art", + "▁Mark eting", + "▁Market ing", + "▁R T", + "▁ RT", + "File Name", + "▁t i", + "▁ ti", + "ij i", + "i ji", + "▁vers us", + "li ve", + "liv e", + "l ive", + "Sy m", + "S ym", + "ko r", + "k or", + "▁e mission", + "▁em ission", + "um m", + "u mm", + "yc z", + "y cz", + "▁clim bed", + "▁climb ed", + "▁plus ieurs", + "к ри", + "ya r", + "y ar", + "os ten", + "ost en", + "o sten", + "▁u sb", + "▁us b", + "▁ usb", + "▁cross ing", + "▁pol ynom", + "▁poly nom", + "▁rem oval", + "▁Ad ams", + "▁Adam s", + "▁i hre", + "▁ih re", + "▁ihr e", + "an den", + "and en", + "ande n", + "▁Ben j", + "▁P hill", + "▁Ph ill", + "▁Phil l", + "▁wound ed", + "▁Cast le", + "▁Cas tle", + "bi ld", + "bil d", + "b ild", + "An notation", + "Process or", + "▁t in", + "▁ti n", + "fo lg", + "fol g", + "▁Stud ents", + "▁Student s", + "▁Mex ican", + "▁administr ative", + "IL ED", + "ILE D", + "I LED", + "▁con qu", + "▁che er", + "▁C es", + "▁Ce s", + "B ecause", + "▁J uni", + "▁Jun i", + "▁Ju ni", + "▁en contr", + "av i", + "a vi", + "V I", + "ak u", + "a ku", + "▁T on", + "▁To n", + "▁sm oking", + "▁b ay", + "▁ba y", + "work s", + "wor ks", + "а т", + "at tered", + "att ered", + "atter ed", + "atte red", + "▁Bo olean", + "▁ Boolean", + "▁B alt", + "▁Ba lt", + "▁Bal t", + "de fer", + "def er", + "path y", + "pat hy", + "pa thy", + "A h", + "▁a kt", + "▁ak t", + "▁ akt", + "▁gover nor", + "▁govern or", + "P ad", + "▁si sters", + "▁sister s", + "▁sist ers", + "La t", + "L at", + "▁re vel", + "▁r evel", + "▁rev el", + "▁reve l", + "▁S Y", + "▁ SY", + "it os", + "ito s", + "i tos", + "▁fil ters", + "▁filter s", + "▁ filters", + "Ch unk", + "con sum", + "cons um", + "▁rem oving", + "▁H err", + "▁He rr", + "▁Her r", + "▁gener ator", + "▁ generator", + "▁C ra", + "▁Cr a", + "▁far mers", + "▁farm ers", + "▁farmer s", + "▁Mem bers", + "▁Member s", + "▁ Members", + "▁over come", + "▁C in", + "▁Ci n", + "ig keit", + "cri ptions", + "cription s", + "cript ions", + "Test s", + "Te sts", + "T ests", + "▁к лу", + "▁sh ake", + "▁sha ke", + "▁ shake", + "▁y y", + "▁ yy", + "pl acement", + "place ment", + "plac ement", + "▁a wards", + "▁aw ards", + "▁award s", + "▁epis odes", + "▁episode s", + "▁Bl ood", + "▁Blo od", + "▁bul let", + "▁bull et", + "▁ bullet", + "▁v iene", + "▁vi ene", + "▁vie ne", + "▁Fin ancial", + "F uture", + "▁r ou", + "▁ro u", + "▁ rou", + "▁bi ologie", + "▁use State", + "ia ni", + "ian i", + "i ani", + "pie ce", + "p iece", + "▁spe aker", + "▁speak er", + "▁re fr", + "▁ref r", + "AR K", + "▁M IT", + "▁MI T", + "▁ MIT", + "▁T an", + "▁Ta n", + "▁B ased", + "▁Bas ed", + "▁Base d", + "▁Ba sed", + "▁ Based", + "▁cult iv", + "▁hung ry", + "▁A y", + "▁H ey", + "▁He y", + "▁ Hey", + "▁excit ement", + "ibr aries", + "Hi t", + "H it", + "▁E nde", + "▁En de", + "▁End e", + "N G", + "FI L", + "F IL", + ".\" )", + ". \")", + "F amily", + "in ery", + "ine ry", + "iner y", + "ne cess", + "ve lope", + "vel ope", + "velop e", + "▁B ot", + "▁Bo t", + "▁ Bot", + "port er", + "por ter", + "porte r", + "p orter", + "▁cl imb", + "▁clim b", + "▁E li", + "▁El i", + "ur ent", + "ure nt", + "uren t", + "u rent", + "▁mist akes", + "▁mistake s", + "áb an", + "á ban", + "mark s", + "mar ks", + "m arks", + "pk t", + "p kt", + "L ibrary", + "st ed", + "ste d", + "s ted", + "ublic e", + "ubl ice", + "▁Administr ation", + "▁Admin istration", + "▁sh apes", + "▁shape s", + "▁sha pes", + "пу бли", + "Go d", + "G od", + "in nen", + "inn en", + "ко ло", + "к оло", + "<< <<", + "ib e", + "i be", + "ê s", + "▁С ША", + "▁Fore ign", + "▁ Foreign", + "▁Marg aret", + "▁g ene", + "▁gen e", + "▁ge ne", + "▁dist urb", + "▁т ер", + "▁те р", + "▁ тер", + "▁on Click", + "▁Engine ering", + "▁stop ping", + "▁sto pping", + "▁restr ictions", + "▁restrict ions", + "▁restriction s", + ", *", + "BU F", + "▁sh adows", + "▁shadow s", + "hc i", + "h ci", + "▁Christ ians", + "▁Christian s", + "▁f ence", + "▁fen ce", + "▁lux ury", + "ak h", + "a kh", + "co ord", + "▁invest igate", + "▁investig ate", + "▁convent ional", + "▁convention al", + "\" —", + "▁vis its", + "▁visit s", + "is é", + "▁S ac", + "▁Sa c", + "class Name", + "▁Psy ch", + "▁ref lected", + "▁reflect ed", + "▁п ло", + "▁ пло", + "▁V ice", + "▁Vi ce", + "▁Vic e", + "ła w", + "ł aw", + "________ ________", + "▁W olf", + "▁Wol f", + "re nte", + "ren te", + "rent e", + "r ente", + "▁Ch ampion", + "▁sim ulation", + "es ota", + "eso ta", + "▁S oon", + "▁So on", + "▁C el", + "▁Ce l", + "▁the ories", + "▁S TR", + "▁ST R", + "▁ STR", + "▁collect ive", + "▁coord inate", + "query Selector", + "em ed", + "eme d", + "e med", + "B reak", + "▁g ef", + "▁ge f", + "▁electric ity", + "▁gather ing", + "at ers", + "ate rs", + "ater s", + "a ters", + "ex per", + "exp er", + "▁R oma", + "▁Rom a", + "▁Ro ma", + "▁Co oper", + "SY MBOL", + "v d", + "ivers ary", + "ain es", + "ai nes", + "aine s", + "a ines", + "▁G rad", + "▁Gr ad", + "▁Gra d", + "▁ Grad", + "▁independ ence", + "wo h", + "w oh", + "▁con sequence", + "▁consequ ence", + "▁convers ations", + "▁conversation s", + "▁R ou", + "▁Ro u", + "▁and ere", + "▁ander e", + "▁System s", + "га р", + "г ар", + "▁mo ist", + "▁mois t", + "fl u", + "f lu", + "ці я", + "ни ш", + "▁r ode", + "▁ro de", + "▁rod e", + "▁p erd", + "▁per d", + "▁pe rd", + "▁s zer", + "▁sz er", + "▁fl ood", + "▁flo od", + "▁in tim", + "▁int im", + "std err", + "▁ref lection", + "▁reflect ion", + "Sc an", + "S can", + "▁dis aster", + "ake spe", + "akes pe", + "▁In valid", + "▁ Invalid", + "▁hum or", + "▁Fried rich", + "▁suggest ions", + "▁suggestion s", + "uv ud", + "De lay", + "Del ay", + "br ief", + "b rief", + "▁и с", + "▁ ис", + "gl ied", + "fa s", + "f as", + "▁S mart", + "▁Sm art", + "▁m edi", + "▁me di", + "▁med i", + "▁ medi", + "sd k", + "s dk", + "▁se us", + "▁seu s", + "▁A rizona", + "▁innoc ent", + "War n", + "W arn", + "ac ious", + "aci ous", + "acio us", + "▁Mos cow", + "▁c aps", + "▁cap s", + "▁ca ps", + "▁ caps", + "Dele gate", + "▁dram atic", + "bo oks", + "book s", + "▁sh ore", + "▁ shore", + "uk i", + "u ki", + "▁Russ ell", + "▁cor relation", + "▁corre lation", + "▁correl ation", + "He lp", + "Hel p", + "▁pub blic", + "zy m", + "z ym", + "com b", + "co mb", + "c omb", + "E Y", + "LEN GTH", + "▁M ün", + "▁_ .", + "▁ _.", + "▁f erm", + "▁fe rm", + "▁fer m", + "▁I an", + "▁St udio", + "▁Stud io", + "▁aff airs", + "▁affair s", + "lo s", + "l os", + "Rule s", + "R ules", + "run ning", + "r unning", + "▁Post ed", + "▁Po sted", + "▁Pos ted", + "P ixel", + "▁d ancing", + "▁dan cing", + "▁agree ments", + "▁agre ements", + "▁agreement s", + "▁P ic", + "▁Pi c", + "an cia", + "anc ia", + "a ncia", + "▁m á", + "ation Token", + "des criptor", + "▁C arter", + "▁Car ter", + "▁Cart er", + "Re lease", + "Rele ase", + "**** ********", + "******** ****", + "****** ******", + "▁out standing", + "ch anges", + "change s", + "chan ges", + "AR RAY", + "▁Bar bara", + "▁Barb ara", + "▁nur se", + "▁nurs e", + "( \r", + "▁Dou glas", + "▁Doug las", + "▁nu cle", + "▁nuc le", + "ou ri", + "our i", + "o uri", + "▁St yle", + "▁ Style", + "av o", + "a vo", + "▁pain ful", + "▁s lic", + "▁sl ic", + "▁sein em", + "▁seine m", + "▁sei nem", + "SUP PORT", + "og ene", + "ogen e", + "oge ne", + "▁sat ell", + "ta gon", + "tag on", + "t agon", + "▁coll apse", + "▁ collapse", + "ve lle", + "vel le", + "v elle", + "MO N", + "M ON", + "augh ters", + "aught ers", + "aughter s", + "▁threat ened", + "▁Il legal", + "▁desper ate", + "st rict", + "str ict", + "stri ct", + "ru s", + "r us", + "сти ту", + "\\\" :", + "\\ \":", + "▁conf lic", + "down load", + "at os", + "ato s", + "a tos", + "▁Pos ition", + "▁ Position", + ".* ;", + ". *;", + "▁the ater", + "▁ple asant", + "▁C ette", + "▁Sing apore", + "he et", + "▁p ir", + "▁pi r", + "▁ac quis", + "▁acqu is", + "▁на зва", + "те ля", + "тел я", + "▁rec ru", + "же ния", + "ё л", + "вер сите", + "▁res pective", + "▁respect ive", + "▁t unnel", + "▁tun nel", + "▁tunn el", + "▁De an", + "D u", + "▁un cle", + "▁unc le", + "▁off ensive", + "co lo", + "col o", + "c olo", + "▁Un like", + "se ries", + "ser ies", + "serie s", + "s eries", + "▁A rn", + "▁Ar n", + "min ute", + "▁des criptor", + "▁ descriptor", + "▁st ones", + "▁stone s", + "▁sto nes", + "IC ATION", + "▁P ad", + "▁Pa d", + "▁ Pad", + "▁i Phone", + "e i", + "▁fant asy", + "▁Kore an", + "▁Korea n", + "\" }", + "▁or th", + "▁ orth", + "hal ten", + "halt en", + "de ep", + "▁K ay", + "▁Ka y", + "requ ency", + "▁du ties", + "▁dut ies", + "aw t", + "a wt", + "▁ne arest", + "▁near est", + "▁dis order", + "ст ру", + "▁Ch ile", + "▁Chi le", + "▁s eq", + "▁se q", + "▁ seq", + "▁transport ation", + "O O", + "▁D ez", + "▁De z", + "ij u", + "i ju", + "▁Result s", + "▁ Results", + "je d", + "j ed", + "iv el", + "ive l", + "i vel", + "HO ST", + "H OST", + "▁ €", + "▁ Î", + "▁c hin", + "▁ch in", + "▁chi n", + "▁m att", + "▁mat t", + "▁ma tt", + "▁v oted", + "▁vo ted", + "▁vote d", + "▁vot ed", + "▁ge hör", + "▁s ue", + "▁su e", + "▁leg acy", + "в ся", + "SO URCE", + "W ORK", + "it is", + "iti s", + "▁$ |", + "▁о бо", + "▁об о", + "▁n r", + "▁ nr", + "▁T amb", + "▁Ta mb", + "▁Tam b", + "▁sn ap", + "▁ snap", + "▁im pressed", + "▁imp ressed", + "▁impress ed", + "▁depos it", + "▁d ivid", + "▁di vid", + "▁div id", + "Seg ment", + "▁к ар", + "▁ка р", + "▁ кар", + "▁G as", + "▁Ga s", + "▁cr imes", + "▁crim es", + "▁crime s", + "▁cri mes", + "▁ins ult", + "▁H um", + "▁Hu m", + "▁bound ed", + "▁k icked", + "▁kick ed", + "▁М у", + "▁| \\", + "▁ |\\", + "ad ded", + "add ed", + "Pro du", + "P rodu", + "▁. /", + "▁ ./", + "▁awk ward", + "▁К ра", + "▁ ї", + "▁CON TR", + "▁be im", + "▁bei m", + "▁place holder", + "▁ placeholder", + "sp i", + "s pi", + "▁B ei", + "▁Be i", + "▁P f", + "ient es", + "ien tes", + "iente s", + "i entes", + "dis k", + "di sk", + "d isk", + "bl k", + "ne o", + "it arian", + "ita rian", + "itar ian", + "▁c ogn", + "▁co gn", + "▁s out", + "▁so ut", + "▁sou t", + "▁tr ash", + "▁tra sh", + "▁tras h", + "▁R ab", + "▁Ra b", + "▁dec line", + "▁decl ine", + "ta t", + "t at", + "▁comb ine", + "▁T ot", + "▁To t", + "▁dr ops", + "▁dro ps", + "▁drop s", + "Time s", + "Tim es", + "T imes", + "ched uler", + "chedul er", + "▁govern ments", + "▁government s", + "Te x", + "T ex", + "▁U sed", + "▁Us ed", + "▁Use d", + "▁ Used", + "за н", + "з ан", + "▁p d", + "▁ pd", + "ме т", + "м ет", + "▁&= &", + "▁N ag", + "▁Na g", + "▁до л", + "▁ дол", + "▁Al ways", + "rt c", + "r tc", + "ск е", + "с ке", + "▁perform ances", + "▁performance s", + "rupt ed", + "rup ted", + "▁д ва", + "▁man agers", + "▁manager s", + "▁manage rs", + "▁P itt", + "▁Pi tt", + "▁myst ery", + "▁myster y", + "▁set tle", + "▁sett le", + "ul se", + "uls e", + "cr oss", + "cro ss", + "c ross", + "quest ion", + "as ha", + "ash a", + "a sha", + "se ed", + "see d", + "s eed", + "ur able", + "ura ble", + "Fin al", + "Fi nal", + "F inal", + "++ ++", + "input s", + "▁back up", + "▁ backup", + "▁Le arning", + "▁Lear ning", + "▁Learn ing", + "▁* ,", + "▁ *,", + "lo go", + "log o", + "l ogo", + "▁se inen", + "▁sein en", + "▁seine n", + "▁sei nen", + "▁vulner able", + "direct ory", + "i ë", + "▁friend ship", + "▁friends hip", + "t u", + "▁V ec", + "▁Ve c", + "▁ Vec", + "rif ice", + "rific e", + "▁б ра", + "▁ бра", + "▁inv olve", + "▁invol ve", + "TO N", + "T ON", + "▁cor rid", + "se par", + "sep ar", + "Dest roy", + "▁j ul", + "▁ju l", + "▁inequ ality", + "▁a in", + "▁ai n", + "▁ ain", + "he x", + "h ex", + "▁w ider", + "▁wide r", + "▁wid er", + "те ли", + "тел и", + "▁j ack", + "▁ja ck", + "▁ jack", + "▁qu ot", + "▁ quot", + "▁G len", + "▁Gl en", + "▁Gle n", + "init ely", + "ih ood", + "i hood", + "▁wa ist", + "▁Man chester", + "reg ular", + "▁( &", + "▁ (&", + "▁mass es", + "▁mas ses", + "▁DE FAULT", + "▁ DEFAULT", + "▁ch airs", + "▁chair s", + "▁cha irs", + "▁F ast", + "▁Fa st", + "▁ Fast", + "▁c itt", + "▁cit t", + "▁ci tt", + "_{ {\\", + "_ {{\\", + "o a", + "▁$ \\{", + "▁$\\ {", + "▁se eds", + "▁see ds", + "▁seed s", + "▁A ld", + "▁Al d", + "▁B att", + "▁Ba tt", + "▁Bat t", + "fa b", + "f ab", + "▁democr acy", + "DT O", + "D TO", + "▁H ij", + "▁Hi j", + "PT R", + "P TR", + "N a", + "▁Har vard", + "si d", + "s id", + "Pr ed", + "Pre d", + "P red", + "fer s", + "fe rs", + "f ers", + "▁s pare", + "▁sp are", + "AM P", + "A MP", + "▁g roupe", + "▁group e", + "▁s ender", + "▁se nder", + "▁send er", + "▁sen der", + "▁ sender", + "▁Christ opher", + "▁prison ers", + "▁prisoner s", + "▁K er", + "▁Ke r", + "▁C rist", + "▁Cr ist", + "▁Cris t", + "▁A LL", + "▁AL L", + "▁ ALL", + "ri ce", + "ric e", + "r ice", + "▁an tes", + "▁ant es", + "▁ante s", + "▁ antes", + "nat ural", + "▁Su san", + "▁Sus an", + "▁J uli", + "▁Jul i", + "▁Ju li", + "▁di ab", + "▁dia b", + "ix on", + "ic ator", + "ica tor", + "▁flex ible", + "▁re serve", + "▁res erve", + "▁reserv e", + "Cont ains", + "▁H il", + "▁Hi l", + "▁I sa", + "▁Is a", + "▁town s", + "▁tow ns", + "G S", + "▁T rad", + "▁Tr ad", + "▁Tra d", + "▁L ock", + "▁Loc k", + "▁Lo ck", + "▁ Lock", + "▁G rund", + "▁Gr und", + "▁Gru nd", + "▁crit icism", + "▁critic ism", + "н ю", + "▁c ă", + "▁polit ician", + "st able", + "sta ble", + "s table", + "Ac cept", + "Sum mary", + "▁tamb ém", + "▁també m", + "}^ {-", + "}^{ -", + "} ^{-", + "▁I M", + "▁ IM", + "id al", + "ida l", + "i dal", + "мо р", + "м ор", + "Bl ue", + "GRO UP", + "▁term inal", + "▁termin al", + "▁complex ity", + "▁loc ally", + "▁local ly", + "DO WN", + "▁N ear", + "▁Ne ar", + "Dep th", + "▁p ole", + "▁pol e", + "▁po le", + "▁e quality", + "▁equ ality", + "▁equal ity", + "Si te", + "S ite", + "▁is instance", + "Sp eed", + "Spe ed", + "S peed", + "ip pi", + "ipp i", + ", &", + "▁E nc", + "▁En c", + "▁ Enc", + "ще н", + "щ ен", + "▁m ater", + "▁mat er", + "▁ma ter", + "▁mate r", + "▁sl aves", + "▁slave s", + "▁sla ves", + "AC TION", + "ACT ION", + "A CTION", + "usal em", + "usa lem", + "▁h az", + "▁ha z", + "▁Be at", + "▁w rest", + "▁wr est", + "▁l lam", + "▁ll am", + "In s", + "I ns", + "ми на", + "▁бу в", + "▁Fr ame", + "▁Fra me", + "▁ Frame", + "us hes", + "ush es", + "▁virtual ly", + "▁virt ually", + "▁P erm", + "▁Per m", + "▁Pe rm", + "▁ Perm", + "▁we ights", + "▁weight s", + "▁weigh ts", + "▁ weights", + "▁ll vm", + "▁ llvm", + "▁c ave", + "▁ca ve", + "▁cav e", + "st ates", + "state s", + "stat es", + "sta tes", + "DM A", + "D MA", + "el lt", + "ell t", + "if act", + "ifa ct", + "i fact", + "v endor", + "▁E mma", + "▁Em ma", + "Loc ale", + "Local e", + "▁S ET", + "▁SE T", + "▁ SET", + "▁ge ometry", + "▁ geometry", + "St yles", + "Style s", + "▁Ref eree", + "▁Refer ee", + "▁we it", + "fi ca", + "fic a", + "f ica", + "▁a ds", + "▁ad s", + "▁ ads", + "gr ay", + "gra y", + "g ray", + "▁B urg", + "▁Bur g", + "▁Bu rg", + "ion a", + "io na", + "i ona", + "dag ger", + "d agger", + "▁Jan uar", + "де й", + "д ей", + "ister schaft", + "pp o", + "p po", + "oid s", + "oi ds", + "o ids", + "▁dé part", + "Sh ader", + "▁con straint", + "▁constr aint", + "▁ constraint", + "Se cret", + "Sec ret", + "▁P eters", + "▁Pe ters", + "▁Peter s", + "▁Pet ers", + "▁Pete rs", + "▁ey eb", + "▁eye b", + "▁m esh", + "▁me sh", + "▁mes h", + "▁ mesh", + "▁c ookie", + "▁cook ie", + "▁ cookie", + "▁P ick", + "▁Pic k", + "▁Pi ck", + "▁n ick", + "▁ni ck", + "▁nic k", + "▁ nick", + "by e", + "b ye", + "▁sav ings", + "▁saving s", + "Tr y", + "T ry", + "py thon", + "▁p atri", + "▁pat ri", + "▁pa tri", + "▁mult ip", + "▁multi p", + "▁mul tip", + "▁ multip", + "▁k inda", + "▁kind a", + "▁kin da", + "▁' _", + "▁ '_", + "▁Fr anz", + "▁Fran z", + "▁cl oth", + "▁clo th", + "зу льта", + "▁fle et", + "▁human ity", + "re sa", + "res a", + "r esa", + "bl ob", + "blo b", + "▁T X", + "▁ TX", + "▁B uch", + "▁Bu ch", + "▁Buc h", + "▁L ond", + "▁Lo nd", + "▁val ley", + "▁m urm", + "▁mur m", + "▁mu rm", + "▁T rade", + "▁Tr ade", + "▁Tra de", + "▁Trad e", + "line width", + "▁e special", + "▁espec ial", + "up per", + "upp er", + "▁h osp", + "▁ho sp", + "▁t anto", + "▁tan to", + "▁tant o", + "▁old est", + "▁ol dest", + "▁R oose", + "▁Ro ose", + "▁h itting", + "▁hit ting", + "do g", + "d og", + "ov i", + "o vi", + "}, \r", + "} ,\r", + "▁compat ible", + "▁ compatible", + "▁We bsite", + "▁Web site", + "po ch", + "p och", + "▁B ag", + "▁Ba g", + "▁ Bag", + "▁accompl ish", + "▁accomp lish", + "Ch rist", + "as set", + "ass et", + "asse t", + "▁U ntil", + "▁Un til", + "▁ Until", + "▁g eld", + "▁ge ld", + "▁gel d", + "List en", + "Li sten", + "L isten", + "S B", + "Set up", + "ic ia", + "ici a", + "i cia", + "▁l um", + "▁lu m", + "▁jan vier", + "PA GE", + "P AGE", + "▁N u", + "/ \"", + "▁divor ce", + "Ex ecute", + "Execut e", + "Exec ute", + "De pend", + "Dep end", + "▁Scott ish", + "▁T s", + "ru ppe", + "rup pe", + "▁ref use", + "▁Ok tober", + "ij k", + "i jk", + "▁A my", + "▁Am y", + "▁di min", + "▁dim in", + "▁g ross", + "▁gr oss", + "▁gro ss", + "▁t rat", + "▁tr at", + "▁tra t", + "is ible", + "isi ble", + "mix er", + "m ixer", + "▁aut res", + "▁au tres", + "▁autre s", + "▁ autres", + "▁ne at", + "▁ot ros", + "▁otro s", + "Vo id", + "V oid", + "▁sc hol", + "▁sch ol", + "▁Wal ker", + "▁Walk er", + "▁t ube", + "▁tu be", + "▁tub e", + "olog ists", + "ologist s", + "▁г руп", + "▁гру п", + "▁h aben", + "▁hab en", + "▁ha ben", + "ub er", + "ube r", + "u ber", + "ACT IVE", + "▁Att endance", + "▁о п", + "▁bl ade", + "opl us", + "o plus", + "▁Or iginal", + "▁Origin al", + "▁ Original", + "▁manufact urer", + "as z", + "a sz", + "ât e", + "â te", + "re r", + "r er", + "▁J son", + "▁ Json", + "▁succeed ed", + "uff le", + "▁b acked", + "▁back ed", + "es ian", + "esi an", + "ti ck", + "t ick", + "Ex ternal", + "▁X IX", + "▁XI X", + "▁he arts", + "▁heart s", + "▁hear ts", + "▁По сле", + "ol u", + "o lu", + "▁ле т", + "▁ лет", + "VI CE", + "V ICE", + "ár io", + "á rio", + "▁fr aud", + "▁fra ud", + "ed u", + "e du", + "Pr imary", + "Prim ary", + "▁g aming", + "▁gam ing", + "▁ga ming", + "▁p lt", + "▁pl t", + "ig ator", + "iga tor", + "IE S", + "I ES", + "Comp iler", + "▁mon ument", + "ag em", + "age m", + "a gem", + "▁R ain", + "▁Ra in", + "▁mo ins", + "ok u", + "o ku", + "os ex", + "ose x", + "o sex", + "▁K ansas", + "▁gep ublice", + "▁J oy", + "▁Jo y", + "Sc ene", + "▁king dom", + "ri ces", + "ric es", + "rice s", + "r ices", + "▁ju in", + "▁uncomfort able", + "▁M oney", + "▁Mon ey", + "▁Mo ney", + "ob b", + "o bb", + "ex pl", + "exp l", + "str cmp", + "▁d read", + "▁dr ead", + "▁dre ad", + "rit ion", + "r ition", + "▁C hi", + "▁Ch i", + "▁demonstr ated", + "▁demonstrate d", + "▁vert ices", + "ч о", + "▁C ulture", + "▁ Culture", + "F X", + "D ictionary", + "▁D ru", + "▁Dr u", + "tr m", + "t rm", + "▁ex amine", + "▁exam ine", + "▁the rap", + "▁ther ap", + "i ème", + "ми ни", + "▁produ ces", + "▁produce s", + "▁photograph s", + "▁thread s", + "▁ threads", + "▁M I", + "▁ MI", + "▁extraord inary", + "ски м", + "ск им", + "с ким", + "▁gepublice erd", + "▁Pol and", + "▁Po land", + "▁guarante ed", + "▁guarantee d", + "R G", + "os c", + "o sc", + "ал и", + "а ли", + "▁те х", + "err no", + "sc ience", + "if fs", + "iff s", + "▁T am", + "▁Ta m", + "▁B eth", + "▁Be th", + "▁Bet h", + "▁Tr avel", + "▁Tra vel", + "▁trans late", + "▁transl ate", + "▁ translate", + "ch é", + "▁l ing", + "▁li ng", + "▁lin g", + "▁ ling", + "▁bel ongs", + "▁belong s", + "▁elect rical", + "▁electric al", + "en sk", + "ens k", + "▁Com pet", + "▁Comp et", + "c g", + "V C", + "to pic", + "top ic", + "t opic", + "▁pre sum", + "▁pres um", + "ве та", + "вет а", + "▁approxim ation", + "▁approx imation", + "▁g rim", + "▁gr im", + "▁gri m", + "▁И з", + "_{ (", + "_ {(", + "ви н", + "в ин", + "ut ion", + "uti on", + "ow ych", + "owy ch", + "å g", + "ster reich", + "▁character istic", + "om ing", + "omin g", + "omi ng", + "o ming", + "▁/* !", + "▁ /*!", + "▁pr ize", + "▁pri ze", + "▁Minn esota", + "te d", + "t ed", + "ц ы", + "▁O m", + "▁ Om", + "▁ind ices", + "▁indic es", + "▁ indices", + "▁s tem", + "▁st em", + "▁ste m", + "re gon", + "reg on", + "ни че", + "▁Sal v", + "▁Sa lv", + "és e", + "é se", + "▁a ged", + "▁ag ed", + "▁age d", + "▁ aged", + "▁P ast", + "▁Pa st", + "▁Pas t", + "▁intern ation", + "▁V ic", + "▁Vi c", + "▁res ume", + "▁ resume", + "akespe are", + "▁est ado", + "▁esta do", + "▁estad o", + "▁ab ilities", + "▁ abilities", + "▁b row", + "▁br ow", + "▁bro w", + "▁N FL", + "▁tr ends", + "▁trend s", + "▁tren ds", + "▁Aust in", + "▁L IMIT", + "▁LI MIT", + "▁ LIMIT", + "▁K or", + "▁Ko r", + "▁f olk", + "▁fol k", + "▁ folk", + "▁w ard", + "▁war d", + "▁wa rd", + "▁ ward", + "▁n est", + "▁ne st", + "▁Jun ior", + "▁Juni or", + "▁maint aining", + "▁maintain ing", + "P ub", + "OB JECT", + "▁blo ody", + "▁blood y", + "▁s j", + "▁d type", + "▁dt ype", + "▁ dtype", + "Pan e", + "P ane", + "▁b acter", + "▁grad ually", + "▁gradu ally", + "m r", + "Te am", + "▁ind icating", + "▁indic ating", + "▁decre ase", + "te k", + "t ek", + "▁Re present", + "▁Rep resent", + "▁develop ers", + "▁developer s", + "Gu id", + "Gui d", + "G uid", + "▁D iet", + "▁Die t", + "▁Di et", + "▁re tr", + "▁r etr", + "▁ret r", + "Nav igation", + "es i", + "e si", + "▁l azy", + "▁la zy", + "Stand ard", + "E r", + "A W", + "▁Ét ats", + "▁ass ured", + "▁assure d", + "Sa n", + "S an", + "▁And re", + "▁Andr e", + "’ ,", + "fa ng", + "fan g", + "f ang", + "ér ation", + "▁indust ries", + "▁in con", + "▁inc on", + "Em it", + "E mit", + "▁г де", + "▁ret riev", + "▁retr iev", + "en i", + "e ni", + "▁Tur key", + "▁Turk ey", + "iz ers", + "ize rs", + "izer s", + "An gle", + "Ang le", + "▁o c", + "▁ oc", + "▁pal m", + "▁pa lm", + "▁s tan", + "▁st an", + "▁sta n", + "▁ stan", + "ль но", + "▁C SS", + "▁CS S", + "▁ CSS", + "▁fr ances", + "▁franc es", + "▁g rin", + "▁gr in", + "▁gri n", + "▁tiem po", + "▁P rix", + "▁Pr ix", + "▁Pri x", + "]) .", + "] ).", + "▁de put", + "▁dep ut", + "▁P in", + "▁Pi n", + "▁ Pin", + "▁si xt", + "▁six t", + "▁predict ed", + "▁pred icted", + "az ure", + "azu re", + "▁Mo tor", + "▁Mot or", + "▁i hm", + "▁ih m", + "▁man us", + "ap os", + "a pos", + "▁instr uments", + "▁instrument s", + "▁co unts", + "▁coun ts", + "▁count s", + "▁aim ed", + "▁ai med", + "▁ aimed", + "pro fit", + "prof it", + "▁d ok", + "▁do k", + "об ра", + "о бра", + "▁e stud", + "▁est ud", + "ie sz", + "ies z", + "i esz", + "▁p iss", + "▁pi ss", + "▁in aug", + "▁vo ters", + "▁vote rs", + "▁vot ers", + "▁pack ages", + "▁package s", + "▁ packages", + "▁c ute", + "▁cut e", + "▁cu te", + "▁f itness", + "▁fit ness", + "▁l eurs", + "▁le urs", + "▁leur s", + "▁s orted", + "▁sort ed", + "▁sor ted", + "ph ant", + "pha nt", + "phan t", + "OP T", + "O PT", + "▁z ip", + "▁ zip", + "se ason", + "sea son", + "em i", + "e mi", + "enc oding", + "wo n", + "w on", + "el ect", + "ele ct", + "e lect", + "▁t ooth", + "▁to oth", + "▁too th", + "▁up coming", + "▁G raham", + "▁Gra ham", + "nu t", + "n ut", + "▁Ar k", + "äl t", + "ä lt", + "▁prec ious", + "ag le", + "a gle", + "né e", + "n ée", + "ни ца", + "ниц а", + "ar is", + "ari s", + "a ris", + "▁p ile", + "▁pi le", + "▁pil e", + "co le", + "col e", + "c ole", + "▁W ITH", + "▁WIT H", + "▁ WITH", + "rou ting", + "r outing", + "▁* **", + "▁** *", + "▁ ***", + "Appe arance", + "ll vm", + "▁O liver", + "▁Ol iver", + "▁P L", + "▁ PL", + "if ndef", + "et zt", + "etz t", + "sk iego", + "ski ego", + "▁p on", + "▁po n", + "▁ pon", + "AR GET", + "ARG ET", + "k ö", + "al led", + "all ed", + "alle d", + "▁= \\", + "▁ =\\", + "su re", + "sur e", + "s ure", + "mat ches", + "match es", + "▁temper atures", + "▁temperature s", + "SE L", + "S EL", + "▁cl one", + "▁clo ne", + "▁ clone", + "▁el ler", + "▁elle r", + "▁ell er", + "▁ eller", + "er na", + "ern a", + "▁п оло", + "▁по ло", + "▁пол о", + "Man agement", + "comp any", + "▁l un", + "▁lu n", + "▁stre aming", + "▁stream ing", + "▁N i", + "▁s í", + "Cont act", + "▁C redit", + "▁Cr edit", + "▁Cre dit", + "▁O ak", + "▁пред став", + "rad ius", + "cl i", + "c li", + "IE NT", + "I ENT", + "▁Lu cy", + "▁Luc y", + "▁calcul ation", + "▁calc ulation", + "▁p ixel", + "▁ pixel", + "▁m ul", + "▁mu l", + "▁ mul", + "▁out comes", + "▁outcome s", + "▁cent ers", + "▁center s", + "▁res idence", + "▁resid ence", + "Con straint", + "▁pre serve", + "▁pres erve", + "▁preserv e", + "pe on", + "uf fix", + "uff ix", + "▁Rober ts", + "▁Robert s", + "▁Rob erts", + "▁pro mot", + "▁pr omot", + "▁prom ot", + "? !", + "bal ance", + "▁cour ts", + "▁court s", + "▁dis g", + "▁di sg", + "PR INT", + "PRI NT", + "▁и х", + "el fare", + "elf are", + "▁ret reat", + "▁А в", + "Co st", + "C ost", + "al so", + "als o", + "▁F ür", + "▁Mär z", + "DI O", + "D IO", + "▁b ez", + "▁be z", + "▁ bez", + "AUT H", + "AU TH", + "De n", + "D en", + "▁a tom", + "▁at om", + "▁ atom", + "▁r oman", + "▁ro man", + "▁rom an", + "▁P el", + "▁Pe l", + "▁Roose velt", + "▁Pl ant", + "▁Plan t", + "Cont ents", + "Content s", + "▁Bet ween", + "▁cou pling", + "▁coup ling", + "str ucture", + "struct ure", + "▁Mar shall", + "▁Mars hall", + "▁Marshal l", + "▁Care er", + "▁rail way", + "▁B ureau", + "▁Bur eau", + "▁poss ibilities", + "▁k or", + "▁ko r", + "▁ kor", + "){ \r", + ") {\r", + "mer o", + "me ro", + "m ero", + "mo v", + "m ov", + "анг л", + "AI N", + "A IN", + "mu nd", + "mun d", + "m und", + "let te", + "lett e", + "l ette", + "▁sum mar", + "▁describ ing", + "▁N AS", + "▁NA S", + "▁E mb", + "▁Em b", + "▁ Emb", + "Inst ruction", + "li est", + "lie st", + "l iest", + "▁S ig", + "▁Si g", + "▁ Sig", + "Bi ll", + "B ill", + "▁v erd", + "▁ver d", + "▁ve rd", + "pl ant", + "plan t", + "▁galax ies", + "\"] )", + "\" ])", + "▁Py Object", + "▁ PyObject", + "▁G y", + "▁m ě", + "▁organ isation", + "▁organis ation", + "He r", + "H er", + "Se p", + "S ep", + "oc om", + "oco m", + "o com", + "▁S ame", + "▁Sam e", + "▁Sa me", + "▁ Same", + "▁b ite", + "▁bit e", + "▁bi te", + "▁Se attle", + "зы ва", + "Ob server", + "Observ er", + "’ .", + "▁m orph", + "▁mor ph", + "ur ches", + "urch es", + "al ph", + "re ement", + "ree ment", + "con sin", + "cons in", + "^ -", + "▁d ann", + "▁da nn", + "▁dan n", + "trans late", + "transl ate", + "ви х", + "Re act", + "▁c ats", + "▁cat s", + "▁ca ts", + "▁b rew", + "▁br ew", + "▁bre w", + "▁ brew", + "▁d s", + "▁ ds", + "▁cir cles", + "▁circ les", + "▁circle s", + "▁d rift", + "▁dr ift", + "▁dri ft", + "ag ma", + "▁Val ent", + "PI N", + "P IN", + "AR M", + "A RM", + "▁sur viv", + "▁surv iv", + "al in", + "ali n", + "a lin", + "Pr ef", + "Pre f", + "P ref", + "friend ly", + "▁uncertain ty", + "▁f d", + "▁ fd", + "▁engine er", + "Be n", + "B en", + "ic ular", + "i cular", + "or est", + "ore st", + "ores t", + "o rest", + "▁hor izontal", + "▁horizon tal", + "▁ horizontal", + "UT C", + "U TC", + "text rm", + "tex trm", + "Li ve", + "L ive", + "Sc ore", + "S core", + "▁Germ ans", + "▁German s", + "▁Ger mans", + "di stance", + "dist ance", + "d istance", + "ut i", + "u ti", + "▁é qu", + "▁ équ", + "▁numer ical", + "▁re ass", + "Act iv", + "▁c od", + "▁co d", + "▁ cod", + "bul let", + "en sing", + "ens ing", + "▁G em", + "▁Ge m", + "▁nav igation", + "▁navig ation", + "▁ navigation", + "add Class", + "▁simultane ously", + "ви й", + "▁йо го", + "▁й ого", + "▁H ö", + "▁har sh", + "prec ated", + "p recated", + "С СР", + "▁Equ ip", + "ad get", + "▁T YPE", + "▁ TYPE", + "▁m g", + "▁ mg", + "IG H", + "▁v in", + "▁vi n", + "▁ vin", + "▁fin dings", + "▁find ings", + "▁finding s", + "iv an", + "iva n", + "i van", + "▁pos session", + "▁poss ession", + "▁possess ion", + "▁т ого", + "▁то го", + "▁ того", + "▁par sed", + "▁parse d", + "▁ parsed", + "ri ors", + "rior s", + "rio rs", + "r iors", + "zeich net", + "ни ков", + "ник ов", + "Work er", + "▁en ables", + "▁enable s", + "▁( $\\", + "▁($ \\", + "▁C opy", + "▁Co py", + "▁Cop y", + "▁ Copy", + "▁orient ation", + "ст ре", + "с тре", + "▁Ind ians", + "▁India ns", + "▁Indian s", + "▁G ary", + "▁Gar y", + "▁Ga ry", + "▁Ins urance", + "is an", + "isa n", + "i san", + "Ch at", + "C hat", + "▁com un", + "▁co mun", + "▁co ron", + "▁cor on", + "ографи я", + "up dated", + "update d", + "▁И н", + "The se", + "Th ese", + "SE C", + "S EC", + "▁boy friend", + "Di agnostics", + "Hi nt", + "H int", + "mu l", + "m ul", + "▁in ode", + "▁i node", + "▁ inode", + "x A", + "ef t", + "e ft", + "OP TION", + "OPT ION", + "un ct", + "unc t", + "an non", + "ann on", + "anno n", + "EN S", + "E NS", + "st rip", + "str ip", + "stri p", + "▁enthus i", + "▁W hit", + "▁Wh it", + "▁Ф и", + "au de", + "aud e", + "a ude", + "▁disag ree", + "▁sn apped", + "▁snap ped", + "Ph ys", + "▁S yn", + "▁Sy n", + "▁s our", + "▁so ur", + "▁sou r", + "▁L ux", + "▁Lu x", + "ug ar", + "uga r", + "u gar", + "til e", + "ti le", + "t ile", + "▁in fection", + "▁inf ection", + "▁infect ion", + "▁F eb", + "▁Fe b", + "▁C hem", + "▁Ch em", + "▁Che m", + "data set", + "dat aset", + "ch ts", + "cht s", + "D ynamic", + "▁с ред", + "▁qu een", + "▁que en", + "work er", + "wor ker", + "sw ap", + "▁tim estamp", + "▁ timestamp", + "▁In tegr", + "▁Int egr", + "▁ Integr", + "▁inter views", + "▁interview s", + "su ch", + "s uch", + "▁l aughter", + "▁laugh ter", + "pro f", + "pr of", + "▁B ird", + "▁Bi rd", + "▁Bir d", + "( |", + "â n", + "▁g ra", + "▁gr a", + "▁ gra", + "& =", + "ze ns", + "zen s", + "z ens", + "get Message", + "▁O st", + "▁Os t", + "▁g ab", + "▁ga b", + "▁mort gage", + "mult icol", + "multi col", + "LE VEL", + "part ition", + "se en", + "see n", + "s een", + "▁dec lar", + "▁decl ar", + "A U", + "▁o x", + "▁ ox", + "▁l igger", + "▁lig ger", + "▁C arm", + "▁Car m", + "▁Ca rm", + "ge me", + "gem e", + "g eme", + "▁Ve gas", + "▁Veg as", + "▁E ug", + "▁Eu g", + "or us", + "o rus", + "▁b rick", + "▁br ick", + "▁as í", + "▁Mag azine", + "HasColumn Type", + "V R", + "lic her", + "li cher", + "lich er", + "liche r", + "l icher", + "▁F uture", + "▁Fut ure", + "▁ Future", + "▁J ug", + "▁Ju g", + "at tan", + "att an", + "atta n", + "con structor", + "construct or", + "V P", + "▁т ур", + "▁ту р", + "▁ тур", + "чи на", + "чин а", + "Comp arator", + "Compar ator", + "▁aut hentic", + "▁mon ster", + "▁trans formed", + "▁transform ed", + "▁firm s", + "▁fir ms", + "F W", + "▁c atalog", + "▁catal og", + "▁ catalog", + "bo ards", + "board s", + "▁dise ases", + "▁disease s", + "▁Benj amin", + "▁hor izon", + "▁Av ailable", + "▁ Available", + "M vc", + "St ud", + "▁l ord", + "▁lo rd", + "▁ lord", + "gen eral", + "gener al", + "па р", + "п ар", + "▁cab inet", + "▁cabin et", + "▁Bas ic", + "▁ Basic", + "Test Case", + "an sk", + "ans k", + "▁S now", + "▁Sn ow", + "ier ten", + "iert en", + "ierte n", + "i erten", + "▁v ocal", + "▁vo cal", + "▁voc al", + "Pad ding", + "P adding", + "ha lt", + "hal t", + "h alt", + "▁Alex and", + "▁Col omb", + "iv amente", + "iva mente", + "▁art ificial", + "▁Atl anta", + "▁m entre", + "▁men tre", + "▁ment re", + "▁est aba", + "▁estab a", + "▁esta ba", + "je kt", + "jek t", + "j ekt", + "▁sle pt", + "▁end less", + "▁endl ess", + "ér o", + "é ro", + "at tery", + "att ery", + "atter y", + "atte ry", + "uu r", + "u ur", + "▁weak ness", + "▁attempt ing", + "BY TE", + "▁found er", + "▁fo under", + "▁fou nder", + "▁sa lv", + "▁sal v", + "▁Medic ine", + "ti d", + "t id", + "▁Sch we", + "▁Schw e", + "ra ction", + "ract ion", + "r action", + "▁ ¿", + "cr ate", + "c rate", + "SER VER", + "▁comp ound", + "▁con ve", + "▁conv e", + "▁c af", + "▁ca f", + "▁hand ful", + "on ne", + "úblic a", + "▁def ensive", + "▁defens ive", + "Al ignment", + "Align ment", + "▁pr éc", + "▁pré c", + "▁signific ance", + "él é", + "é lé", + "ar ta", + "art a", + "Da m", + "D am", + "▁per pet", + "▁c aller", + "▁call er", + "▁cal ler", + "ic ients", + "ici ents", + "icient s", + "ce p", + "c ep", + "▁Mult i", + "▁Mul ti", + "▁ Multi", + "▁st olen", + "▁sto len", + "▁stole n", + "▁focus ing", + "em bed", + "emb ed", + "▁b ree", + "▁br ee", + "▁bre e", + "▁A B", + "▁ AB", + "▁occasion s", + "▁occas ions", + "se a", + "s ea", + "Pro v", + "Pr ov", + "P rov", + "че ние", + "▁C ategory", + "▁ Category", + "▁s q", + "▁ sq", + "▁Ф е", + "V A", + "Di ff", + "D iff", + "Tr i", + "T ri", + "iss ement", + "isse ment", + "▁act ress", + "▁П е", + "▁j ej", + "▁je j", + "▁tw isted", + "▁twist ed", + "▁N icol", + "▁Nic ol", + "▁Ni col", + "▁jun ior", + "▁junio r", + "▁juni or", + "So und", + "S ound", + "▁Bra sil", + "▁Bras il", + "▁ju ice", + "▁> >>", + "▁>> >", + "▁ >>>", + "▁A lb", + "▁Al b", + "▁soft ly", + "▁Mc K", + "▁G ren", + "▁Gr en", + "▁Gre n", + "▁ital iano", + "▁cre atures", + "▁creat ures", + "▁creature s", + "▁res idential", + "▁resident ial", + "▁resid ential", + "▁Inst agram", + "uck s", + "uc ks", + "u cks", + "▁k iller", + "▁kill er", + "▁kil ler", + "▁John ny", + "▁enter prise", + "D to", + "ch estra", + "che stra", + "ches tra", + "chestr a", + "▁T el", + "▁Te l", + "▁Act iv", + "▁ Activ", + "fa ctor", + "fac tor", + "fact or", + "f actor", + "ou st", + "ous t", + "o ust", + "▁vac uum", + "ра л", + "р ал", + "') ->", + "' )->", + "▁L eft", + "▁Le ft", + "▁ Left", + "▁de fect", + "▁def ect", + "▁defe ct", + "▁nine te", + "▁nin ete", + "fa re", + "far e", + "f are", + "▁reg ret", + "▁s har", + "▁sh ar", + "▁sha r", + "ctr ine", + "me sh", + "mes h", + "m esh", + "ci ty", + "cit y", + "c ity", + "ic it", + "ici t", + "i cit", + "▁F em", + "▁Fe m", + "lim ited", + "limit ed", + "ok a", + "o ka", + "!\\ !\\", + "Don ald", + "з но", + "▁pro vision", + "▁prov ision", + "▁discuss ions", + "▁discussion s", + "Dr ag", + "D rag", + "▁In cl", + "▁Inc l", + "Ex it", + "E xit", + "▁A bd", + "▁Ab d", + "st ory", + "sto ry", + "ie ve", + "iev e", + "i eve", + "▁by ł", + "ol ving", + "olv ing", + "woh ner", + "▁gu idelines", + "▁guide lines", + "▁guid elines", + "▁st raw", + "▁str aw", + "▁stra w", + "ü ss", + "▁бу ло", + "▁bur den", + "▁spat ial", + "▁stret ched", + "▁stretch ed", + "▁I nf", + "▁In f", + "▁ Inf", + "▁type def", + "▁typed ef", + "▁ro bot", + "▁rob ot", + "▁D oc", + "▁Do c", + "▁ Doc", + "pl iers", + "plier s", + "wa l", + "w al", + "ca mp", + "cam p", + "c amp", + "▁dif fé", + "▁diff é", + "▁Mc G", + "▁t el", + "▁te l", + "ar ette", + "aret te", + "▁sub sequently", + "▁subsequ ently", + "▁subsequent ly", + "▁h oney", + "▁hon ey", + "▁ho ney", + "FUN C", + "▁establish ment", + "te sy", + "tes y", + "▁któ ry", + "▁се ль", + "▁F O", + "▁ FO", + "▁Is lands", + "▁Island s", + "▁m p", + "▁ mp", + "Scal ar", + "▁Y an", + "▁Ya n", + "ck en", + "cke n", + "c ken", + "▁var iation", + "▁vari ation", + "i ą", + "op tim", + "opt im", + "az or", + "tu ple", + "t uple", + "▁gr avity", + "▁grav ity", + "▁con clude", + "▁concl ude", + "▁col lections", + "▁collection s", + "▁collect ions", + "▁colle ctions", + "és z", + "é sz", + "▁L iver", + "▁Li ver", + "▁Live r", + "▁Liv er", + "▁eth nic", + "comp ile", + "▁p arl", + "▁par l", + "▁pa rl", + "Sur face", + "{ '", + "▁par agraph", + "▁para graph", + "▁ paragraph", + "pos ite", + "po site", + "ít ulo", + "ob a", + "o ba", + "bin ary", + "b inary", + "ro b", + "r ob", + "▁Pe dro", + "▁Ped ro", + "▁f is", + "▁fi s", + "▁Gr ande", + "▁Grand e", + "▁Gran de", + "▁Gra nde", + "od ox", + "odo x", + "▁pos ting", + "▁post ing", + "< !--", + "▁rac ial", + "▁ra cial", + "CO M", + "C OM", + "ё м", + "▁A UT", + "▁AU T", + "▁ AUT", + "▁d ishes", + "▁dis hes", + "▁dish es", + "assert True", + "▁G row", + "▁Gr ow", + "▁Gro w", + "▁sl id", + "▁ju illet", + "сс о", + "с со", + "Run ner", + "Sa l", + "S al", + "Sa me", + "Sam e", + "S ame", + "▁Stud y", + "▁Col onel", + "▁J oin", + "▁Jo in", + "▁ Join", + "ar ms", + "arm s", + "▁l y", + "▁ ly", + "▁co oper", + "▁cur ves", + "▁curve s", + "▁curv es", + "He alth", + "▁M OD", + "▁MO D", + "▁ MOD", + "▁pr imo", + "▁prim o", + "▁pri mo", + "ock ets", + "ocket s", + "multi column", + "multicol umn", + "▁С ан", + "▁Са н", + "▁H unter", + "▁Hun ter", + "▁Hunt er", + "Custom er", + "ot hy", + "oth y", + "o thy", + "Des ign", + "De sign", + "ma ss", + "mas s", + "m ass", + "▁fam ille", + "▁famil le", + "▁fue ron", + "▁fu eron", + "▁fuer on", + "ä m", + "▁head quarters", + "▁d ign", + "▁di gn", + "▁dig n", + "▁Ro bin", + "▁Rob in", + "▁me ets", + "▁meet s", + "▁so it", + "па да", + "пад а", + ")\" );", + ") \");", + "▁w rapper", + "▁wrap per", + "▁ wrapper", + "▁theoret ical", + "▁u d", + "▁ ud", + "pl icity", + "plic ity", + "plicit y", + "▁w p", + "▁ wp", + "▁испо ль", + "▁c amps", + "▁camp s", + "▁cam ps", + "▁A gency", + "▁Ag ency", + "g c", + "hu m", + "h um", + "AT T", + "A TT", + "B tn", + "C ent", + "▁H elen", + "▁He len", + "▁Hel en", + "▁am plit", + "▁ampl it", + "▁Mem orial", + "und ial", + "SH IFT", + "wi k", + "w ik", + "▁Lie utenant", + "VAL ID", + "▁B ath", + "▁Ba th", + "▁Bat h", + "▁Jeff erson", + "▁C ut", + "▁Cu t", + "▁ Cut", + "▁ser vers", + "▁serv ers", + "▁server s", + "▁serve rs", + "▁ servers", + "ly ph", + "▁CO PY", + "▁COP Y", + "▁comput ers", + "▁computer s", + "▁compute rs", + "const ruction", + "construct ion", + "▁P DF", + "▁PD F", + "▁ PDF", + "▁pro tagon", + "▁prot agon", + "▁fore head", + "custom er", + "Un is", + "U nis", + "▁sign ing", + "▁sig ning", + ". ’", + "F etch", + "▁S core", + "▁Sc ore", + "▁ Score", + "hu man", + "hum an", + "h uman", + "▁down town", + "▁downt own", + "In tern", + "Int ern", + "Inter n", + "▁bes ides", + "▁beside s", + "▁д во", + "▁пра ви", + "▁ прави", + "▁c c", + "▁ cc", + "▁D ebug", + "▁De bug", + "▁Deb ug", + "▁ Debug", + "▁Cl ose", + "▁ Close", + "el ihood", + "eli hood", + "▁al gorithms", + "▁algorithm s", + "▁H amb", + "▁Ham b", + "▁Ha mb", + "ч на", + "▁c ust", + "▁cu st", + "▁mo unted", + "▁mount ed", + "par en", + "pa ren", + "pare n", + "p aren", + "▁isol ated", + "▁A gr", + "▁Ag r", + "▁or bit", + "▁orb it", + "print k", + "▁t urb", + "▁tu rb", + "▁tur b", + "▁gru po", + "ми и", + "\"\" \"", + "\" \"\"", + "▁h ills", + "▁hill s", + "ря д", + "▁B od", + "▁Bo d", + "▁об ще", + "est one", + "esto ne", + "eston e", + "e stone", + "▁satisf ying", + "▁satisfy ing", + "▁I van", + "▁Iv an", + "▁associ ate", + "name d", + "na med", + "nam ed", + "n amed", + "oc cup", + "occ up", + "GP IO", + "G PIO", + "hi t", + "h it", + "▁dis tract", + "▁di stract", + "▁dist ract", + "▁bar rel", + "▁barr el", + "▁in variant", + "di d", + "d id", + "▁l ieu", + "▁li eu", + "▁lie u", + "sc ene", + "UN K", + "▁Ont ario", + "▁M ission", + "▁Miss ion", + "zi al", + "z ial", + "▁comp ete", + "▁compet e", + "▁cou ples", + "▁couple s", + "▁coup les", + "SH A", + "S HA", + "▁s ei", + "▁se i", + "▁m igration", + "▁migr ation", + "ac ked", + "ack ed", + "▁b arn", + "▁bar n", + "▁ba rn", + "hal f", + "h alf", + "▁neigh bour", + "▁neighb our", + "ft e", + "f te", + "▁od ds", + "▁odd s", + "▁optim ization", + "▁I C", + "▁ IC", + "▁H end", + "▁He nd", + "▁Hen d", + "pay ment", + "M r", + "') :", + "' ):", + "vo ir", + "v oir", + "▁R ange", + "▁Rang e", + "▁Ran ge", + "▁ Range", + "▁polit icians", + "▁politician s", + "▁K han", + "▁Kh an", + "▁shel ter", + "▁tim ing", + "▁ti ming", + "Create d", + "Creat ed", + "C reated", + "▁sept embre", + "li t", + "l it", + "▁S hel", + "▁She l", + "▁Sh el", + "▁c ouch", + "▁co uch", + "▁cou ch", + "▁d är", + "ult ur", + "▁G iov", + "▁Gi ov", + "ô le", + "RE AM", + "▁O cean", + "▁M B", + "▁ MB", + "▁lie gt", + "▁o v", + "▁ ov", + "▁car pet", + "та р", + "т ар", + "▁го дина", + "▁годи на", + "▁S ão", + "▁о тно", + "▁от но", + "ab ling", + "abl ing", + "a bling", + "in th", + "int h", + "▁purs ue", + "▁Const itution", + "an j", + "▁F BI", + "▁ar row", + "▁arr ow", + "▁ arrow", + "ph ones", + "phone s", + "▁kn ocked", + "▁knock ed", + "▁de com", + "▁dec om", + "ie k", + "i ek", + "ь е", + "St rip", + "Str ip", + "▁V enez", + "▁Ven ez", + "▁Ve nez", + "▁p upp", + "▁pu pp", + "▁pup p", + "bi an", + "bia n", + "b ian", + "▁cot ton", + "h p", + "▁the atre", + "▁accept able", + "cuss ion", + "▁r ounds", + "▁round s", + "▁act ively", + "▁activ ely", + "▁active ly", + "▁among st", + "▁a bc", + "▁ab c", + "▁ abc", + "F M", + "Pop up", + "▁divers ity", + "us z", + "u sz", + "▁employ er", + "spec ially", + "special ly", + "▁sus pected", + "▁suspect ed", + "▁c rypt", + "▁cry pt", + "▁O scar", + "▁Os car", + "no r", + "n or", + "▁bab ies", + "во м", + "▁m undo", + "▁li bert", + "▁lib ert", + "▁liber t", + "S G", + "ah ren", + "ahr en", + "a hren", + "▁magn itude", + "T M", + "' +", + "▁об ъ", + "▁G ust", + "▁Gu st", + "▁gr ain", + "▁gra in", + "мен т", + "м ент", + "to Equal", + "▁m os", + "▁mo s", + "▁ mos", + "▁consist ently", + "▁consistent ly", + "х у", + "▁domin ant", + "Con verter", + "Convert er", + "at able", + "ata ble", + "a table", + "▁J ag", + "▁Ja g", + "scri ptions", + "script ions", + "scription s", + "s criptions", + "x B", + "▁ ©", + "fol der", + "fold er", + "f older", + "▁sub stance", + "▁subst ance", + "▁по с", + "L o", + "BU S", + "B US", + "bas ic", + "us sen", + "uss en", + "▁co ins", + "▁coin s", + ": -", + "▁N elson", + "▁Nel son", + "In ner", + "ograf ía", + "▁ex empl", + "▁exem pl", + "ch g", + "▁sy nd", + "▁syn d", + "dyn amic", + "d ynamic", + "ver ted", + "vert ed", + "▁EV ENT", + "▁ EVENT", + "se ek", + "see k", + "av ier", + "avi er", + "a vier", + "▁p rot", + "▁pro t", + "▁pr ot", + "▁ prot", + "-- ----", + "---- --", + "--- ---", + "----- -", + "- -----", + "▁con vention", + "▁conv ention", + "▁convent ion", + "▁станов ника", + "gl ing", + "g ling", + "hor a", + "ho ra", + "h ora", + "ши й", + "▁wh ilst", + "ser ialize", + "serial ize", + "s erialize", + "▁R ing", + "([ '", + "( ['", + "▁c her", + "▁ch er", + "▁che r", + "▁ cher", + "сь кі", + "▁D anny", + "▁Dan ny", + "▁re aches", + "▁reach es", + "▁el igible", + "▁P arent", + "▁Par ent", + "▁Pa rent", + "▁ Parent", + "▁came ras", + "▁cam eras", + "▁camera s", + "▁discipl ine", + "▁s illy", + "▁sil ly", + "re ts", + "ret s", + "r ets", + "yt ics", + "▁Reg ional", + "▁Region al", + "▁B aby", + "▁Ba by", + "▁Bab y", + "te le", + "t ele", + "WAR NING", + "WARN ING", + "su pp", + "sup p", + "s upp", + "▁refer ring", + "▁mer ch", + "▁merc h", + "ol ves", + "olve s", + "olv es", + "em et", + "eme t", + "e met", + "ck e", + "c ke", + "▁M unicip", + "▁Mun icip", + "Wh ite", + "▁ Ś", + "ri os", + "rio s", + "r ios", + "log ging", + "▁d x", + "▁ dx", + "▁su sp", + "▁sus p", + "ex ternal", + "▁Liber al", + "▁Lib eral", + "▁Init ialize", + "▁Initial ize", + "▁ Initialize", + "▁exhib ition", + "▁exhibit ion", + "▁ext ensions", + "▁extension s", + "▁extens ions", + "▁ extensions", + "ke eper", + "keep er", + "kee per", + "SY S", + "▁J ake", + "▁Ja ke", + "▁Jak e", + "fo oter", + "foot er", + "foo ter", + "▁ph ones", + "▁phone s", + "▁ phones", + "▁real m", + "▁contribut ed", + "▁contribute d", + "ME SS", + "▁For mat", + "▁Form at", + "▁ Format", + "Per iod", + "▁h id", + "▁hi d", + "▁ hid", + "▁me tres", + "▁met res", + "▁D im", + "▁Di m", + "▁ Dim", + "ache lor", + "achel or", + "▁T ak", + "▁Ta k", + "▁ве ли", + "▁g ram", + "▁gr am", + "▁gra m", + "▁ gram", + "▁M Y", + "▁ MY", + "on ders", + "ond ers", + "onder s", + "onde rs", + "'; \r", + "' ;\r", + "▁F ro", + "▁Fr o", + "▁advant ages", + "▁advantage s", + "io v", + "i ov", + "▁she ets", + "▁sheet s", + "ce mbre", + "c embre", + "ž e", + "] \r", + "▁D J", + "subset eq", + "UP DATE", + "▁b locked", + "▁bl ocked", + "▁block ed", + "▁pan els", + "▁pa nels", + "▁panel s", + "E A", + "nd e", + "n de", + "ê t", + "Bu l", + "B ul", + "▁m eters", + "▁me ters", + "▁met ers", + "▁meter s", + "jo ur", + "j our", + "▁rap port", + "▁rapp ort", + "▁J ak", + "▁Ja k", + "▁V AL", + "▁VA L", + "▁ VAL", + "▁p up", + "▁pu p", + "▁k a", + "▁ ka", + "for ced", + "force d", + "▁ав гу", + "ener gy", + "e nergy", + "▁V a", + "not es", + "no tes", + "note s", + "n otes", + "▁relax ed", + "C r", + "id ding", + "idd ing", + "▁def ines", + "▁define s", + "▁defin es", + "▁kiss ed", + "▁inv asion", + "▁invas ion", + "▁sc reens", + "▁screen s", + "C trl", + "▁pass engers", + "▁passenger s", + "▁Х о", + "ation ship", + "ations hip", + "per cent", + "\\ }", + "▁be ating", + "▁beat ing", + "life ray", + "lifer ay", + "▁V M", + "▁ VM", + "▁Gab riel", + "▁g allery", + "▁gall ery", + "▁Л о", + "iv ot", + "ivo t", + "▁r ental", + "▁ren tal", + "▁rent al", + "▁sh ocked", + "▁shock ed", + "▁Ste in", + "▁B h", + "▁ ло", + "Un e", + "U ne", + "ге н", + "г ен", + "▁kom mun", + "an ka", + "ank a", + "▁C ape", + "▁Cap e", + "▁Ca pe", + "Re ady", + "Read y", + "▁к ри", + "▁ кри", + "tr ag", + "tra g", + "t rag", + "Al ign", + "Ali gn", + "▁host ed", + "▁ho sted", + "▁\\ (", + "▁S ession", + "▁ Session", + "ys k", + "y sk", + "Pen ding", + "P ending", + "ellig ence", + "elli gence", + "▁Never theless", + "bit ro", + "bitr o", + "ho lm", + "hol m", + "quir y", + "▁mechan ical", + "▁D é", + "an eous", + "ane ous", + "▁psych ological", + "▁a broad", + "▁ab road", + "▁a voir", + "▁av oir", + "▁separ ation", + "▁sep aration", + "▁Haw ai", + "iej sc", + "▁N ether", + "▁Ne ther", + "▁Net her", + "▁sub tle", + "bi rd", + "b ird", + "▁mark er", + "▁mar ker", + "▁ marker", + "▁со зда", + "ва ла", + "вал а", + "▁Work ing", + "▁Wor king", + "▁h over", + "▁ho ver", + "▁ hover", + "%%%% %%%%", + "▁м ат", + "▁ма т", + "▁ мат", + "▁s oup", + "▁so up", + "▁sou p", + "Al ert", + "ch r", + "c hr", + "▁P CI", + "▁PC I", + "▁ PCI", + "▁m ús", + "ient ras", + "ien tras", + "▁St orage", + "▁Sto rage", + "▁ Storage", + "▁av ailability", + "▁op era", + "▁oper a", + "▁P roduction", + "▁Produ ction", + "▁Product ion", + "ia ne", + "ian e", + "i ane", + "▁Bet ter", + "▁B utton", + "▁But ton", + "▁ Button", + "▁Pe ace", + "▁Mor ris", + "▁s ib", + "▁si b", + "▁f iber", + "▁fi ber", + "▁fib er", + "Int ent", + "▁D esc", + "▁De sc", + "▁Des c", + "▁ Desc", + "ning en", + "n ingen", + "ze j", + "z ej", + "av an", + "ava n", + "a van", + "cover ed", + "cov ered", + "▁s yst", + "▁sy st", + "▁sys t", + "_ +", + "▁орга ни", + "▁Re lig", + "▁Rel ig", + "ци аль", + "▁s pite", + "▁sp ite", + "▁re prés", + "▁~ ~", + "▁ ~~", + "▁to xic", + "▁a pro", + "▁ap ro", + "▁apr o", + "X Y", + "▁tr ips", + "▁tri ps", + "▁trip s", + "▁pl aats", + "▁con vey", + "▁conv ey", + "▁conve y", + "Pr im", + "P rim", + "▁о ста", + "▁ос та", + "▁ост а", + "ok o", + "o ko", + "▁l obby", + "▁lob by", + "▁recommend ations", + "▁recommendation s", + "SP ACE", + "▁overwhel ming", + "ennes see", + "▁ac quire", + "▁acqu ire", + "w m", + "LOB AL", + "▁D EF", + "▁DE F", + "▁ DEF", + "je r", + "j er", + "▁re cur", + "▁rec ur", + "om men", + "omm en", + "▁j og", + "▁jo g", + "▁n ast", + "▁na st", + "▁nas t", + "▁L P", + "▁ LP", + "jo n", + "j on", + "▁w ishes", + "▁wish es", + "▁wis hes", + "▁N ancy", + "▁support ers", + "▁supp orters", + "^{ -\\", + "^{- \\", + "▁T rib", + "▁Tr ib", + "▁Tri b", + "▁ Ä", + "▁disappoint ed", + "▁у ни", + "x D", + "li nt", + "lin t", + "l int", + "I p", + "▁Islam ic", + "än de", + "änd e", + "ä nde", + "end ment", + "dt ype", + "d type", + "▁di gest", + "▁dig est", + "▁Set tings", + "▁Setting s", + "▁ Settings", + "ér a", + "é ra", + "▁aggress ive", + "▁intellig ent", + "eder börd", + "ster dam", + "pc i", + "p ci", + "▁over flow", + "▁ overflow", + "im b", + "i mb", + "re ach", + "rea ch", + "r each", + "cept or", + "cep tor", + "▁yield s", + "▁Se bast", + "▁ut ility", + "▁util ity", + "▁р и", + "▁ ри", + "▁fac ulty", + "▁In ternal", + "▁Intern al", + "▁Inter nal", + "▁ Internal", + "▁attract ed", + "▁attra cted", + "рі в", + "р ів", + "▁mix ing", + "▁R uth", + "▁Ru th", + "▁esc aped", + "▁escape d", + "▁E asy", + "▁dr ain", + "▁r ings", + "▁ring s", + "▁ rings", + "qu ire", + "quir e", + "Av ailable", + "▁ц и", + "▁ ци", + "▁conv ince", + "▁convin ce", + "or sch", + "ors ch", + "ут бо", + "CP P", + "C PP", + "ra ge", + "rag e", + "r age", + "ч і", + "▁p rod", + "▁pro d", + "▁pr od", + "▁ prod", + "▁p ig", + "▁pi g", + "▁C atal", + "▁Cat al", + "▁Ca tal", + "▁al ias", + "▁ali as", + "▁ alias", + "▁че мпи", + "▁чем пи", + "Pl ace", + "P lace", + "▁g orge", + "▁depend ence", + "▁cr uel", + "▁cru el", + "▁ther mal", + "▁therm al", + "ut down", + "ref resh", + "▁re sort", + "▁res ort", + "▁S HA", + "▁SH A", + "▁ SHA", + "ти й", + "fo od", + "foo d", + "f ood", + "▁N ad", + "▁Na d", + "▁pregn ancy", + "▁pro jection", + "▁project ion", + "▁pa ís", + "▁полу чи", + "▁the mes", + "▁them es", + "▁theme s", + "▁fun eral", + "▁cas o", + "▁ca so", + "ле кт", + "лек т", + "Ex tra", + "Ext ra", + "▁t issue", + "▁dr agon", + "▁drag on", + "▁l ig", + "▁li g", + "▁ lig", + "▁n ei", + "▁ne i", + "▁com edy", + "▁come dy", + "▁comed y", + "те м", + "т ем", + "сла в", + "с лав", + "▁pass enger", + "Cl one", + "i ção", + "yg on", + "y gon", + "▁H alf", + "▁Hal f", + "▁la bour", + "▁lab our", + "▁vill ages", + "▁village s", + "▁ві й", + "▁О т", + "▁L isa", + "▁Li sa", + "▁Lis a", + "_ [", + "ba g", + "b ag", + "▁d iver", + "▁di ver", + "▁div er", + "▁dive r", + "▁M L", + "▁ ML", + "▁transl ated", + "▁translate d", + "▁per ò", + "ab ama", + "aba ma", + "▁cas tle", + "▁cast le", + "▁ castle", + "* \\", + "▁reg ia", + "!! !!", + "!!! !", + "! !!!", + "*> (", + "* >(", + "▁Work s", + "▁Wor ks", + "▁N ature", + "▁Nat ure", + "▁Natur e", + "NE L", + "N EL", + "▁P om", + "▁Po m", + "tt a", + "t ta", + "▁Jam ie", + "▁p unch", + "▁pun ch", + "tain ment", + "▁K rieg", + "▁Kr ieg", + "▁restr icted", + "▁restrict ed", + "mob ile", + "m obile", + "▁grand mother", + "Arg uments", + "Argument s", + "▁s inc", + "▁si nc", + "▁sin c", + "▁Mon th", + "▁Mont h", + "▁ Month", + "esc ape", + "e scape", + "▁opt ical", + "▁L ane", + "▁La ne", + "▁Lan e", + "▁Deutsch land", + "▁S aison", + "▁Sa ison", + "▁V irtual", + "▁ Virtual", + "pe z", + "p ez", + "In line", + "ow any", + "owa ny", + "rad io", + "r adio", + "ö ß", + "▁O thers", + "▁Other s", + "MA IN", + "M AIN", + "sc al", + "s cal", + "▁D allas", + "▁Dal las", + "▁an chor", + "▁anc hor", + "▁anch or", + "▁ anchor", + "en cias", + "enc ias", + "encia s", + "enci as", + "▁re porter", + "▁rep orter", + "▁report er", + "▁veget ables", + "▁enforce ment", + "▁Wis consin", + "▁con dem", + "▁cond em", + "▁e b", + "▁ eb", + "▁s its", + "▁sit s", + "▁si ts", + "▁calcul ations", + "▁calculation s", + "▁calc ulations", + "▁\" --", + "▁\"- -", + "ue lle", + "uel le", + "u elle", + "▁tip o", + "▁ti po", + "▁P AR", + "▁PA R", + "▁ PAR", + "co rd", + "cor d", + "c ord", + "▁ро ків", + "ph an", + "pha n", + "p han", + "▁kon nte", + "▁z ap", + "▁za p", + "wr iting", + "writ ing", + "en gu", + "eng u", + "▁pert urb", + "Fac e", + "F ace", + "ag og", + "ago g", + "▁De cl", + "▁Dec l", + "▁ Decl", + "est ly", + "▁War ren", + "▁H ills", + "▁Hill s", + "▁Hil ls", + "▁ref resh", + "▁refr esh", + "▁refres h", + "▁ refresh", + "▁fl ip", + "io p", + "i op", + "▁key board", + "is to", + "ist o", + "i sto", + "▁prom oted", + "▁promote d", + "▁promot ed", + "back s", + "ba cks", + "b acks", + "Enc oding", + "▁ ال", + "▁g min", + "ро б", + "р об", + "▁follow ers", + "▁p epper", + "um ble", + "umb le", + "▁sp ray", + "▁spr ay", + "▁dr ives", + "▁dri ves", + "▁driv es", + "▁drive s", + "P ush", + "cook ie", + "c ookie", + "▁gel dig", + "▁geld ig", + "ig ung", + "igu ng", + "vis it", + "▁at omic", + "▁atom ic", + "▁ atomic", + "▁A thlet", + "▁Ath let", + "▁Or igin", + "▁Ori gin", + "▁ Origin", + "▁H appy", + "▁G ra", + "▁Gr a", + "▁att ribut", + "▁п ов", + "▁по в", + "▁ пов", + "▁n ost", + "▁no st", + "▁nos t", + "▁ nost", + "ur u", + "u ru", + "▁Ne ither", + "▁ma ar", + "ject ions", + "je ctions", + "jection s", + "▁re nov", + "▁ren ov", + "fin ity", + "f inity", + "gener ic", + "init ialize", + "initial ize", + "pgf set", + "▁hyp othes", + "▁ma cro", + "▁mac ro", + "ma ps", + "map s", + "m aps", + "▁f are", + "▁far e", + "▁fa re", + "▁ fare", + "Be st", + "B est", + "uch t", + "uc ht", + "u cht", + "co d", + "c od", + "▁h orm", + "▁hor m", + "▁ho rm", + "▁P oll", + "▁Pol l", + "▁Po ll", + "▁host ing", + "▁Re ading", + "▁Read ing", + "Cert ificate", + "▁и ма", + "▁им а", + "▁C ov", + "▁Co v", + "▁P red", + "▁Pr ed", + "▁Pre d", + "▁ Pred", + "re direct", + "red irect", + "▁l attice", + "▁port folio", + "▁o ven", + "▁ov en", + "▁ oven", + "ie len", + "iel en", + "iele n", + "i elen", + "sub scribe", + "foot note", + "но ю", + "▁mom ento", + "▁moment o", + "▁d ich", + "▁di ch", + "▁dic h", + "▁ent ert", + "▁enter t", + "▁g é", + "▁connect ing", + "▁n acional", + "▁o tt", + "▁ot t", + "▁ ott", + "ні в", + "н ів", + "▁rac ist", + "▁penal ty", + "ül t", + "ü lt", + "▁Israel i", + "▁( †", + "▁desc end", + "▁ос іб", + "▁b elly", + "▁bel ly", + "▁bell y", + "ні сть", + "▁encounter ed", + "T ip", + "▁gu ilt", + "▁d amp", + "▁da mp", + "▁dam p", + "ze ug", + "▁Mem ory", + "▁ Memory", + "Check ed", + "▁Sh akespeare", + "hi ll", + "h ill", + "▁w oke", + "▁wo ke", + "▁sal ary", + "eth eless", + "ethe less", + "e theless", + "▁Т и", + "er de", + "erd e", + "▁He in", + "▁g it", + "▁gi t", + "▁ git", + "=\" \"", + "= \"\"", + "ül l", + "ü ll", + "ge ben", + "geb en", + "g eben", + "Pr es", + "Pre s", + "P res", + "ie val", + "iev al", + "i eval", + "mark er", + "mar ker", + "▁д ан", + "▁да н", + "▁ дан", + "▁oct obre", + "RO L", + "R OL", + "▁jan u", + "▁ja nu", + "▁) :", + "▁ ):", + "br anch", + "▁J erry", + "▁Jer ry", + "ke hr", + "▁contr acts", + "▁contract s", + "▁aff air", + "▁Росси и", + "ja ck", + "j ack", + "AN G", + "A NG", + "▁dro pping", + "▁drop ping", + "▁d ic", + "▁di c", + "sch ool", + "▁Fin land", + "▁d ort", + "▁do rt", + "▁K ings", + "▁King s", + "▁Kin gs", + "▁Arg ument", + "▁ Argument", + "▁Sim ilarly", + "▁Similar ly", + "▁V erm", + "▁Ver m", + "▁Ve rm", + "▁pret end", + "! _", + "łu g", + "ł ug", + "же ння", + "жен ня", + "da ting", + "dat ing", + "d ating", + "cs v", + "c sv", + "▁dialog ue", + "▁dial ogue", + "STR U", + "▁public ly", + "wed ge", + "w edge", + "▁H och", + "▁Ho ch", + "▁spe aks", + "▁speak s", + "▁compens ation", + "an ca", + "anc a", + "text tt", + "▁Fil ter", + "▁ Filter", + "▁part ly", + "▁us eless", + "▁use less", + "▁г у", + "▁ гу", + "▁d eter", + "▁de ter", + "▁det er", + "IE W", + "▁con secut", + "▁cons ecut", + "▁conse cut", + "▁h oly", + "▁hol y", + "▁ho ly", + "▁grad uated", + "▁gradu ated", + "▁graduate d", + "an dal", + "and al", + "anda l", + "ți e", + "ț ie", + "▁W ant", + "▁Wa nt", + "▁Aust ria", + "or den", + "ord en", + "fr ag", + "f rag", + "▁f oo", + "▁fo o", + "▁ foo", + "cl aimed", + "claim ed", + "во е", + "▁not able", + "▁no table", + "▁journal ist", + "▁M ail", + "▁Ma il", + "▁Mai l", + "▁ Mail", + "!( \"", + "! (\"", + "ps e", + "p se", + "▁C lay", + "▁Cl ay", + "iv i", + "i vi", + "▁sc ales", + "▁scale s", + "▁scal es", + "▁er ste", + "▁erst e", + "▁ers te", + "Data Type", + "▁D iam", + "▁Di am", + "í r", + "loc ale", + "local e", + "▁rel uct", + "ien st", + "iens t", + "ast ro", + "astr o", + "act ly", + "я х", + "▁Vill age", + "▁Villa ge", + "▁Vil lage", + "▁d aughters", + "▁daughter s", + "▁manufact urers", + "▁manufacturer s", + "▁print ing", + "▁prin ting", + "ч ка", + "Nd Ex", + "Ch anges", + "Change s", + "▁/ ******/", + "ver tex", + "vert ex", + "▁b rows", + "▁br ows", + "▁bro ws", + "▁brow s", + "▁K ö", + "not ations", + "notation s", + "▁i ls", + "▁il s", + "▁ ils", + "at el", + "ate l", + "C ir", + "▁meaning ful", + "q a", + "▁C old", + "▁Col d", + "▁Co ld", + "ue to", + "u eto", + "you r", + "yo ur", + "y our", + "m f", + "мо в", + "м ов", + "▁Ü ber", + "▁fam ilia", + "▁famil ia", + "▁ste ep", + "▁pres idential", + "▁president ial", + "▁presid ential", + "▁z á", + "▁ zá", + "▁w ars", + "▁war s", + "▁wa rs", + "▁C re", + "▁Cr e", + "▁after wards", + "▁afterward s", + "ha lb", + "hal b", + "▁strugg led", + "▁struggle d", + "Ch art", + "Char t", + "User Id", + "ac ular", + "a cular", + "iv ia", + "ivi a", + "i via", + "▁u gly", + "▁K unst", + "E s", + "▁Q String", + "▁C ow", + "▁Co w", + "Rad ius", + "▁Gr iff", + "▁V as", + "▁Va s", + "HA L", + "H AL", + "Mod ified", + "ra le", + "ral e", + "r ale", + "mem cpy", + "▁в клю", + "▁r s", + "▁ rs", + "▁h alt", + "▁ha lt", + "▁hal t", + "▁ halt", + "▁Miss iss", + "▁h uvud", + "ec a", + "e ca", + "▁Jahrhund ert", + "E urope", + "Sign ature", + "▁grand father", + "▁O regon", + "gu e", + "g ue", + "xy gen", + "fr ames", + "frame s", + "▁hab its", + "▁ha bits", + "▁habit s", + "Support ed", + "Supp orted", + "▁low ered", + "▁lower ed", + "▁rad iation", + "▁radi ation", + "ab en", + "abe n", + "a ben", + "▁Pro gress", + "▁ Progress", + "▁C osta", + "▁Co sta", + "▁Cost a", + "▁Cos ta", + "▁dev oted", + "▁gest ure", + "▁Dez ember", + "▁qu oted", + "▁quote d", + "▁quot ed", + "▁difficult ies", + "т ре", + "▁sustain able", + "▁d ense", + "▁den se", + "▁dens e", + "▁ih rer", + "▁ihr er", + "▁ihre r", + "▁firm ly", + "â t", + "om ent", + "ome nt", + "omen t", + "o ment", + "▁c out", + "▁co ut", + "▁cou t", + "▁ cout", + "▁p oi", + "▁po i", + "d jango", + "▁pro found", + "▁prof ound", + "▁Wil helm", + "▁fl ush", + "▁flu sh", + "▁ flush", + "▁av ril", + "LA B", + "L AB", + "▁B row", + "▁Br ow", + "▁Bro w", + "▁pro pose", + "▁prop ose", + "▁propos e", + "▁r anks", + "▁ran ks", + "▁rank s", + "WI D", + "W ID", + "▁mut ual", + "▁text s", + "▁tex ts", + "▁S ale", + "▁Sal e", + "▁Sa le", + "▁qu asi", + "▁n og", + "▁no g", + "▁ nog", + "▁nouve au", + "▁c v", + "▁ cv", + "▁no ble", + "▁nob le", + "▁dé cembre", + "▁déc embre", + "▁cl ever", + "▁cle ver", + "▁P ir", + "▁Pi r", + "▁graph ics", + "▁graphic s", + "▁ graphics", + "▁G R", + "▁ GR", + "че ской", + "▁s ag", + "▁sa g", + "ict ions", + "iction s", + "i ctions", + "na nt", + "nan t", + "n ant", + "▁th é", + "C G", + "▁Jac ques", + "W M", + "▁F inn", + "▁Fin n", + "▁Fi nn", + "▁dev ast", + "зо м", + "хо в", + "х ов", + "▁En tre", + "▁Ent re", + ". ;", + "▁fl uct", + "▁flu ct", + "▁Sc iences", + "▁Sci ences", + "▁Science s", + "▁т у", + "▁ ту", + "path s", + "pat hs", + "▁sh orter", + "▁short er", + "▁suggest ion", + "ER Y", + "▁D ire", + "▁Di re", + "▁Dir e", + "at eurs", + "ate urs", + "ateur s", + "▁round ed", + "▁t art", + "▁tar t", + "▁ta rt", + "ю ще", + "up er", + "u per", + "▁secret s", + "▁sec rets", + "▁secre ts", + "▁compan ion", + "▁K EY", + "▁ KEY", + "T ile", + "▁B ibli", + "x s", + "▁ang ular", + "▁ angular", + "pa g", + "p ag", + "er ness", + "ern ess", + "erne ss", + "▁S orry", + "▁Sor ry", + "▁ Sorry", + "▁pre diction", + "▁predict ion", + "▁pred iction", + "▁M aking", + "▁Ma king", + "▁Mak ing", + "на род", + "ol are", + "ola re", + "olar e", + "rp c", + "r pc", + "▁t ens", + "▁te ns", + "▁ten s", + "en as", + "ena s", + "e nas", + "▁Re ally", + "▁Real ly", + "H I", + "port al", + "por tal", + "▁for me", + "▁form e", + "gan g", + "ga ng", + "g ang", + "▁l ane", + "▁la ne", + "▁lan e", + "▁ lane", + "▁s tag", + "▁st ag", + "▁sta g", + "▁Mar x", + "▁Ma rx", + "▁L LC", + "▁LL C", + "▁d are", + "▁da re", + "▁dar e", + "▁Olymp ic", + "▁p ant", + "▁pan t", + "▁pa nt", + "build ing", + "; ;", + "▁c ops", + "▁co ps", + "▁cop s", + "▁r ushed", + "▁rush ed", + "▁rus hed", + "▁L ot", + "▁Lo t", + "▁init iative", + "▁initi ative", + "▁inv ite", + "▁Saf ety", + "▁Safe ty", + "FA ILED", + "FAIL ED", + "▁habit ants", + "en sen", + "ens en", + "ense n", + "▁l ég", + "▁W elcome", + "▁Wel come", + "Valid ate", + "▁qu atre", + "▁G ray", + "▁Gr ay", + "▁Gra y", + "▁E ve", + "▁Ev e", + "▁C omb", + "▁Com b", + "▁Co mb", + "▁ Comb", + "▁p endant", + "a qu", + "con figure", + "config ure", + "▁A dm", + "▁Ad m", + "▁rif le", + "▁Exper ience", + "Decl aration", + "▁å r", + "▁ år", + "ill ery", + "ille ry", + "iller y", + "os pel", + "osp el", + "▁A rena", + "▁Ar ena", + "▁Are na", + "▁bo ards", + "▁board s", + "▁ boards", + "▁pur ple", + "▁p ills", + "▁pil ls", + "▁pill s", + "ueto oth", + "li que", + "l ique", + "▁pop ulations", + "▁population s", + "▁popul ations", + "▁acc ent", + "▁ac cent", + "▁r anges", + "▁range s", + "▁ran ges", + "▁rang es", + "▁Anal ysis", + "▁ Analysis", + "▁d ictionary", + "▁Dr agon", + "▁Drag on", + "re ction", + "rect ion", + "r ection", + "▁vis itor", + "▁visit or", + "seg ment", + "▁д р", + "▁F uck", + "▁Fu ck", + "д ж", + "▁ident ification", + "Class Name", + "boot strap", + "▁sur faces", + "▁surface s", + "▁surf aces", + "▁scream ing", + "кт у", + "к ту", + "pl ain", + "sh adow", + "incl udes", + "include s", + "▁j azz", + "▁ja zz", + "▁á l", + "▁ ál", + "ri ka", + "rik a", + "r ika", + "ho p", + "h op", + "▁i on", + "▁io n", + "▁ ion", + "vr e", + "v re", + "▁newsp apers", + "▁newspaper s", + "▁i hn", + "▁ih n", + "▁P arse", + "▁Par se", + "▁Pars e", + "▁ Parse", + "П о", + "▁strict ly", + "▁re covered", + "▁recover ed", + "▁U na", + "▁Un a", + "▁err e", + "▁er re", + "▁ erre", + "iss ues", + "issue s", + "▁exp ense", + "че ния", + "▁do nc", + "▁don c", + "Bi n", + "B in", + "▁Com ment", + "▁Comm ent", + "▁ Comment", + "▁sac rifice", + "▁sacrific e", + "T uple", + "() [", + "( )[", + "▁tra vers", + "▁trav ers", + "Im p", + "I mp", + "J e", + "▁Lin ux", + "▁е ё", + "▁P i", + "▁ Pi", + "▁cur ios", + "▁cu rios", + "▁r age", + "▁rag e", + "▁ra ge", + "▁ rage", + "▁e scal", + "▁es cal", + "▁esc al", + "▁al ignment", + "▁align ment", + "▁pent ru", + "▁cur r", + "▁cu rr", + "▁ curr", + "▁b este", + "▁be ste", + "▁best e", + "▁bes te", + "[] ,", + "[ ],", + "▁// !", + "H ub", + "Vis ibility", + "▁A sk", + "▁As k", + "ab ul", + "a bul", + "co lon", + "col on", + "colo n", + "▁D ays", + "▁Day s", + "▁Da ys", + "Aut hentication", + "ві т", + "▁l od", + "▁lo d", + "xF C", + "x FC", + "Look up", + "js ce", + "Al pha", + "▁harm ony", + "▁harmon y", + "▁W ard", + "▁War d", + "▁Wa rd", + "trans fer", + "▁H orn", + "▁Hor n", + "▁Ho rn", + "▁s d", + "▁ sd", + "so ap", + "▁z ich", + "▁Con sole", + "▁Cons ole", + "▁ Console", + "▁ко ли", + "▁Ph one", + "▁ Phone", + "pa per", + "p aper", + "й н", + "▁z m", + "▁ zm", + "Do ne", + "Don e", + "D one", + "ph ase", + "pha se", + "phas e", + "▁Jul ia", + "▁Ju lia", + "▁Juli a", + "▁ed ited", + "▁edit ed", + "af fe", + "aff e", + "Sy ntax", + "yl l", + "y ll", + "▁Lu cas", + "▁Luc as", + "▁and eren", + "▁andere n", + "▁ander en", + "[ <", + "▁Data base", + "▁Dat abase", + "▁ Database", + "▁spect ral", + "▁spectra l", + "ass ador", + "ска та", + "с ката", + "▁import ante", + "▁important e", + "▁х а", + "▁ ха", + "t z", + "▁s tere", + "▁st ere", + "▁ste re", + "▁ster e", + "▁m elt", + "▁me lt", + "▁mel t", + "▁C row", + "▁Cr ow", + "▁Cro w", + "ш ка", + "it utes", + "itut es", + "itute s", + "itu tes", + "▁satisf ies", + "▁L iga", + "▁Li ga", + "▁t omb", + "▁to mb", + "▁tom b", + "▁f ühr", + "▁ führ", + "▁sol ely", + "▁sole ly", + "▁E ither", + "▁t ennis", + "▁ten nis", + "▁s igh", + "▁si gh", + "▁sig h", + "ser de", + "s erde", + "ub a", + "u ba", + "ę d", + "le z", + "l ez", + "Fac t", + "F act", + "▁sque ez", + "▁Thom pson", + "▁N L", + "▁ NL", + "▁P ara", + "▁Par a", + "▁Pa ra", + "▁? ?", + "▁ ??", + "▁fin ishing", + "▁finish ing", + "She et", + "S heet", + "LIN K", + "L INK", + "▁б ро", + "▁ бро", + "▁l over", + "▁lo ver", + "▁love r", + "▁lov er", + "m achine", + "▁L esser", + "▁Les ser", + "▁Less er", + "pon d", + "po nd", + "p ond", + "▁pain tings", + "▁paint ings", + "▁painting s", + "▁assum ptions", + "▁assumption s", + "▁mod ification", + "fr e", + "f re", + "▁U lt", + "▁Ul t", + "▁A F", + "▁ AF", + "R V", + "bin ding", + "bind ing", + "b inding", + "▁toile t", + "ra r", + "r ar", + "▁an ge", + "▁ang e", + "▁ ange", + "▁she ep", + "PRO TO", + "act ic", + "a ctic", + "▁S peed", + "▁Sp eed", + "▁Spe ed", + "▁ Speed", + "▁I ce", + "gn u", + "g nu", + "ow ned", + "own ed", + "Sub scription", + "yr ics", + "y rics", + "▁back ward", + ">\" .", + "> \".", + "pi t", + "p it", + "▁real istic", + "öff ent", + "az i", + "a zi", + "DE R", + "D ER", + "b ucket", + "én y", + "é ny", + "xF E", + "x FE", + "▁f ancy", + "▁fan cy", + "ex cept", + "▁S ul", + "▁Su l", + "▁l aser", + "▁la ser", + "▁las er", + "Mon itor", + "▁c omic", + "▁com ic", + "▁co mic", + "▁Arch itect", + "▁ex pr", + "▁exp r", + "▁ expr", + "ount ers", + "oun ters", + "ounter s", + "▁Mel bourne", + "com plex", + "comp lex", + "'. $", + "' .$", + "om ot", + "omo t", + "o mot", + "▁M enu", + "▁Me nu", + "▁Men u", + "▁ Menu", + "astic search", + "▁ed iting", + "▁edit ing", + "Pre sent", + "Pres ent", + "P resent", + "op les", + "ople s", + "opl es", + "o ples", + "è ncia", + "▁в то", + "gl ise", + "she et", + "s heet", + "▁he lic", + "▁hel ic", + "▁str anger", + "▁strange r", + "▁strang er", + "▁ex ec", + "▁ exec", + "FE R", + "F ER", + "in ian", + "ini an", + "SET TING", + "▁M ix", + "▁Mi x", + "▁ Mix", + "▁com plain", + "▁compl ain", + "▁in crement", + "▁incre ment", + "CS S", + "C SS", + "mm a", + "m ma", + "sl ide", + "▁про тив", + "▁проти в", + "▁Lim ited", + "Con sole", + "Cons ole", + "▁eng aging", + "ul er", + "ule r", + "u ler", + "▁O ptions", + "▁Option s", + "▁Opt ions", + "▁ Options", + "▁l ens", + "▁le ns", + "▁len s", + "Ma il", + "M ail", + "▁bar rier", + "▁barr ier", + "trans port", + "▁c ups", + "▁cu ps", + "▁cup s", + "it err", + "ite rr", + "iter r", + "▁const ants", + "▁constant s", + "▁ constants", + "▁T ech", + "▁Te ch", + "iz io", + "izi o", + "сту па", + "ступ а", + "▁Sw eden", + "at hon", + "ath on", + "a thon", + "▁M agn", + "▁Mag n", + "▁Ma gn", + "trans ition", + "де ла", + "es k", + "e sk", + "So ft", + "S oft", + "fun ctions", + "function s", + "ne a", + "n ea", + "Im plement", + "Impl ement", + "Imp lement", + "ev ery", + "ever y", + "eve ry", + "e very", + "▁Man ufact", + "▁improve ments", + "▁improvement s", + "▁Ind iana", + "▁India na", + "▁Indian a", + "▁host s", + "▁ho sts", + "C V", + "We st", + "W est", + "to wn", + "t own", + "can vas", + "▁ш ко", + "▁Col umn", + "▁ Column", + "▁Par ker", + "▁Park er", + "▁es pa", + "▁esp a", + "▁Pub lish", + "▁которы й", + "av is", + "avi s", + "a vis", + "▁Z w", + "▁emphas is", + "ol v", + "o lv", + "▁re curs", + "▁rec urs", + "▁recur s", + "it aire", + "ita ire", + "▁B ishop", + "▁Bi shop", + "▁Bis hop", + "ne ro", + "ner o", + "n ero", + "▁d eny", + "▁de ny", + "▁den y", + "▁do ub", + "▁dou b", + "peon ato", + "▁C ourse", + "▁Cour se", + "▁Que ens", + "▁Queen s", + "▁bl ur", + "el ed", + "ele d", + "e led", + "iz o", + "i zo", + "▁dé but", + "▁Mod ule", + "▁Mo dule", + "▁ Module", + "▁anx ious", + "▁st are", + "▁star e", + "▁sta re", + "▁Pro position", + "▁K u", + "▁i c", + "▁ ic", + "Per cent", + "Qu ant", + "▁И сто", + "▁h ex", + "▁he x", + "▁ hex", + "ass oci", + "asso ci", + "▁arrang ement", + "▁arrange ment", + "▁bo ats", + "▁boat s", + "Un d", + "U nd", + "▁sl ots", + "▁slot s", + "се н", + "с ен", + "necess ary", + "▁app earing", + "▁appe aring", + "▁appear ing", + "▁R ule", + "▁Ru le", + "▁ Rule", + "▁G T", + "▁ GT", + "For ce", + "et to", + "ett o", + "e tto", + "ze nia", + "zen ia", + "▁o uts", + "▁out s", + "▁ou ts", + "▁ outs", + "▁vari ations", + "▁variation s", + "▁wh ites", + "▁white s", + "▁g lo", + "▁gl o", + "▁B R", + "▁ BR", + "ic ky", + "ick y", + "▁j ury", + "▁ju ry", + "▁jur y", + "▁treat ments", + "▁treatment s", + "▁The ater", + "kn ow", + "k now", + "▁pro files", + "▁prof iles", + "▁profile s", + "▁con spir", + "▁class room", + "▁B ass", + "▁Bas s", + "▁Ba ss", + "▁law yers", + "▁lawyer s", + "v ue", + "▁A rc", + "▁Ar c", + "▁ Arc", + "▁s la", + "▁sl a", + "▁att ending", + "▁attend ing", + "n x", + "m x", + "TO P", + "T OP", + "▁b ored", + "▁bo red", + "▁bore d", + "▁bor ed", + "pre vious", + "prev ious", + "r w", + "pt ic", + "љ у", + "▁app ar", + "▁ap par", + "▁P ont", + "▁Po nt", + ": _", + "ii i", + "i ii", + "▁j erk", + "▁jer k", + "hed ral", + "сс а", + "с са", + "▁Pr ize", + "▁Pri ze", + "▁Р и", + "б ре", + "▁hand les", + "▁handle s", + "▁j ak", + "▁ja k", + "▁Afghan istan", + "▁b oring", + "▁bo ring", + "▁bor ing", + "if ik", + "ifi k", + "▁sh ade", + "▁sha de", + "air o", + "ai ro", + "a iro", + "od ay", + "oda y", + "o day", + "▁pl ates", + "▁plate s", + "▁plat es", + "▁Championship s", + "▁Champion ships", + "▁che eks", + "▁cheek s", + "ri ke", + "rik e", + "r ike", + "▁kön nen", + "▁app le", + "▁ap ple", + "▁appl e", + "▁ apple", + "▁Ed die", + "▁Edd ie", + "▁s od", + "▁so d", + "▁tr ains", + "▁tra ins", + "▁train s", + "pan ic", + "pa nic", + "▁Ad vent", + "▁Adv ent", + "ub re", + "u bre", + "▁d å", + "▁S ymbol", + "▁Sym bol", + "▁ Symbol", + "▁с те", + "▁ст е", + "▁ сте", + "Sa m", + "S am", + "inher it", + "cam era", + "▁c ours", + "▁co urs", + "▁cour s", + "▁cou rs", + "▁make up", + "re gex", + "reg ex", + "▁U E", + "▁ UE", + "▁Det roit", + "▁W eight", + "▁We ight", + "▁ Weight", + "▁P iet", + "▁Pi et", + "▁a ria", + "▁ar ia", + "▁ aria", + "DI RECT", + "DIR ECT", + "ace ae", + "▁In fo", + "▁Inf o", + "▁ Info", + "an ya", + "any a", + "back end", + "▁T ennessee", + "pi cker", + "pic ker", + "pick er", + "p icker", + "▁Le o", + "▁P oss", + "▁Po ss", + "▁Pos s", + "pr ises", + "prise s", + "▁m ature", + "▁mat ure", + "сь ких", + "▁F ant", + "▁Fa nt", + "Re ason", + "▁m oy", + "▁mo y", + "▁B aker", + "▁Ba ker", + "▁Bak er", + "▁sub set", + "▁subs et", + "▁ subset", + "▁Stan ley", + "▁el even", + "▁ele ven", + "▁elev en", + "ol ate", + "ola te", + "o late", + "▁fort une", + "Status Code", + "▁ent ities", + "▁ entities", + "▁Ok ay", + "▁ Okay", + "ц о", + "an os", + "ano s", + "a nos", + "rel ative", + "▁order ing", + "▁ord ering", + "▁No body", + "▁Nob ody", + "▁str len", + "▁ strlen", + "▁r ope", + "▁ro pe", + "▁cig arette", + "hol ds", + "hold s", + "h olds", + "ir able", + "ira ble", + "value Of", + "St ub", + "▁phot ography", + "▁photograph y", + "es tra", + "est ra", + "estr a", + "e stra", + "▁cult ures", + "▁culture s", + "▁decl aration", + "▁declar ation", + "mer cial", + "LI ED", + "au te", + "aut e", + "a ute", + "al ter", + "alt er", + "Sub mit", + "▁Mag ic", + "▁r hythm", + "Pay ment", + "ni h", + "n ih", + "▁inter section", + "l é", + "EN TRY", + "/ )", + "▁m og", + "▁mo g", + "ru st", + "rus t", + "r ust", + "▁threat s", + "▁Mil itary", + "ap or", + "a por", + "▁s igu", + "▁si gu", + "▁sig u", + "set minus", + "▁I ng", + "▁In g", + "st ation", + "stat ion", + "T ake", + "▁s hed", + "▁sh ed", + "▁she d", + "▁Fr ancia", + "▁Franc ia", + "▁Fra ncia", + "▁Fran cia", + "pos ts", + "po sts", + "post s", + "Mar ker", + "Mark er", + "Lower Case", + "▁be find", + "▁bef ind", + "▁C zech", + "▁Cz ech", + "ícul a", + "▁Per formance", + "▁W es", + "▁We s", + "▁L arry", + "▁Lar ry", + "▁o st", + "▁os t", + "▁ ost", + "▁em ails", + "▁email s", + "▁Re lease", + "▁ Release", + "▁ad apter", + "▁adapt er", + "▁ adapter", + "▁pad re", + "ac io", + "aci o", + "a cio", + "▁з ем", + "▁gen etic", + "▁ge netic", + "▁U nd", + "▁Un d", + "▁ Und", + "▁accept ance", + "да н", + "д ан", + "▁Girl s", + "▁Gir ls", + "comp iler", + "compile r", + "su n", + "s un", + "▁whe els", + "▁wheel s", + "▁thorough ly", + "gr und", + "gru nd", + "g rund", + "un ction", + "unct ion", + "▁e lla", + "▁el la", + "▁ell a", + "▁ ella", + "X FF", + "ug s", + "u gs", + "ient os", + "ien tos", + "iento s", + "▁D M", + "▁ DM", + "▁polit ique", + "▁campaign s", + "▁Tok yo", + "▁album s", + "KERN EL", + "pd ata", + "p data", + "▁lap top", + "▁lapt op", + "▁v ál", + "▁vá l", + "▁f ou", + "▁fo u", + "or b", + "o rb", + "▁T ower", + "▁To wer", + "▁Tow er", + "▁Get ting", + "▁cor ners", + "▁corner s", + "▁corn ers", + "pl ess", + "ple ss", + "ples s", + "p less", + "▁special ist", + "▁i v", + "▁ iv", + "Ui nt", + "U int", + "▁name ly", + "▁nam ely", + "▁sc aling", + "▁scal ing", + "Ext ensions", + "Extension s", + "▁cent ro", + "omorph ism", + "▁dé f", + "), \\", + ") ,\\", + "▁contr ary", + "▁contra ry", + "▁str iking", + "▁stri king", + "▁B ere", + "▁Be re", + "▁Ber e", + "▁fore cast", + "▁z ones", + "▁zone s", + "▁zo nes", + "sm art", + "s mart", + "as hi", + "ash i", + "ri n", + "r in", + "NE W", + "▁sim ulations", + "▁simulation s", + "▁R ather", + "▁Ra ther", + "▁Rat her", + "▁Writ ing", + "▁Wr iting", + "▁$ [", + "▁as sh", + "▁ass h", + "▁f ailing", + "▁fa iling", + "▁fail ing", + "▁man if", + "▁B og", + "▁Bo g", + "▁D ir", + "▁Di r", + "▁ Dir", + "▁influ enced", + "▁influence d", + "conf irm", + "▁we igh", + "▁in ventory", + "▁invent ory", + "▁a pare", + "▁ap are", + "▁e u", + "▁ eu", + "char acter", + "io m", + "i om", + "▁o rb", + "▁or b", + "▁ orb", + "dev ices", + "device s", + "▁L ED", + "▁LE D", + "▁ LED", + "▁prop ortion", + "▁proport ion", + "▁Hon or", + "▁Ho nor", + "▁appro aching", + "▁approach ing", + "de leg", + "del eg", + "▁B B", + "▁ BB", + "hel pers", + "help ers", + "helper s", + "re pository", + "rep ository", + "▁б ере", + "▁бе ре", + "▁inhab it", + "▁s ão", + "▁travel ed", + "▁trav eled", + "ne x", + "n ex", + "▁C lin", + "▁Cl in", + "CE PT", + "▁off ense", + "▁in cent", + "▁inc ent", + "ID S", + "I DS", + "▁coeff icients", + "▁coefficient s", + "▁l p", + "▁ lp", + "чно го", + "ч ного", + "▁c d", + "▁ cd", + "mu st", + "mus t", + "m ust", + "▁soon er", + "ez e", + "e ze", + "C at", + "ma ker", + "make r", + "m aker", + "▁r anked", + "▁ran ked", + "▁rank ed", + "ful ness", + "▁part ially", + "▁partial ly", + "▁parti ally", + "Pro m", + "Pr om", + "P rom", + "▁ф он", + "▁фо н", + "▁Pro bably", + "▁c ached", + "▁cache d", + "▁ca ched", + "▁ cached", + "▁bal anced", + "▁balance d", + "ah oma", + "aho ma", + "▁Mur ray", + "▁a li", + "▁al i", + "▁ ali", + "iv os", + "ivo s", + "▁b ark", + "▁bar k", + "IT EM", + "ITE M", + "▁Kir che", + "▁alloc ated", + "▁allocate d", + "Al t", + "A lt", + "▁am éric", + "íl ia", + "í lia", + "▁c ens", + "▁ce ns", + "▁lic enses", + "▁license s", + "▁ licenses", + "ac z", + "a cz", + "▁G ate", + "▁Ga te", + "▁ Gate", + "▁B L", + "▁ BL", + "▁re public", + "▁rep ublic", + "RO W", + "▁состав ля", + "▁соста вля", + "▁Fil ip", + "▁Ind ivid", + "▁tr ials", + "▁tri als", + "▁trial s", + "/* !", + "▁G P", + "▁ GP", + "ni ka", + "nik a", + "n ika", + "▁ex em", + "▁ad vers", + "▁adv ers", + "um ped", + "ump ed", + "▁Dev ice", + "▁ Device", + "wa ke", + "w ake", + "Ex ec", + "ar ding", + "ard ing", + "ardi ng", + "▁pobl ación", + "▁k een", + "▁ke en", + "▁b itch", + "▁bit ch", + "▁embed ded", + "▁B ond", + "▁Bo nd", + "▁Bon d", + "ri des", + "ride s", + "rid es", + "r ides", + "▁W oman", + "▁Wo man", + ". [", + "ér é", + "é ré", + "▁Hash Map", + "▁ HashMap", + "▁co unting", + "▁coun ting", + "▁count ing", + "▁Init ial", + "▁ Initial", + "▁ver se", + "▁vers e", + "▁ verse", + "▁Vere in", + ">\" ,", + "> \",", + "▁an th", + "▁ant h", + "▁ anth", + "ci d", + "c id", + "▁h unt", + "▁hun t", + "на л", + "н ал", + "ci es", + "cie s", + "c ies", + "Pi n", + "P in", + "▁# !", + "ва я", + "sn d", + "s nd", + "▁u k", + "▁ uk", + "▁sw ift", + "▁tempor ada", + "▁environment s", + "▁environ ments", + "claim er", + "eme tery", + "emet ery", + "j är", + "▁ча ст", + "▁час т", + "Trans port", + "▁A rr", + "▁Ar r", + "▁ Arr", + "▁P aper", + "▁Pa per", + "▁Pap er", + "▁b ew", + "▁be w", + "▁ bew", + "▁har vest", + "▁- ----", + "▁-- ---", + "▁--- --", + "▁ -----", + "product s", + "ле т", + "л ет", + "ident ifier", + "RO OT", + "▁M ak", + "▁Ma k", + "▁App ro", + "▁Ap pro", + "▁ Appro", + "ie ri", + "ier i", + "i eri", + "▁F ly", + "▁Fl y", + "▁is set", + "▁iss et", + "▁ isset", + "▁determ ination", + "▁determin ation", + "Ge ometry", + "▁emer ging", + "sub scription", + "ol y", + "o ly", + "▁R ace", + "▁Ra ce", + "▁B ah", + "▁Ba h", + "▁Config uration", + "▁ Configuration", + "▁Inter est", + "ско в", + "ск ов", + "с ков", + "ist rz", + "istr z", + "▁S han", + "▁Sh an", + "▁Sha n", + "▁P ain", + "▁Pa in", + "CON NE", + "ma jor", + "m ajor", + "▁St ay", + "▁Sta y", + "▁bron ze", + "▁f itting", + "▁fit ting", + "▁J ar", + "▁Ja r", + "mg r", + "m gr", + "▁S har", + "▁Sh ar", + "▁Sha r", + "FL O", + "F LO", + "ut er", + "ute r", + "u ter", + "с ы", + "▁cont acts", + "▁contact s", + "▁f iring", + "▁fi ring", + "▁fir ing", + "на н", + "н ан", + "▁prof es", + "sk é", + "s ké", + "▁rule d", + "▁ru led", + "▁rul ed", + "=\" /", + "an dro", + "and ro", + "▁ens uring", + "iz en", + "ize n", + "i zen", + "▁че рез", + "ise cond", + "i second", + "ob il", + "obi l", + "o bil", + "▁re ck", + "▁rec k", + "▁ reck", + ")} (", + ") }(", + "bit map", + "▁B run", + "▁Br un", + "▁Bru n", + "▁Jer usalem", + "▁W o", + "▁Republic ans", + "▁Republican s", + "mat ic", + "m atic", + "▁E arl", + "▁d ock", + "▁do ck", + "▁doc k", + "▁M all", + "▁Mal l", + "▁Ma ll", + "k k", + "▁ Й", + "▁C OL", + "▁CO L", + "▁ COL", + "▁lat ach", + "UI nt", + "U Int", + "ци ональ", + "цион аль", + "циона ль", + "▁seg ments", + "▁segment s", + "▁re fund", + "▁ref und", + "fa c", + "f ac", + "▁Art icle", + "▁B orn", + "▁Bo rn", + "▁Bor n", + "² .", + "br and", + "bra nd", + "b rand", + "{$ \\", + "{ $\\", + "▁s s", + "▁ ss", + "▁Re sources", + "▁Res ources", + "▁Resource s", + "▁ Resources", + "▁re cycl", + "▁rec ycl", + "▁$ $\\", + "▁$$ \\", + "▁Conne ction", + "▁Connect ion", + "▁ Connection", + "▁imp erial", + "▁imper ial", + "▁pract ically", + "▁practical ly", + "▁– ,", + "▁Dis play", + "▁ Display", + "ier no", + "mo uth", + "m outh", + "ed es", + "ede s", + "e des", + "ba hn", + "b ahn", + "▁C atherine", + "▁high way", + "un ting", + "unt ing", + "▁Any way", + "Sp ell", + "Spe ll", + "▁L iste", + "▁List e", + "▁Li ste", + "▁Lis te", + "▁ret rieve", + "▁retr ieve", + "▁retriev e", + "▁z d", + "▁ zd", + "stra ße", + "▁dom inated", + "▁domin ated", + "to uch", + "t ouch", + "▁m b", + "▁ mb", + "LO NG", + "L ONG", + "as ures", + "asure s", + "TL S", + "T LS", + "▁accompl ished", + "▁accomp lished", + "▁accomplish ed", + "▁f ears", + "▁fe ars", + "▁fear s", + "▁seem ingly", + "▁d ag", + "▁da g", + "▁ dag", + "▁b ureau", + "▁bur eau", + "▁Gro ß", + "▁accord ance", + ". ]", + "ou x", + "o ux", + "▁col onial", + "▁colon ial", + "▁compass ion", + "th umb", + "▁s wo", + "▁sw o", + "on line", + "▁J i", + "▁work shop", + "▁works hop", + "▁l ub", + "▁lu b", + "év rier", + "ш і", + ">\" ;", + "> \";", + "▁gener ous", + "▁gene rous", + "ro us", + "rou s", + "r ous", + "av id", + "avi d", + "a vid", + "igen ous", + "▁R aw", + "▁Ra w", + "▁ Raw", + "▁sw ap", + "▁ swap", + "h c", + "java script", + "jav ascript", + "Fact or", + "Fac tor", + "F actor", + "▁gar bage", + "▁M icro", + "▁Mic ro", + "▁Mi cro", + "co u", + "c ou", + "ü ber", + "▁f atal", + "▁fa tal", + "▁fat al", + "▁trans parent", + "▁b earing", + "▁be aring", + "▁bear ing", + "▁celebr ated", + "▁celebrate d", + "VI S", + "V IS", + "▁B M", + "▁ BM", + "▁pr ince", + "▁prin ce", + "to l", + "t ol", + "▁' ", + "\\ \">", + "▁du rant", + "▁dur ant", + "▁vent ure", + "▁F itz", + "▁Fi tz", + "▁C BD", + "▁CB D", + "▁b acking", + "▁back ing", + "▁w are", + "▁war e", + "▁wa re", + "▁ ware", + "ev e", + "e ve", + "O G", + "ed ish", + "edi sh", + "▁Giov anni", + "▁Sh are", + "▁Shar e", + "▁Sha re", + "▁ Share", + "▁rec ipes", + "▁recipe s", + "▁recip es", + "big g", + "bi gg", + "b igg", + "▁minor ity", + "▁n ar", + "▁na r", + "▁ nar", + "oll ary", + "ollar y", + "▁F E", + "▁ FE", + "sh irt", + "▁redu ces", + "▁reduce s", + "Ch e", + "C he", + "▁NOT E", + "▁NO TE", + "j query", + "▁F low", + "▁Fl ow", + "▁Flo w", + "▁ Flow", + "task s", + "pr event", + "pre vent", + "prev ent", + "▁со вет", + "▁сов ет", + "it as", + "ita s", + "▁exam ined", + "▁examine d", + "ho n", + "h on", + "▁M ine", + "▁Min e", + "▁Mi ne", + "▁grad ient", + "▁V ien", + "▁Vi en", + "▁b eds", + "▁be ds", + "▁bed s", + "ET H", + "E TH", + "fl at", + "f lat", + "an son", + "ans on", + "▁in tu", + "▁int u", + "▁fl ows", + "▁flo ws", + "▁flow s", + "но к", + "▁E ine", + "▁Ein e", + "ро ди", + "род и", + "▁ко р", + "▁к ор", + "▁ кор", + "▁aff ection", + "▁af fection", + "▁affect ion", + "▁p orts", + "▁por ts", + "▁port s", + "▁ ports", + "__ .", + "_ _.", + "re po", + "rep o", + "ail and", + "ai land", + "▁по да", + "▁под а", + "int age", + "inta ge", + "▁Prote ction", + "î t", + "▁[ {", + "▁l amp", + "▁la mp", + "▁benef icial", + "ка де", + "▁Станов ништво", + "▁l ined", + "▁li ned", + "▁line d", + "▁lin ed", + "▁ lined", + "▁Ex change", + "▁f itted", + "▁fit ted", + "▁v erk", + "▁ver k", + "▁focus es", + "vo d", + "v od", + "▁Car lo", + "▁Carl o", + "▁ра спо", + "▁рас по", + "ain ted", + "aint ed", + "ainte d", + "a inted", + "▁r ape", + "▁rap e", + "▁ra pe", + "▁t ogg", + "▁to gg", + "ac ker", + "ack er", + "a cker", + "T w", + "ra h", + "r ah", + "trans l", + "▁je alous", + "▁re pository", + "▁rep ository", + "▁ repository", + "re marks", + "rem arks", + "remark s", + "▁i e", + "▁ ie", + "í d", + "▁sk ull", + "ra c", + "r ac", + "() ]", + "( )]", + "ri en", + "rie n", + "r ien", + "? (", + "▁K ids", + "▁Ki ds", + "▁Kid s", + "▁sw itched", + "▁switch ed", + "▁G ew", + "▁Ge w", + "▁be ef", + "▁appear ances", + "▁appearance s", + "▁Coll ins", + "▁V illa", + "▁Vill a", + "▁Vi lla", + "▁Vil la", + "▁z ona", + "▁zo na", + "▁n eu", + "▁ne u", + "те льно", + "тель но", + "▁х удо", + "▁oper ational", + "▁operation al", + "ON LY", + "▁h ockey", + "▁ś wi", + "ö k", + "Sl ice", + "Ref resh", + "▁n uts", + "▁nu ts", + "▁nut s", + "sa y", + "s ay", + "▁ста нови", + "▁станов и", + "х е", + "▁le aning", + "▁lean ing", + "▁H aus", + "▁Ha us", + "▁o ral", + "▁or al", + "▁ oral", + "▁ Ž", + "▁Sup pose", + "▁Supp ose", + "▁ess ence", + "EN TER", + "ENT ER", + "B ucket", + "▁C ant", + "▁Can t", + "▁Ca nt", + "▁N azi", + "▁Na zi", + "▁Naz i", + "ш ти", + "▁Vol ume", + "▁ Volume", + "▁wor thy", + "▁worth y", + "▁ worthy", + "B u", + "Ent ries", + "on ie", + "oni e", + "o nie", + "▁h ood", + "▁ho od", + "▁ hood", + "▁emp ire", + "▁dé velop", + "▁p robe", + "▁pro be", + "▁pr obe", + "▁prob e", + "▁ probe", + "▁K night", + "▁Kn ight", + "▁peace ful", + "hu b", + "h ub", + "▁ál bum", + "su it", + "s uit", + "▁sil k", + "+ =", + "▁p ione", + "▁pi one", + "' \"", + "ка ми", + "▁N ull", + "▁Nu ll", + "▁ Null", + "Label s", + "au tres", + "aut res", + "autre s", + "to LowerCase", + "▁b uzz", + "▁bu zz", + "▁w ashed", + "▁was hed", + "▁wash ed", + "' *", + "itzer land", + "▁r amp", + "▁ra mp", + "▁ram p", + "▁к ни", + "▁k un", + "col ors", + "color s", + "colo rs", + "▁vacc ine", + "an imation", + "anim ation", + "▁Just in", + "mem set", + "▁c ensus", + "▁cens us", + "in fl", + "inf l", + "▁statist ical", + "▁trop ical", + "Dis abled", + "Disable d", + "\r \r", + "▁Cra ig", + "Page s", + "Pag es", + "P ages", + "▁mag az", + "▁comput ing", + "▁flo ors", + "▁floor s", + "oin e", + "oi ne", + "o ine", + "▁tit olo", + "▁an ci", + "▁anc i", + "▁Indust ry", + "▁г лав", + "▁гла в", + "Bo ot", + "B oot", + "Cl ip", + "▁d v", + "▁ dv", + "▁met all", + "▁metal l", + "▁meta ll", + "▁Is abel", + "▁Isa bel", + "▁look up", + "▁ lookup", + "▁ц ер", + "▁це р", + "▁ цер", + "▁car ries", + "f u", + "tp l", + "t pl", + "pe rp", + "per p", + "▁St orm", + "▁Sto rm", + "eh icle", + "▁S even", + "▁Se ven", + "▁Sev en", + "љ а", + "▁l ut", + "▁lu t", + "th reshold", + "▁d ull", + "▁du ll", + "▁E ND", + "▁EN D", + "▁ END", + "▁O tto", + "▁Ot to", + "▁Ott o", + "▁there by", + "TE MP", + "T EMP", + "▁S cal", + "▁Sc al", + "▁ Scal", + "Com put", + "Comp ut", + "ip v", + "i pv", + "▁ins ane", + "▁myster ious", + "▁M is", + "▁Mi s", + "uch ar", + "uc har", + "u char", + "as ma", + "asm a", + "au ch", + "auc h", + "a uch", + "ne tt", + "net t", + "n ett", + "El em", + "E lem", + "de rive", + "der ive", + "▁murder ed", + "ak ten", + "akt en", + "akte n", + "ро ван", + "ров ан", + "рова н", + "▁a nos", + "▁an os", + "▁ano s", + "▁ anos", + "}} ^", + "} }^", + "▁F uß", + "▁Fu ß", + "▁S ister", + "▁Si ster", + "▁volunte er", + ":: _", + ": :_", + "er ta", + "ert a", + "▁бо лее", + "og rá", + "▁Im Gui", + "sa me", + "sam e", + "s ame", + "Sh adow", + "▁re actions", + "▁reaction s", + "▁react ions", + "▁purch asing", + "PRE FIX", + "▁emb od", + "со м", + "▁alt ogether", + "▁prom oting", + "▁promot ing", + "U V", + "▁ind uced", + "▁indu ced", + "▁eer ste", + "▁eerst e", + "Li fe", + "Lif e", + "L ife", + "hd d", + "h dd", + "ní ch", + "▁c hill", + "▁ch ill", + "▁chi ll", + "RG B", + "R GB", + "red uce", + "redu ce", + "FR OM", + "F ROM", + "dir name", + "▁t une", + "▁tu ne", + "▁tun e", + "▁r ay", + "▁ra y", + "▁ ray", + "T D", + "▁к ъ", + "▁Febru ar", + "▁suspend ed", + "▁susp ended", + "▁u pp", + "▁up p", + "▁ upp", + "er i", + "e ri", + "pr eter", + "pre ter", + "pret er", + "▁E R", + "▁ ER", + "то н", + "т он", + "▁c atal", + "▁cat al", + "▁ca tal", + "▁h iring", + "▁hi ring", + "▁п ів", + "▁пі в", + "▁Olymp ics", + "▁Olympic s", + "da le", + "dal e", + "d ale", + ":: {", + ": :{", + "▁expl oring", + "▁explo ring", + "▁с тал", + "▁ста л", + "▁ст ал", + "▁univers ities", + "Class es", + "▁ча с", + "▁C ool", + "▁Co ol", + "▁S ony", + "▁So ny", + "▁Son y", + "th al", + "tha l", + "t hal", + "▁es crit", + "▁esc rit", + "▁cor ruption", + "▁corrupt ion", + "az ar", + "aza r", + "▁N eb", + "▁Ne b", + "▁Py thon", + "▁c him", + "▁ch im", + "▁chi m", + "▁cap ability", + "cy cl", + "c ycl", + "▁re try", + "▁r etry", + "▁ret ry", + "▁retr y", + "▁ retry", + "++ ]", + "▁t oy", + "▁to y", + "▁T erry", + "▁Ter ry", + "▁Terr y", + "View ById", + "▁v ine", + "▁vi ne", + "▁vin e", + "▁Kit chen", + "▁B iden", + "▁Bi den", + "Back end", + "gl ich", + "g lich", + "re lation", + "rel ation", + "▁rat ings", + "▁ra tings", + "▁rating s", + "Execut or", + "ibr ation", + ">( )", + "> ()", + "▁he al", + "if iable", + "ifi able", + "par k", + "p ark", + "▁P ete", + "▁Pe te", + "▁Pet e", + "▁tr aged", + "▁tra ged", + "▁trag ed", + "▁ch uck", + "▁wire less", + "▁wir eless", + "Re place", + "Rep lace", + "IR Q", + "▁се зо", + "i ß", + "▁j unto", + "▁jun to", + "Lo w", + "L ow", + "▁s id", + "▁si d", + "▁ sid", + "Tag Helpers", + "TagHelper s", + "▁comp aring", + "▁compar ing", + "▁c elle", + "▁cell e", + "▁ce lle", + "▁cel le", + "▁obt aining", + "▁obtain ing", + "▁qu ar", + "▁q uar", + "Br o", + "B ro", + "▁E C", + "▁ EC", + "in ea", + "ine a", + "i nea", + "▁F ue", + "▁Fu e", + "▁Prince ss", + "▁Prin cess", + "ij o", + "i jo", + "ge ns", + "gen s", + "g ens", + "PO L", + "P OL", + "è tres", + "▁h ind", + "▁hi nd", + "▁ hind", + "Var iant", + "Vari ant", + "▁rece ives", + "▁receive s", + "go d", + "g od", + "ik en", + "ike n", + "i ken", + "na il", + "n ail", + "▁amer ican", + "▁ american", + "br as", + "bra s", + "b ras", + "(' \\", + "( '\\", + "ie ce", + "if ference", + "iffer ence", + "iffe rence", + "▁b ubble", + "▁bub ble", + "▁B ear", + "▁Be ar", + "un ivers", + "uni vers", + "▁demand ing", + "sa ved", + "save d", + "s aved", + "▁cred entials", + "MS M", + "M SM", + "▁struct ural", + "Con s", + "Co ns", + "C ons", + "▁Way ne", + "▁blank et", + "▁re pet", + "▁rep et", + "▁repe t", + "Ne g", + "N eg", + "▁exclusive ly", + "▁exclus ively", + "IF I", + "I FI", + "бур г", + "▁arg uing", + "▁Re pub", + "▁Rep ub", + "▁f rowned", + "▁fr owned", + "Met ric", + "M etric", + "sk im", + "ski m", + "s kim", + "▁П ет", + "▁Пе т", + "▁rele ases", + "▁release s", + "▁t ast", + "▁ta st", + "▁p reference", + "▁pre ference", + "▁prefer ence", + "▁S üd", + "▁Sü d", + "oc c", + "o cc", + "▁r x", + "▁ rx", + "activ ate", + "cl am", + "c lam", + "▁фи ль", + "▁Sud denly", + "▁cr ushing", + "▁crush ing", + "▁L ower", + "▁Lo wer", + "▁Low er", + "▁ Lower", + "ei ng", + "e ing", + "wa lt", + "wal t", + "w alt", + "▁Г ер", + "▁Ге р", + "▁m ö", + "ри сто", + "la gen", + "lag en", + "lage n", + "l agen", + "▁co aching", + "▁coach ing", + "ight ers", + "igh ters", + "ighter s", + "▁bas ement", + "▁base ment", + "▁F IX", + "▁FI X", + "▁ FIX", + "Te le", + "T ele", + "With out", + "▁Com mons", + "▁Comm ons", + "▁Common s", + "ul ly", + "ull y", + "h box", + "fl ash", + "▁por tal", + "▁port al", + "▁ portal", + "ot ype", + "o type", + "▁S or", + "▁So r", + "▁trou bles", + "▁trouble s", + "ar si", + "ars i", + "▁с тан", + "▁ста н", + "▁ст ан", + "▁ стан", + "CA M", + "C AM", + "▁de notes", + "▁den otes", + "▁denote s", + "LA NG", + "LAN G", + "L ANG", + "▁Be yond", + "▁Bey ond", + "▁Bo wl", + "▁Bow l", + "▁import antly", + "▁important ly", + "▁W R", + "▁ WR", + "▁rel ating", + "▁a nder", + "▁and er", + "▁an der", + "▁ ander", + "▁gr inned", + "▁grin ned", + "▁D ak", + "▁Da k", + "▁Brook lyn", + "▁d p", + "▁ dp", + "▁P oly", + "▁Pol y", + "▁Po ly", + "▁ Poly", + "▁Sch ul", + "▁B uffer", + "▁Buff er", + "▁ Buffer", + "▁h older", + "▁hold er", + "▁hol der", + "▁ holder", + "IC AL", + "I CAL", + "▁tra iler", + "▁trail er", + "er ek", + "ere k", + "e rek", + "▁n ě", + "▁ ně", + "sh aped", + "shape d", + "sha ped", + ": `", + "▁de code", + "▁dec ode", + "▁ decode", + "▁co unted", + "▁coun ted", + "▁count ed", + "▁v amp", + "▁va mp", + "▁re late", + "▁rel ate", + "▁M ason", + "▁Ma son", + "▁Mas on", + "▁t itled", + "▁title d", + "▁tit led", + "▁Kent ucky", + "▁particip ated", + "▁participate d", + "▁Jenn ifer", + "▁mat rices", + "Cal endar", + "st s", + "s ts", + "Ass oci", + "▁f orum", + "▁for um", + "▁fo rum", + "▁s phere", + "▁sp here", + "▁spher e", + "▁S EO", + "▁SE O", + "pop up", + "▁Current ly", + "CL E", + "C LE", + "▁vol unt", + "▁stell ar", + "for all", + "Is s", + "I ss", + "im et", + "ime t", + "i met", + "q p", + "la test", + "lat est", + "late st", + "▁config ured", + "▁configure d", + "ab ol", + "a bol", + "ig ent", + "igen t", + "ige nt", + "i gent", + "▁delay ed", + "ff ic", + "f fic", + "▁g ing", + "▁gi ng", + "▁ ging", + "▁s cent", + "▁sc ent", + "▁scen t", + "▁disg ust", + "▁disgu st", + "he sis", + "hes is", + "h esis", + "im en", + "ime n", + "i men", + "▁re ign", + "▁П и", + "ul as", + "ula s", + "u las", + "um ing", + "umin g", + "umi ng", + "u ming", + "in nings", + "inn ings", + "Re nd", + "R end", + "id ity", + "idi ty", + "▁do zens", + "▁dozen s", + "wa rf", + "war f", + "▁Del hi", + "▁bi ological", + "▁corrid or", + "Vis ual", + "▁I z", + "▁s uits", + "▁su its", + "▁suit s", + "Py Object", + "ia go", + "i ago", + "▁div ide", + "▁divid e", + "pe nt", + "pen t", + "p ent", + "hel lo", + "hell o", + "h ello", + "▁b eta", + "▁be ta", + "▁bet a", + "▁ beta", + "▁ex terior", + "▁fin est", + "▁fine st", + "▁B ir", + "▁Bi r", + "▁f reed", + "▁fr eed", + "▁free d", + "▁fre ed", + "▁K el", + "▁Ke l", + "Se m", + "S em", + "▁fr uits", + "▁fruit s", + "▁fru its", + "▁serv ants", + "▁servant s", + "▁pub lisher", + "▁publish er", + "▁cop per", + "ol ation", + "o lation", + "se p", + "s ep", + "▁chair man", + "ti k", + "t ik", + "▁m others", + "▁mother s", + "▁mo thers", + "A ug", + "▁je ans", + "[] )", + "[ ])", + "▁D ATA", + "▁DA TA", + "▁ DATA", + "▁reve als", + "▁reveal s", + "▁un conscious", + "▁h acer", + "▁ha cer", + "▁hace r", + "ric ulum", + "▁T ogether", + "▁ш та", + "▁ шта", + "or sz", + "ors z", + "▁c anal", + "▁can al", + "▁ca nal", + "ös t", + "ö st", + "▁equ als", + "▁equal s", + "▁eq uals", + "▁ equals", + "▁по мо", + "▁al location", + "▁all ocation", + "▁alloc ation", + "st änd", + "▁ч ер", + "▁че р", + "ac king", + "ack ing", + "▁motiv ation", + "со н", + "с он", + "▁R ole", + "▁Ro le", + "▁Rol e", + "▁ Role", + "App ly", + "Ap ply", + "ig es", + "ige s", + "i ges", + "* {", + "▁f ires", + "▁fire s", + "▁fi res", + "▁fir es", + "Us ed", + "Use d", + "U sed", + "▁he ute", + "sk iej", + "ski ej", + "▁Or leans", + "yl an", + "y lan", + "▁warm th", + "▁w elfare", + "▁wel fare", + "je m", + "j em", + "▁си сте", + "be z", + "b ez", + "ř e", + "ke e", + "k ee", + "▁segu ito", + "un ge", + "ung e", + "▁y oga", + "▁yo ga", + "▁d ug", + "▁du g", + "▁rest ored", + "▁restore d", + "Dr oid", + "D roid", + "▁P ent", + "▁Pe nt", + "▁Pen t", + "▁ran king", + "▁rank ing", + "mo r", + "m or", + ".~ (\\", + "ograph ical", + "ographic al", + "▁p ian", + "▁pi an", + "▁g ates", + "▁gate s", + "▁ga tes", + "▁с ти", + "▁ст и", + "▁ сти", + "s quare", + "▁im plicit", + "▁impl icit", + "▁G ram", + "▁Gr am", + "▁Gra m", + "▁Apr ès", + "▁Ap rès", + "▁Ass istant", + "▁p ac", + "▁pa c", + "▁P ope", + "▁Po pe", + "▁Pop e", + "г ре", + "▁sc attering", + "▁scatter ing", + "стра тив", + "▁all ocate", + "▁alloc ate", + "▁Man hattan", + "▁а нг", + "▁ан г", + "▁ анг", + "▁inter rupted", + "▁interrupt ed", + "ér ieur", + "éri eur", + "érie ur", + "数 据", + "Sign al", + "Sig nal", + "▁Con tract", + "▁Cont ract", + "▁ Contract", + "ór ia", + "ó ria", + "WI TH", + "W ITH", + "хо дя", + "ход я", + "Ag greg", + "A ggreg", + "cul es", + "cu les", + "cule s", + "c ules", + "J an", + "▁s to", + "▁st o", + "▁ sto", + "▁G PIO", + "▁GP IO", + "▁ GPIO", + "▁ident ifying", + "▁identify ing", + "▁p id", + "▁pi d", + "▁ pid", + "ę p", + "▁di git", + "▁dig it", + "el ia", + "eli a", + "e lia", + "inv oke", + "▁Fl oren", + "▁Flor en", + "▁Flo ren", + "▁sh allow", + "▁shall ow", + "get Class", + "getC lass", + "▁advert is", + "ем ы", + "е мы", + "▁H R", + "▁ HR", + "ym an", + "y man", + "▁C E", + "▁ CE", + "▁sec ured", + "▁secure d", + "▁secur ed", + "▁rel atives", + "▁relative s", + "▁relativ es", + "▁s ob", + "▁so b", + "▁s tab", + "▁st ab", + "▁sta b", + "Trans ition", + "▁w en", + "▁we n", + "▁ wen", + "sh ops", + "shop s", + "▁k ont", + "▁kon t", + "▁ko nt", + "▁h acia", + "▁ha cia", + "H y", + "в ри", + "sh ell", + "she ll", + "s hell", + "▁ant ib", + "▁anti b", + "env ironment", + "environ ment", + "um bs", + "umb s", + "Tr acker", + "Track er", + "Tra cker", + "en tr", + "ent r", + "▁Polit ical", + "ex tract", + "ext ract", + "extra ct", + "extr act", + "=\" {{", + "▁m erc", + "▁me rc", + "▁mer c", + "▁p oc", + "▁po c", + "▁Re set", + "▁Res et", + "▁ Reset", + "▁pur ely", + "▁pure ly", + "▁M ul", + "▁Mu l", + "▁gorge ous", + "▁Î n", + "ri ven", + "riv en", + "rive n", + "r iven", + "▁rom ance", + "▁roman ce", + "▁d av", + "▁da v", + "че ского", + "ér ica", + "éri ca", + "éric a", + "▁tra ject", + "▁a rise", + "▁ar ise", + "▁sw ung", + "▁p ockets", + "▁pocket s", + "▁trad itions", + "▁tradition s", + "▁re ver", + "▁r ever", + "▁rev er", + "▁reve r", + ">> >", + "> >>", + "▁n d", + "▁ nd", + "▁di vis", + "▁div is", + "▁bel oved", + "▁quant ities", + "▁é d", + "▁ éd", + "ien do", + "i endo", + "▁tal ented", + "▁talent ed", + "▁C ad", + "▁Ca d", + "▁В ла", + "▁imm igration", + "▁immigr ation", + "▁ju ris", + "▁jur is", + "▁a er", + "▁e aten", + "▁eat en", + "▁m iejsc", + "▁sum mon", + "pe ople", + "▁g ains", + "▁gain s", + "▁ga ins", + "▁пра во", + "▁restr iction", + "▁restrict ion", + "st ub", + "▁b out", + "▁bo ut", + "▁bou t", + "▁slave ry", + "▁sla very", + "▁comput ation", + "▁ar mor", + "▁arm or", + "▁e k", + "▁ ek", + "▁Muslim s", + "▁co operation", + "▁cooper ation", + "▁enh anced", + "▁enhance d", + "os lav", + "▁ab rupt", + "▁pod cast", + "▁hospital s", + "▁hosp itals", + "нь о", + "▁hot els", + "▁hotel s", + "▁Wik ipedia", + "▁ж ен", + "▁же н", + "▁ жен", + "G LOBAL", + "▁Commun ist", + "an gles", + "ang les", + "angle s", + "▁t high", + "▁th igh", + "▁K irk", + "▁Kir k", + "▁t ends", + "▁ten ds", + "▁tend s", + "▁M ode", + "▁Mod e", + "▁Mo de", + "▁ Mode", + "▁N atur", + "▁Nat ur", + "▁de let", + "▁del et", + "▁po pul", + "▁pop ul", + "▁Ch amber", + "▁Cha mber", + "▁Conserv ative", + "kr ieg", + "k rieg", + "▁Class ic", + "▁die sem", + "▁dies em", + "▁diese m", + "▁em power", + "▁emp ower", + "▁M es", + "▁Me s", + "▁de alt", + "▁deal t", + "▁e stad", + "▁est ad", + "▁esta d", + "▁Se it", + "▁cred its", + "▁credit s", + "sub subsection", + "Inv oke", + "▁phys ician", + "це в", + "ц ев", + "ás a", + "á sa", + "▁g ob", + "▁go b", + "▁R ug", + "▁Ru g", + "▁м іс", + "▁мі с", + "sh aller", + "shal ler", + "shall er", + "▁k ol", + "▁ko l", + "▁ kol", + "▁c ared", + "▁car ed", + "▁care d", + "▁ca red", + "▁of icial", + "no s", + "n os", + "▁j el", + "▁je l", + "▁ jel", + "null able", + "GU I", + "G UI", + "▁r app", + "▁rap p", + "▁ra pp", + "▁An nie", + "▁Ann ie", + "▁st ocks", + "▁stock s", + "▁sto cks", + "▁develop er", + "▁pl acement", + "▁place ment", + "▁plac ement", + "▁ placement", + "(\" <", + "▁l avor", + "▁la vor", + "▁lav or", + "▁acc us", + "Mar t", + "Ma rt", + "M art", + "amer ikan", + "▁sk etch", + "▁sent iment", + "▁а мерикан", + "An chor", + "Mer ge", + "Pe ople", + "▁rend ered", + "▁render ed", + "▁la und", + "▁n ons", + "▁no ns", + "▁non s", + "▁bl ew", + "▁ble w", + "k b", + "ate gor", + "ateg or", + "▁franç aise", + "▁français e", + "KE N", + "K EN", + "method s", + "▁Part icip", + "nost i", + "nos ti", + "n osti", + "▁com merce", + "▁commer ce", + "▁ commerce", + "▁до ма", + "▁d re", + "▁dr e", + "▁t win", + "▁tw in", + "▁ded ic", + "▁U TC", + "▁ UTC", + "We ek", + "▁differ ential", + "▁different ial", + "л ё", + "▁Ch oose", + "▁Cho ose", + "▁\" (", + "▁то м", + "▁ том", + "▁про фе", + "em ark", + "e mark", + "▁fe ared", + "▁fear ed", + "sk o", + "s ko", + "Br anch", + "▁in vention", + "▁inv ention", + "▁invent ion", + "er mine", + "erm ine", + "▁car act", + "▁ca ract", + "ро го", + "р ого", + "lo yd", + "▁ку ль", + "▁ куль", + "▁del icate", + "Or gan", + "▁Im pro", + "▁Imp ro", + "▁r ip", + "▁ri p", + "▁ rip", + "Up dated", + "Update d", + "ul ent", + "ule nt", + "▁o bra", + "▁ob ra", + "s uspend", + "Line s", + "Lin es", + "Li nes", + "L ines", + "▁b anda", + "▁band a", + "▁ban da", + "ot ta", + "ott a", + "o tta", + "▁k ole", + "▁ko le", + "▁kol e", + "il io", + "ili o", + "i lio", + "▁output s", + "▁ outputs", + "est ro", + "estr o", + "AAAA AAAA", + "R UN", + "ne nt", + "nen t", + "n ent", + "▁d ated", + "▁da ted", + "▁dat ed", + "▁date d", + "▁ dated", + "▁s py", + "▁sp y", + "▁c rap", + "▁cr ap", + "▁in coming", + "▁inc oming", + "▁ф ев", + "▁фе в", + "PH Y", + "P HY", + "▁O range", + "▁Or ange", + "▁ob server", + "▁observ er", + "▁observe r", + "▁up stairs", + "ion ed", + "io ned", + "ione d", + "i oned", + "▁a tr", + "▁at r", + "▁ atr", + "igh bor", + "▁expect ation", + "Hi s", + "H is", + "im edia", + "i media", + "com put", + "comp ut", + "▁arg v", + "▁ argv", + "▁ear liest", + "та ли", + "тал и", + "т али", + "мо н", + "м он", + "ol len", + "oll en", + "ra ke", + "r ake", + "▁pat ience", + "ходи т", + "ход ит", + "▁де ка", + "▁bu yers", + "▁buy ers", + "▁buyer s", + "▁Conne ct", + "▁ Connect", + "▁Univers al", + "▁adjust ed", + "▁adj usted", + "im eq", + "ime q", + "el lers", + "ell ers", + "elle rs", + "eller s", + "▁ru in", + "▁Cr usher", + "▁Freder ick", + "ott age", + "otta ge", + "▁com prom", + "▁comp rom", + "▁compr om", + "ia sm", + "ias m", + "i asm", + "wa ve", + "w ave", + "▁encour aging", + "▁be ans", + "▁bean s", + "▁ beans", + "▁per ceived", + "… ]", + "▁gl obe", + "▁glob e", + "▁glo be", + "▁S F", + "▁ SF", + "he rent", + "her ent", + "here nt", + "▁a like", + "▁al ike", + "▁ali ke", + "▁hur ried", + "qu el", + "que l", + "q uel", + "▁mus icians", + "▁music ians", + "▁musician s", + "ar z", + "a rz", + "по в", + "п ов", + "drop down", + "ac l", + "a cl", + "pre view", + "prev iew", + "p review", + "▁under neath", + "ze ś", + "▁fem ales", + "▁female s", + "list ener", + "listen er", + "▁C AN", + "▁CA N", + "▁ CAN", + "▁T ow", + "▁To w", + "▁pe ers", + "▁peer s", + "tl s", + "t ls", + "at ra", + "atr a", + "a tra", + "se nder", + "send er", + "sen der", + "s ender", + "TIME OUT", + "fu rt", + "fur t", + "f urt", + "▁Gu erra", + "{} )", + "{ })", + "▁D urch", + "▁Dur ch", + "▁s ki", + "▁sk i", + "▁ ski", + "il las", + "ill as", + "illa s", + "▁S of", + "▁So f", + "▁Organ ization", + "▁C leveland", + "▁b utt", + "▁but t", + "▁bu tt", + "▁sim ilarly", + "▁similar ly", + "▁assert True", + "▁ assertTrue", + "▁inev itable", + "ne ll", + "nel l", + "n ell", + "▁R af", + "▁Ra f", + "DIS ABLE", + "am ine", + "ami ne", + "amin e", + "a mine", + "▁Com plete", + "▁Comp lete", + "▁ Complete", + "▁be iden", + "▁bei den", + "▁Chall enge", + "Rad io", + "R adio", + "▁Not ice", + "He x", + "H ex", + "▁C uba", + "▁Cub a", + "▁Cu ba", + "▁aug ust", + "▁Philipp ines", + "Mar gin", + "M argin", + "ja l", + "j al", + "gener ator", + "▁t atto", + "▁ta tto", + "▁H em", + "▁He m", + "▁S alt", + "▁Sal t", + "▁Sa lt", + "un ately", + "unate ly", + "▁terr ain", + "▁terra in", + ",\\ ,", + ", \\,", + "гра д", + "▁c rop", + "▁cr op", + "▁cro p", + "Name d", + "Na med", + "N amed", + "▁W onder", + "▁Wo nder", + "▁Won der", + "es sen", + "ess en", + "esse n", + "▁f ist", + "▁fi st", + "▁fis t", + "▁z oom", + "▁zo om", + "▁ zoom", + "пе н", + "п ен", + "▁ru ling", + "▁rul ing", + "un likely", + "as sy", + "ass y", + "or ent", + "ore nt", + "oren t", + "o rent", + "▁g ibt", + "▁gi bt", + "▁A w", + "sim eq", + "s imeq", + "▁r aid", + "▁ra id", + "▁ raid", + "▁Com par", + "▁Comp ar", + "▁ Compar", + "▁free ly", + "▁fre ely", + "▁esp añ", + "▁espa ñ", + "▁py thon", + "▁ python", + "▁diagn osis", + "▁ch ips", + "▁chip s", + "▁chi ps", + "R azor", + "▁V ert", + "▁Ver t", + "▁Ve rt", + "▁ Vert", + "For ward", + "▁P é", + "▁compar able", + "▁anal ys", + "▁analy s", + "St d", + "S td", + "▁Franç ois", + "▁c ó", + "jo s", + "j os", + "▁p eg", + "▁pe g", + "▁ peg", + "CON ST", + "cl usive", + "▁voy age", + "▁Sch l", + "▁Sc hl", + "Group Layout", + "oi se", + "ois e", + "o ise", + "сс е", + "с се", + "▁cr ush", + "▁cru sh", + "▁Die se", + "▁Di ese", + "▁Dies e", + "▁be kan", + "▁bek an", + "ci t", + "c it", + "▁Ein wohner", + "▁L an", + "▁La n", + "▁dress ing", + "▁s olved", + "▁sol ved", + "▁solve d", + "М а", + "▁C hel", + "▁Ch el", + "▁Che l", + "par ed", + "pa red", + "pare d", + "p ared", + "▁se aled", + "▁sea led", + "▁seal ed", + "}) )", + "} ))", + "anc ouver", + "se h", + "s eh", + "ta bles", + "table s", + "tab les", + "t ables", + "▁red dit", + "▁redd it", + "▁ reddit", + "▁m our", + "▁mo ur", + "▁mou r", + "▁clean up", + "▁ cleanup", + "ov ić", + "ovi ć", + "▁Ur ban", + "oc t", + "o ct", + "то ра", + "тор а", + "▁Le gal", + "▁Leg al", + "▁J ur", + "▁Ju r", + "▁N as", + "▁Na s", + "C ity", + "▁un fortunately", + "▁unfortunate ly", + "▁P ER", + "▁PE R", + "▁ PER", + "ma kers", + "make rs", + "maker s", + "m akers", + "▁sig lo", + "▁k in", + "▁ki n", + "▁ kin", + "co des", + "code s", + "cod es", + "c odes", + "ля р", + "NI NG", + "N ING", + "▁C ec", + "▁Ce c", + "▁C T", + "▁ CT", + "▁R acing", + "▁Ra cing", + "da n", + "d an", + "▁He rz", + "▁Her z", + "▁gen ius", + "▁e urop", + "▁eu rop", + "serv let", + "ow ego", + "owe go", + "▁Im agine", + "▁Imp erial", + "▁Imper ial", + "Re gex", + "Reg ex", + "c é", + "HE D", + "H ED", + "det ect", + "з ни", + "io c", + "i oc", + "Anal ysis", + "Analy sis", + "▁* =", + "▁f ever", + "▁fe ver", + "▁Ob viously", + "F oot", + "Line ar", + "Lin ear", + "▁p ró", + "▁pr ó", + "▁satell ite", + "▁B eng", + "▁Be ng", + "▁Ben g", + "bound s", + "b ounds", + "▁J azz", + "▁Ja zz", + "▁C urt", + "▁Cur t", + "▁Cu rt", + "▁поли ти", + "▁b ild", + "▁bi ld", + "▁bil d", + "▁ bild", + "▁\" \");", + "▁\"\" );", + "▁\"\") ;", + "▁document ary", + "▁gr asp", + "▁gra sp", + "▁gras p", + "▁d la", + "▁dl a", + "TR A", + "T RA", + "▁read ily", + "To r", + "T or", + "C ACHE", + "▁Const ruction", + "▁Construct ion", + "▁d ía", + "да т", + "д ат", + "▁G rey", + "▁Gr ey", + "▁Gre y", + "run ner", + "le ading", + "▁co oked", + "▁cook ed", + "ro log", + "rol og", + "r olog", + "▁annoy ing", + "DE LETE", + "amer ican", + "▁Niger ia", + "▁d ai", + "▁da i", + "▁ dai", + "▁sac rific", + "▁serv ant", + "▁s kb", + "▁sk b", + "▁ skb", + "▁b arg", + "▁bar g", + "▁ba rg", + "pix el", + "p ixel", + "In ject", + "ca ched", + "cache d", + "c ached", + "▁cou pled", + "▁couple d", + "▁coup led", + "un gle", + "ung le", + "pro b", + "pr ob", + "p rob", + ">{ @", + "ла го", + "default s", + "▁por trait", + "▁port rait", + "▁d ental", + "▁den tal", + "▁dent al", + "▁d estro", + "▁dest ro", + "▁r ue", + "▁ru e", + "▁hy brid", + "▁ й", + "▁CO MP", + "▁COM P", + "▁ COMP", + "▁B ent", + "▁Be nt", + "▁Ben t", + "Com pare", + "Comp are", + "Compar e", + "bo th", + "bot h", + "b oth", + "kl ahoma", + "ais er", + "ai ser", + "aise r", + "a iser", + "Su re", + "Sur e", + "S ure", + "▁s olving", + "▁sol ving", + "▁l ista", + "▁li sta", + "▁list a", + "▁ lista", + "▁у чи", + "▁Ev ans", + "▁Eva ns", + "▁f usion", + "▁fus ion", + "▁compl aint", + "▁complain t", + "H P", + "He ap", + "al ways", + "M gr", + "▁appro x", + "▁ approx", + "display style", + "lo rd", + "lor d", + "l ord", + "in sn", + "ins n", + "▁Fe ature", + "▁ Feature", + "RP C", + "R PC", + "▁v et", + "▁ve t", + "▁ vet", + "К а", + "▁kil omet", + "▁kilom et", + "▁deliver ing", + "▁const itution", + "sh ine", + "ле к", + "▁го род", + "▁горо д", + "▁prob able", + "▁run ner", + "▁ runner", + "hr en", + "hre n", + "h ren", + "▁N ep", + "▁Ne p", + "▁over night", + "pr ead", + "pre ad", + "p read", + "л та", + "фор ма", + "CL O", + "C LO", + "ie sa", + "ies a", + "i esa", + "▁object ives", + "▁objective s", + "con tract", + "cont ract", + "contr act", + "EX P", + "▁col ours", + "▁colour s", + "xi co", + "xic o", + "x ico", + "C lean", + "▁light ly", + "▁scen arios", + "▁scenario s", + "▁qu arters", + "▁quarter s", + "▁quart ers", + "▁quar ters", + "▁ quarters", + "▁D ear", + "▁De ar", + "▁l uc", + "▁lu c", + "▁app et", + "▁ap pet", + "▁appe t", + "▁de port", + "▁dep ort", + "Sa fe", + "▁me nos", + "▁men os", + "▁Paul o", + "▁Pa ulo", + "CI AL", + "C IAL", + "ці в", + "ц ів", + "▁R oc", + "▁Ro c", + "▁c aring", + "▁car ing", + "▁ca ring", + "▁elect ro", + "▁de cember", + "▁dec ember", + "▁dece mber", + "▁Phil osoph", + "▁col ored", + "▁color ed", + "▁ colored", + "it sch", + "its ch", + "ropol itan", + "os ti", + "ost i", + "▁N ut", + "▁Nu t", + "▁consecut ive", + "Pe er", + "ar ness", + "arn ess", + "▁ż e", + "▁ że", + "▁A round", + "▁Ar ound", + "af ka", + "▁d io", + "▁di o", + "ci p", + "c ip", + "▁to ys", + "▁toy s", + "cr o", + "c ro", + "▁m iser", + "▁mis er", + "▁mi ser", + "▁mise r", + "check box", + "▁F isher", + "▁Fish er", + "▁gover ned", + "▁govern ed", + "▁h á", + "▁En able", + "▁ Enable", + "▁t rivial", + "▁occup ation", + "ro rs", + "ror s", + "r ors", + "▁l av", + "▁la v", + "▁ lav", + "▁m ou", + "▁mo u", + "▁b ord", + "▁bo rd", + "▁bor d", + "ли ч", + "Ro om", + "R oom", + "') \r", + "' )\r", + "▁art ic", + "▁m ientras", + "ch air", + "cha ir", + "uation s", + "u ations", + "▁comm ented", + "▁comment ed", + "▁trigger ed", + "Can not", + "C annot", + "▁Marc us", + "▁p unct", + "▁pun ct", + "▁achie vement", + "▁achieve ment", + "е ди", + "ext ensions", + "extension s", + "ad ers", + "ade rs", + "ader s", + "a ders", + "jo urs", + "jour s", + "j ours", + "ir lines", + "irl ines", + "▁со стоя", + "V IEW", + "▁Nap ole", + "Conf irm", + "▁por que", + "........ ........", + "▁LI ABILITY", + "Wall et", + "W allet", + "Sub ject", + "al gorithm", + "▁tr iple", + "▁tri ple", + "▁trip le", + "ru b", + "r ub", + "▁se cur", + "▁sec ur", + "▁hand some", + "▁hands ome", + "▁d od", + "▁do d", + "r ès", + "ac ja", + "ch od", + "cho d", + "н ва", + "es ar", + "esa r", + "an chor", + "anc hor", + "anch or", + "▁Soph ie", + "▁Украї ни", + "Up per", + "am ous", + "amo us", + "Fe atures", + "Feature s", + "▁б ли", + "▁ бли", + "Supp ress", + "Sup press", + "▁kil om", + "▁Z u", + "▁belong ed", + "▁Red dit", + "▁pro ces", + "▁proc es", + "▁с тар", + "▁ста р", + "▁ст ар", + "▁F est", + "▁Fe st", + "/ %", + "▁P am", + "▁Pa m", + "st orm", + "sto rm", + "W W", + "P aul", + "▁t ales", + "▁tal es", + "▁ta les", + "▁tale s", + "▁рай она", + "▁райо на", + "▁район а", + "▁spread ing", + "▁s ched", + "▁sc hed", + "▁sch ed", + "▁sche d", + "▁ sched", + "le ased", + "lease d", + "Non Null", + "▁High way", + "▁Re serve", + "▁Res erve", + "▁c ater", + "▁cat er", + "▁ca ter", + "▁t ire", + "▁ti re", + "▁tir e", + "▁por ch", + "qu ier", + "US A", + "U SA", + "▁Sw iss", + "▁ È", + "▁br ave", + "▁bra ve", + "▁explos ion", + "l r", + "▁class ified", + "Ab out", + "▁P ict", + "▁Pic t", + "▁Pi ct", + "▁Dub lin", + "▁separ ately", + "▁separate ly", + "▁bank ing", + "▁ban king", + "▁Christian ity", + "mi gr", + "m igr", + "Ro b", + "R ob", + "се р", + "с ер", + "▁el f", + "▁ elf", + "▁employ ers", + "▁employer s", + "▁S low", + "▁Sl ow", + "▁j uli", + "▁ju li", + "▁jul i", + "west ern", + "w estern", + "▁anal yst", + "▁analy st", + "▁analys t", + "ob serv", + "obs erv", + "▁N ice", + "▁Nic e", + "▁Ni ce", + "▁G C", + "▁ GC", + "▁Let ter", + "▁ha rass", + "▁har ass", + "User name", + "▁A unt", + "▁Au nt", + "▁с ент", + "Su p", + "S up", + "IC ES", + "ICE S", + "RE NT", + "R ENT", + "rat io", + "r atio", + "▁Мо ск", + "▁an gles", + "▁ang les", + "▁angle s", + "▁angl es", + "▁ angles", + "▁l lev", + "▁ll ev", + "_ *", + "▁n it", + "▁ni t", + "▁ nit", + "▁w reck", + "▁pat rol", + "▁loyal ty", + "▁n ationale", + "▁nat ionale", + "▁national e", + "▁nation ale", + "go m", + "g om", + "}$ -", + "} $-", + "▁dis pute", + "▁disput e", + "▁disp ute", + "▁r us", + "▁ru s", + "▁ rus", + "▁П рез", + "▁Пре з", + "▁Indust rial", + "▁dem ocratic", + "▁democr atic", + "b w", + "li mp", + "lim p", + "l imp", + "ur bed", + "urb ed", + "▁mie jsce", + "▁miejsc e", + "ру д", + "▁t ex", + "▁te x", + "▁ tex", + "▁develop ments", + "▁development s", + "▁B right", + "▁Br ight", + "▁Brig ht", + "▁var ying", + "▁va rying", + "▁vary ing", + "fa ct", + "fac t", + "f act", + "▁Port al", + "▁Por tal", + "as is", + "asi s", + "a sis", + "▁горо да", + "▁город а", + "▁cre ativity", + "▁creat ivity", + ")) ))", + "))) )", + ") )))", + ".\" ;", + ". \";", + "ie ux", + "ieu x", + "▁prov isions", + "▁provision s", + "uv e", + "u ve", + "La ng", + "L ang", + "miss ing", + "ра т", + "р ат", + "ph ony", + "▁out line", + "pa s", + "p as", + "el m", + "e lm", + "mon itor", + "TC P", + "T CP", + "ka t", + "k at", + "uc ed", + "uce d", + "u ced", + "\\\" ,", + "\\ \",", + "yn a", + "y na", + "ра бо", + "раб о", + "oc ate", + "oca te", + "▁c ares", + "▁car es", + "▁care s", + "▁ca res", + "▁f ins", + "▁fin s", + "▁fi ns", + "▁he ap", + "▁ heap", + "▁small est", + "äch st", + "▁I X", + "▁ IX", + "re cv", + "rec v", + "key word", + "▁at tra", + "▁att ra", + "▁attr a", + "▁sel bst", + "Un expected", + "Une xpected", + "Sm all", + "▁насе ље", + "▁H us", + "▁Hu s", + "Enc oder", + "Encode r", + "▁un set", + "▁uns et", + "▁home less", + "▁hom eless", + "▁Johann es", + "▁U RI", + "▁ URI", + "ant age", + "anta ge", + "▁in hib", + "▁appreci ated", + "▁appreciate d", + "ie lte", + "iel te", + "ielt e", + "i elte", + "▁st ays", + "▁stay s", + "▁sta ys", + "▁alle ged", + "▁alleg ed", + "▁c oding", + "▁co ding", + "▁cod ing", + "▁tv å", + "pipe line", + "p ipeline", + "▁W or", + "▁Wo r", + "File Path", + "▁accept ing", + "▁Ex cell", + "▁L uther", + "▁Lu ther", + "▁Friend s", + "▁c urt", + "▁cur t", + "▁cu rt", + "▁' $", + "▁ '$", + "▁tight ly", + "▁cz ę", + "▁un necessary", + "▁F ed", + "▁Fe d", + "▁А нд", + "▁Ан д", + "▁H P", + "▁ HP", + "▁String Builder", + "en burg", + "' (", + "vm a", + "v ma", + "▁Ab raham", + "W L", + "▁Re ference", + "▁Refer ence", + "▁ Reference", + "J o", + "Bl ob", + "Blo b", + "▁H ugh", + "▁Hug h", + "▁Hu gh", + "▁Bul gar", + "MESS AGE", + "з во", + "▁avoid ed", + "▁po ems", + "▁poem s", + "▁с ы", + "▁ сы", + "▁O pp", + "▁Op p", + "av irus", + "avi rus", + "Pre view", + "Prev iew", + "P review", + "▁k er", + "▁ke r", + "▁ ker", + "ue va", + "u eva", + "fl ix", + "▁char ging", + "▁charg ing", + "▁motiv ated", + "▁O rd", + "▁Or d", + "▁ Ord", + "▁av eva", + "▁ave va", + "x l", + "▁flex ibility", + "ag na", + "agn a", + "▁rac ism", + "d h", + "▁b aking", + "▁ba king", + "F riend", + "ble r", + "bl er", + "b ler", + "▁Log ger", + "▁ Logger", + "Te n", + "T en", + "nav igation", + "▁att achment", + "▁attach ment", + "▁ attachment", + "▁b ajo", + "▁ba jo", + "▁pr icing", + "▁pri cing", + "▁T ip", + "▁Ti p", + "▁ Tip", + "da r", + "d ar", + "G G", + "To ols", + "Tool s", + "Too ls", + "T ools", + "vol ution", + "v olution", + "am as", + "ama s", + "a mas", + "▁b ibli", + "▁adapt ed", + "ox y", + "o xy", + "▁F reedom", + "▁Free dom", + "ri co", + "ric o", + "r ico", + "▁coll apsed", + "▁collapse d", + "z m", + "pl o", + "p lo", + "▁c ô", + "▁r t", + "▁ rt", + "än ger", + "äng er", + "änge r", + "▁D R", + "▁ DR", + "▁Bit coin", + "go w", + "g ow", + "▁ch ez", + "▁che z", + "▁ chez", + "▁ot ro", + "▁te il", + "▁ teil", + "ла га", + "▁St ars", + "▁Star s", + "▁Sta rs", + "▁invest ing", + "▁a board", + "▁ab oard", + "▁f lights", + "▁fl ights", + "▁flight s", + "▁genu inely", + "▁genuine ly", + "▁prom ising", + "Rot ation", + "O cc", + "▁su oi", + "▁suo i", + "string ify", + "ac ies", + "aci es", + "a cies", + "▁G round", + "▁Gr ound", + "▁Gro und", + "▁sequ ences", + "▁sequence s", + "▁c ure", + "▁cur e", + "▁cu re", + "out ine", + "▁! !", + "▁ !!", + "▁G ay", + "▁Ga y", + "▁garden s", + "▁gard ens", + "▁G las", + "▁Gl as", + "▁Tai wan", + "reg istry", + "▁# {", + "▁ #{", + "▁ins pection", + "▁insp ection", + "▁inspect ion", + "Te ll", + "T ell", + "▁` ${", + "p matrix", + "▁reg ulation", + "▁regul ation", + "fin ish", + "▁Ed ge", + "▁ Edge", + "Sp rite", + "S prite", + "▁Conf eder", + "▁immigr ants", + "▁elder ly", + "um ed", + "ume d", + "u med", + "▁Quest ion", + "▁ Question", + "Gate way", + "fo ny", + "fon y", + "f ony", + "ît re", + "î tre", + "▁co sm", + "▁cos m", + "Ro und", + "R ound", + "▁ign oring", + "▁ignor ing", + "▁K i", + "▁sens itivity", + "âte au", + "ât eau", + "▁engine ers", + "▁engineer s", + "▁cor rel", + "▁corre l", + "ir teen", + "irt een", + "▁Sw itzerland", + "▁inher it", + "▁ inherit", + "wo r", + "w or", + "▁mid night", + "▁P un", + "▁Pu n", + "ak te", + "akt e", + "a kte", + "Dis able", + "▁es per", + "▁esp er", + "▁not ation", + "▁ notation", + "▁Univers idad", + "so l", + "s ol", + "de rn", + "der n", + "d ern", + "in ge", + "ing e", + "▁inv itation", + ")} }", + ") }}", + "▁ â", + "▁ess ays", + "▁essay s", + "ar med", + "arm ed", + "ch sel", + "chs el", + "▁не го", + "▁ него", + "▁confirm ation", + "un ity", + "unit y", + "uni ty", + "▁Br other", + "▁Bro ther", + "▁ Є", + "ni ce", + "nic e", + "n ice", + "▁S ue", + "▁Su e", + "▁t ray", + "▁tr ay", + "▁tra y", + "ро и", + "C ookie", + "▁Feder ation", + "IC T", + "I CT", + "▁p éri", + "stud ent", + "▁V ent", + "▁Ven t", + "▁Ve nt", + "K K", + "ST EM", + "aw k", + "▁re un", + "▁pe oples", + "▁people s", + "io res", + "ior es", + "iore s", + "i ores", + "ou bt", + "▁St age", + "▁Sta ge", + "▁ Stage", + "▁c harm", + "▁ch arm", + "▁char m", + "▁cha rm", + "ie ur", + "ieu r", + "i eur", + "▁util ize", + "▁utiliz e", + "▁d istribute", + "▁dist ribute", + "▁distribut e", + "▁g otta", + "▁go tta", + "▁got ta", + "▁block ing", + "H ot", + "br ew", + "bre w", + "b rew", + "▁b onds", + "▁bon ds", + "▁bond s", + "le af", + "Pro te", + "Pr ote", + "P rote", + "▁d ice", + "▁di ce", + "▁dic e", + "▁Nor man", + "▁Norm an", + "▁о кт", + "▁ок т", + "▁in spir", + "▁insp ir", + "Pr iv", + "P riv", + "▁P uerto", + "▁то ва", + "RS T", + "R ST", + "▁s f", + "▁ sf", + "▁qu ale", + "▁qual e", + "ni ck", + "nic k", + "n ick", + "▁sup press", + "▁supp ress", + "ча т", + "ч ат", + "▁H ello", + "▁Hel lo", + "▁Hell o", + "▁ Hello", + "▁crow ded", + "▁crowd ed", + "hba r", + "h bar", + "▁lo ads", + "▁load s", + "▁ loads", + "▁cor rection", + "▁correct ion", + "▁corre ction", + "ad just", + "adj ust", + "▁E state", + "▁Est ate", + "▁Esta te", + "text sc", + "▁cool ing", + "iv eau", + "ive au", + "▁bet ting", + "==== ========", + "======== ====", + "re mark", + "rem ark", + "r emark", + "▁im plications", + "▁impl ications", + "▁p oz", + "▁po z", + "ün g", + "ü ng", + "▁reg ards", + "▁regard s", + "▁a mid", + "▁am id", + "▁habit antes", + "G I", + "▁F ou", + "▁Fo u", + "▁j ar", + "▁ja r", + "▁ jar", + "▁requ iring", + "▁D rupal", + "▁Dru pal", + "▁li ability", + "cz as", + "c zas", + "▁l yrics", + "▁ly rics", + "▁N ort", + "▁No rt", + "▁Nor t", + "si l", + "s il", + "▁M ey", + "▁Me y", + "UN IT", + "ва ния", + "f uture", + "hi r", + "h ir", + "CA L", + "C AL", + "LAB EL", + "▁S weet", + "▁stat ue", + "bor ne", + "born e", + "b orne", + "Not ify", + "▁her itage", + "▁d orm", + "▁do rm", + "▁l ever", + "▁le ver", + "▁lev er", + "▁mut tered", + "} &", + "▁inter mediate", + "▁Wat son", + "▁view ing", + "▁vie wing", + "kt or", + "k tor", + "enti eth", + "xx x", + "x xx", + "at u", + "a tu", + "▁Inst all", + "▁ Install", + "Cont in", + "▁t oute", + "▁to ute", + "▁tou te", + "▁tout e", + "▁P T", + "▁ PT", + "▁u ri", + "▁ur i", + "▁ uri", + "Call ed", + "Cal led", + "C alled", + "▁O FF", + "▁OF F", + "▁ OFF", + "ig lia", + "ic hi", + "ich i", + "i chi", + "с ни", + "V o", + "▁exhib it", + "▁asym pt", + "▁G ulf", + "л ли", + "do min", + "dom in", + "d omin", + "▁départ ement", + "mi l", + "m il", + "▁B ez", + "▁Be z", + "▁l ately", + "▁late ly", + "▁lat ely", + "▁def ining", + "▁defin ing", + "▁E L", + "▁ EL", + "omorph ic", + "▁f ebru", + "▁fe bru", + "▁febr u", + "IS TER", + "IST ER", + "I STER", + "res olved", + "resolve d", + "те й", + "т ей", + "▁S pect", + "▁Sp ect", + "▁Spec t", + "▁Spe ct", + "▁sem pre", + "▁Se pt", + "▁Sep t", + "▁cl earing", + "▁cle aring", + "▁clear ing", + "▁diam eter", + "in do", + "ind o", + "▁soc cer", + "▁D CHECK", + "▁DC HECK", + "vo te", + "v ote", + "▁n omin", + "▁no min", + "▁nom in", + "Type d", + "Ty ped", + "Typ ed", + "Miss ing", + "W as", + "▁Cent ury", + "▁direct ors", + "▁dire ctors", + "▁director s", + "▁mode rate", + "▁moder ate", + "▁Ill uminate", + "▁ Illuminate", + "▁челове к", + "▁B apt", + "▁Ba pt", + "▁Qu ant", + "▁ Quant", + "▁tre ating", + "▁treat ing", + "ag i", + "a gi", + "Si l", + "S il", + "ring e", + "rin ge", + "r inge", + "ł ą", + "el lan", + "ell an", + "ella n", + "▁f ino", + "▁fin o", + "▁fi no", + "Capt ure", + "C apture", + "▁S ic", + "▁Si c", + "▁st amp", + "▁sta mp", + "▁stam p", + "▁B uen", + "▁Bu en", + "▁seg undo", + "▁in verse", + "▁d up", + "▁du p", + "▁ dup", + "▁br oker", + "▁bro ker", + "▁broke r", + "▁search ed", + "▁sear ched", + "be ans", + "bean s", + "▁A BC", + "▁AB C", + "is ha", + "ish a", + "i sha", + "▁Lin ked", + "▁Link ed", + "▁ Linked", + "▁Nich olas", + "▁Sw edish", + "he mal", + "hem al", + "▁E M", + "▁ EM", + "▁j ego", + "▁je go", + "че ский", + "чески й", + "lo t", + "l ot", + "▁dis cret", + "▁disc ret", + "▁discre t", + "▁E g", + "pi ck", + "pic k", + "p ick", + "am on", + "amo n", + "a mon", + "▁Rail way", + "ка р", + "к ар", + "▁nav igate", + "▁navig ate", + "▁Comm ander", + "▁Command er", + "▁disappe ar", + "▁con gress", + "▁congr ess", + "▁graph ic", + "sp r", + "s pr", + "FLO AT", + "▁S erial", + "▁Se rial", + "▁Ser ial", + "▁ Serial", + "▁я нва", + "so cial", + "soc ial", + "s ocial", + "bu ch", + "b uch", + "▁se al", + "▁sea l", + "▁c ement", + "▁ce ment", + "▁Y e", + "ot ti", + "ott i", + "o tti", + "▁The od", + "remove Class", + "▁Jul ie", + "▁Ju lie", + "▁Juli e", + "▁gr öß", + "ST REAM", + "▁G B", + "▁ GB", + "▁Ben ef", + "▁Mat rix", + "▁ Matrix", + "▁ke ine", + "▁cont inent", + "▁contin ent", + "▁ja ar", + "DA I", + "D AI", + "▁S equ", + "▁Se qu", + "▁ Sequ", + "kre is", + "▁c rown", + "▁cr own", + "▁crow n", + "▁cro wn", + "Init ialize", + "Initial ize", + "ax y", + "a xy", + "▁C IA", + "▁int end", + "▁inte nd", + "▁b ub", + "▁bu b", + "▁mask s", + "▁mas ks", + "▁sit uated", + "▁situ ated", + "▁E du", + "▁Ed u", + "▁particip ating", + "ше й", + "ш ей", + "_{ -", + "_ {-", + "▁Tele vision", + "▁pre ferences", + "▁prefer ences", + "▁preference s", + "▁D rop", + "▁Dr op", + "▁ Drop", + "re view", + "rev iew", + "▁vi olation", + "▁viol ation", + "▁ch rist", + "▁chr ist", + "q q", + "▁M yst", + "▁My st", + "comm ands", + "command s", + "▁prim itive", + "ill ance", + "▁r anging", + "▁ran ging", + "▁rang ing", + "▁Adv anced", + ") &", + "▁О б", + "▁sub str", + "▁subst r", + "▁subs tr", + "▁ substr", + "▁clos ure", + "▁clo sure", + "▁ closure", + "tw itter", + "ne z", + "n ez", + "▁pr zed", + "▁prz ed", + "▁prze d", + "▁mer ged", + "▁merge d", + "ur os", + "uro s", + "u ros", + "▁j er", + "▁je r", + "▁ jer", + "▁_ (", + "▁ _(", + "ar an", + "ara n", + "a ran", + "▁P atri", + "▁Pat ri", + "▁Pa tri", + "▁T un", + "▁Tu n", + "U K", + "il iation", + "ili ation", + "▁Ke ith", + "Own Property", + "op sis", + "ops is", + "Ma d", + "M ad", + "▁def ence", + "A ir", + "=$ {", + "= ${", + "cript ors", + "criptor s", + "So m", + "S om", + "▁ ±", + "▁HA VE", + "~~~~ ~~~~", + "▁be aten", + "▁beat en", + "▁int imate", + "▁intim ate", + "op ic", + "o pic", + "▁p řed", + "▁př ed", + "Sh op", + "S hop", + "Table s", + "Tab les", + "T ables", + "▁S I", + "▁ SI", + "re name", + "ren ame", + "rena me", + "r ename", + "▁product ive", + "rib ly", + "r ibly", + "▁L uck", + "▁Lu ck", + "▁Luc k", + "▁kl ub", + "}} ^{", + "}}^ {", + "} }^{", + "▁F ish", + "▁Fi sh", + "PR I", + "P RI", + "en ario", + "ena rio", + "▁pse ud", + "Or d", + "O rd", + "▁quel ques", + "▁D od", + "▁Do d", + "▁p unto", + "▁pun to", + "▁punt o", + "se nal", + "sen al", + "▁Br others", + "▁Bro thers", + "▁Brother s", + "▁diab etes", + "P aint", + "▁person as", + "▁persona s", + "в ър", + "▁n ep", + "▁ne p", + "▁El len", + "▁Ell en", + "▁Elle n", + "▁h ä", + "cr tc", + "c rtc", + "▁frustr ation", + ". ^{[", + "▁s printf", + "▁sprint f", + "▁ sprintf", + "+ -", + "En code", + "Enc ode", + "▁насе лення", + "Draw able", + "▁b ore", + "▁bo re", + "▁bor e", + "▁E ld", + "▁El d", + "те т", + "т ет", + "T ick", + "ar ator", + "ara tor", + "▁Fin ance", + "▁agric ultural", + ")^ {-", + ")^{ -", + ") ^{-", + "may be", + "Sche dule", + "▁[ …]", + "et ection", + "ete ction", + "ль ного", + "льно го", + "▁he els", + "▁En joy", + "Sy s", + "S ys", + "orsz ág", + "CONT ROL", + "cc cc", + "▁D ictionary", + "▁ Dictionary", + "Ne ed", + "N eed", + "▁He aven", + "▁vess els", + "▁vessel s", + "ec ycle", + "e cycle", + "ti es", + "t ies", + "▁e nde", + "▁en de", + "▁end e", + "▁ ende", + "SI NG", + "S ING", + "De scribe", + "Desc ribe", + "▁Pub lished", + "▁Publish ed", + "▁win ds", + "▁wind s", + "neh men", + "▁D ES", + "▁DE S", + "Hor izontal", + "▁L ost", + "▁Los t", + "▁Lo st", + "-- -----------", + "---- ---------", + "-------- -----", + "--- ----------", + "------------ -", + "----- --------", + "---------- ---", + "------ -------", + "--------- ----", + "------- ------", + "----------- --", + "- ------------", + "▁p x", + "▁ px", + "}( {\\", + "} ({\\", + "▁Hein rich", + "oms nitt", + "ho s", + "h os", + "Ro ll", + "R oll", + "tor ch", + "▁equ ity", + "▁eq uity", + "▁collect ing", + "▁l ifting", + "▁lif ting", + "▁lift ing", + "sub figure", + "Ne ver", + "N ever", + "▁L ength", + "▁Le ngth", + "▁ Length", + "▁w inners", + "▁win ners", + "▁winner s", + "▁U SD", + "▁US D", + "▁st esso", + "▁а бо", + "▁al tri", + "▁alt ri", + "▁produ cers", + "▁produce rs", + "▁producer s", + "mon s", + "mo ns", + "m ons", + "▁Pop ular", + "Com b", + "Co mb", + "C omb", + "ab lo", + "abl o", + "a blo", + "RE SET", + "RES ET", + "т ва", + "Over lay", + "▁id iot", + "▁idi ot", + "ex ist", + "Be havior", + "UB LE", + "ier re", + "i erre", + "mine craft", + "▁f os", + "▁fo s", + "▁encuent ra", + "▁scream ed", + "▁polynom ial", + "▁c one", + "▁con e", + "▁co ne", + "▁c ited", + "▁cit ed", + "▁ci ted", + "▁president e", + "▁presid ente", + "▁re sign", + "▁res ign", + "▁y elled", + "▁i k", + "▁ ik", + "Pl us", + "▁Ми ха", + "▁The me", + "▁Th eme", + "▁ Theme", + "▁re li", + "▁r eli", + "▁rel i", + "ne m", + "n em", + "▁a men", + "▁am en", + "▁ amen", + "▁ Ј", + "Th anks", + "Thank s", + "Than ks", + "▁al umin", + "▁sh elf", + "▁shel f", + "!\" );", + "! \");", + "append Child", + "▁l ogs", + "▁lo gs", + "▁log s", + "▁ logs", + "▁re gex", + "▁reg ex", + "▁ regex", + "▁p unk", + "▁pun k", + "CO RE", + "▁b orders", + "▁border s", + "▁bord ers", + "▁bor ders", + "▁Requ ired", + "▁ Required", + "▁f law", + "▁fl aw", + "▁cin ema", + "▁v í", + "▁ ví", + "▁ab ortion", + "▁abort ion", + "jour nal", + "j ournal", + "in itions", + "init ions", + "inition s", + "state ment", + "stat ement", + "▁o urs", + "▁our s", + "▁ou rs", + "▁ ours", + "ó t", + "▁Tur ner", + "▁Turn er", + "in us", + "ev es", + "eve s", + "e ves", + "▁magazine s", + "▁magaz ines", + "… …", + "la ce", + "l ace", + "sl ider", + "slide r", + "▁l ocate", + "▁loc ate", + "▁des arroll", + "P an", + "To m", + "T om", + "▁Land es", + "▁Lan des", + "ol ia", + "oli a", + "o lia", + "▁u nm", + "▁un m", + "▁Sen ator", + "▁ad minister", + "▁admin ister", + "▁ко ји", + "▁' {", + "▁) {", + "▁ ){", + "▁G olf", + "▁Gol f", + "▁g ele", + "▁ge le", + "▁gel e", + "▁d rank", + "▁dr ank", + "pos ing", + "po sing", + "p osing", + "▁en semble", + "he ap", + "sign ature", + "то й", + "ци й", + "scri ber", + "scr iber", + "scribe r", + "scrib er", + "▁ch amp", + "▁cha mp", + "ni o", + "n io", + "la yers", + "lay ers", + "layer s", + "▁tr ump", + "▁mod al", + "▁mo dal", + "▁ modal", + "on ces", + "once s", + "че ння", + "чен ня", + "▁C ort", + "▁Co rt", + "▁Cor t", + "▁sun light", + "▁M use", + "▁Mus e", + "▁Mu se", + "ém ent", + "é ment", + "▁curios ity", + "▁v r", + "▁ vr", + "O ct", + "yl on", + "y lon", + "▁rel ativ", + "st y", + "s ty", + "] /", + "az u", + "a zu", + "▁U SS", + "▁US S", + "▁person a", + "▁pers ona", + "Me n", + "M en", + "▁w ides", + "▁wide s", + "▁wid es", + "▁K as", + "▁Ka s", + "ic ies", + "ici es", + "i cies", + "▁C off", + "▁Co ff", + "▁con solid", + "▁cons olid", + "▁inter active", + "▁interact ive", + "op ing", + "o ping", + "La nd", + "L and", + "▁energ ies", + "▁independ ently", + "▁independent ly", + "inner HTML", + "Requ ire", + "Re quire", + "▁abs urd", + "▁IN FO", + "▁ INFO", + "▁b und", + "▁bu nd", + "▁ bund", + "anz ös", + "▁G ent", + "▁Ge nt", + "▁Gen t", + "▁scholar s", + "▁schol ars", + "▁C reated", + "▁Create d", + "▁Creat ed", + "▁Cre ated", + "▁ Created", + "▁mar ine", + "▁mari ne", + ".. .'", + "... '", + "EN V", + "E NV", + "ach te", + "acht e", + "a chte", + "am ents", + "ament s", + "amen ts", + "a ments", + "▁tr ucks", + "▁truck s", + "▁re wards", + "▁reward s", + "og s", + "o gs", + "Gr een", + "Gre en", + "G reen", + "▁n ä", + "▁inher ited", + "▁inherit ed", + "im ated", + "imate d", + "ima ted", + "imat ed", + "▁F REE", + "▁FR EE", + "▁ FREE", + "▁ext ens", + "da g", + "d ag", + "▁g low", + "▁gl ow", + "▁glo w", + "ar di", + "ard i", + "N F", + "▁evalu ated", + "▁evaluate d", + "▁eval uated", + "▁o ps", + "▁op s", + "▁ ops", + "▁cle aned", + "▁clean ed", + "▁Prov ince", + "▁Provinc e", + "ha bil", + "hab il", + "h abil", + "гра фі", + "▁T CP", + "▁ TCP", + "▁я кі", + "▁як і", + "▁de ce", + "▁dec e", + "▁cont empl", + "▁acquis ition", + "}) $.", + "})$ .", + "} )$.", + "=\" -", + "▁se ctors", + "▁sector s", + "▁sect ors", + ":: <", + "u ß", + "▁trab aj", + "th an", + "tha n", + "t han", + "▁S ta", + "▁St a", + "Mem bers", + "Member s", + "▁r v", + "▁ rv", + ")^ {\\", + ")^{ \\", + ") ^{\\", + "mit t", + "mi tt", + "m itt", + "▁W ang", + "▁Wa ng", + "▁W end", + "▁We nd", + "▁G lass", + "▁Gl ass", + "▁Glas s", + "▁t xt", + "▁tx t", + "▁ txt", + "▁Cam eron", + "ie ls", + "iel s", + "i els", + "▁im mer", + "▁imm er", + "▁ immer", + "▁насе ления", + ".. . /", + "▁ро ди", + "▁ роди", + "▁sophistic ated", + "▁R he", + "▁Rh e", + "us sy", + "uss y", + "▁Sy ria", + "▁Car oline", + "▁Carol ine", + "riter ion", + "ér c", + "é rc", + "Lo ve", + "L ove", + "▁cy cles", + "▁cycle s", + "▁cycl es", + "▁Ter ms", + "▁Term s", + "▁med ieval", + "▁medi eval", + "ь я", + "▁m issions", + "▁miss ions", + "▁mission s", + "Har d", + "Ha rd", + "H ard", + "▁rég ion", + "▁Ph oenix", + "De ep", + "▁sam pling", + "▁dismiss ed", + "prop ri", + "p ropri", + "▁jud ges", + "▁judge s", + "▁judg es", + "ał a", + "a ła", + "ul os", + "ulo s", + "u los", + "▁L ion", + "▁Li on", + "▁loc als", + "▁local s", + "neg ative", + "ogen eous", + "ogene ous", + "▁A pi", + "▁Ap i", + "▁ Api", + "▁d ici", + "▁di ci", + "▁dic i", + "▁а пре", + "▁author ized", + "▁ authorized", + "ze rw", + "zer w", + "▁p g", + "▁ pg", + "▁A WS", + "▁key word", + "▁ keyword", + "▁entrepre neur", + "▁п рое", + "▁про е", + "▁V ancouver", + "it ating", + "ita ting", + "itat ing", + "F ast", + "▁acknowled ged", + "▁acknowledge d", + "▁tour ist", + "▁tou rist", + "▁G rid", + "▁Gr id", + "▁ Grid", + "▁En try", + "▁Ent ry", + "▁ Entry", + "▁g ebru", + "▁ge bru", + "▁geb ru", + "sa t", + "s at", + "ber ger", + "berg er", + "▁T F", + "▁ TF", + "▁m t", + "▁ mt", + "▁Mar cel", + "▁Marc el", + "▁Tw enty", + "▁ ”", + "{} {", + "{ }{", + "hi nt", + "hin t", + "h int", + "▁an onymous", + "Cam p", + "C amp", + "▁** _", + "By Comparator", + "U C", + "▁t ö", + "Event Handler", + "▁t ours", + "▁to urs", + "▁tour s", + "▁tou rs", + "▁lon ely", + "▁Sum mary", + "▁ Summary", + "st ick", + "s tick", + "All owed", + "Allow ed", + "лі в", + "л ів", + "▁B rew", + "▁Br ew", + "▁Bre w", + "AME TER", + "▁review ed", + "ir at", + "ira t", + "i rat", + "▁n erve", + "▁nerv e", + "▁ner ve", + "▁L inda", + "▁Lin da", + "▁Lind a", + "▁dec is", + "▁sp okes", + "▁spoke s", + "▁spo kes", + "▁qu ed", + "▁que d", + "▁q ued", + "▁F T", + "▁ FT", + "▁в ін", + "▁ві н", + "ou sing", + "ous ing", + "o using", + "▁L arge", + "▁Lar ge", + "▁ Large", + "▁op ponents", + "▁oppon ents", + "▁opponent s", + "▁D isc", + "▁Dis c", + "▁Di sc", + "Found ation", + "EQ UAL", + "og g", + "o gg", + "Re try", + "Ret ry", + "R etry", + "CHAN NEL", + "▁Е вро", + "▁% .", + "▁ %.", + "▁i i", + "▁ ii", + "de ad", + "d ead", + "▁M ale", + "▁Mal e", + "▁Ma le", + "Com pleted", + "Comp leted", + "Complete d", + "ty p", + "t yp", + "▁Ty ler", + "Dis k", + "Di sk", + "D isk", + "Hi de", + "H ide", + "iju ana", + "▁public ations", + "▁publication s", + "fo x", + "f ox", + "vis ed", + "vi sed", + "v ised", + "Fore ign", + "Write Line", + "де ра", + "дер а", + "▁remain der", + "Pi cker", + "P icker", + "we alth", + "▁G or", + "▁Go r", + "sequ ently", + "▁coll ision", + "▁Harr ison", + "▁Harris on", + "▁work place", + "▁N ormal", + "▁Nor mal", + "▁Norm al", + "▁ Normal", + "▁B irth", + "▁Bir th", + "▁cons ume", + "▁consum e", + "Sh ift", + "▁avoid ing", + "▁C ha", + "▁Ch a", + "▁An ti", + "▁Ant i", + "▁ch arts", + "▁char ts", + "▁chart s", + "▁P av", + "▁Pa v", + "ст вом", + "ство м", + "ual mente", + "an ed", + "ane d", + "a ned", + "▁A uch", + "▁Au ch", + "rd ev", + "r dev", + "▁she er", + "▁an gl", + "▁ang l", + "sub str", + "Gener ate", + "> =", + "▁B ev", + "▁Be v", + "▁ч ем", + "▁че м", + "▁camp o", + "▁cam po", + "▁lect ure", + "hy per", + "▁Balt imore", + "mi x", + "m ix", + "ke iten", + "keit en", + "▁ра ди", + "▁l asted", + "▁la sted", + "▁last ed", + "▁las ted", + "▁discrim ination", + "ig te", + "igt e", + "ok al", + "oka l", + "o kal", + "Ph ase", + "▁T itel", + "▁Tit el", + "▁Fif th", + "▁di agnostic", + "su ng", + "sun g", + "s ung", + "▁giorn ata", + "os ta", + "ost a", + "o sta", + "is co", + "isc o", + "▁S ara", + "▁Sa ra", + "▁Sar a", + "m v", + "▁el ő", + "▁R osen", + "▁Ro sen", + "▁Ros en", + "▁Rose n", + "▁E SP", + "▁ES P", + "ph er", + "p her", + "▁a j", + "▁ aj", + "Path s", + "Pat hs", + "▁R alph", + "▁ž e", + "▁ že", + "ре в", + "р ев", + "▁о коло", + "▁ок оло", + "▁Ag reement", + "▁Word Press", + "an try", + "ant ry", + "▁p icks", + "▁pick s", + "▁pi cks", + "▁pic ks", + "▁N ur", + "▁Nu r", + "chedul ed", + "ki e", + "k ie", + "▁represent ations", + "▁representation s", + "++ ){", + "++) {", + "ess ment", + "▁count less", + "Block s", + "Bl ocks", + "Blo cks", + "ym e", + "y me", + "▁c lo", + "▁cl o", + "▁B ened", + "▁Be ned", + "▁Ben ed", + "ch ars", + "char s", + "cha rs", + "▁A gent", + "▁Ag ent", + "▁Age nt", + "▁ Agent", + "▁hist oria", + "▁histor ia", + "▁F loor", + "▁Fl oor", + "▁Flo or", + "▁ten ía", + "▁long est", + "▁lon gest", + "fr ica", + "▁b ef", + "▁be f", + "▁mechan isms", + "▁mechanism s", + "ла зи", + "▁h eter", + "▁he ter", + "▁het er", + "▁athlet es", + "▁period ic", + "▁V otes", + "▁Vo tes", + "ри сти", + "▁n á", + "▁ ná", + "▁m aid", + "▁ma id", + "▁mai d", + "▁s wear", + "▁sw ear", + "▁swe ar", + "▁wip ed", + "▁graph s", + "▁grap hs", + "▁t hesis", + "▁the sis", + "▁th esis", + "▁sens ation", + "pers istence", + "▁V il", + "▁Vi l", + "ac s", + "a cs", + "▁de el", + "sc rib", + "scri b", + "scr ib", + "ie ro", + "ier o", + "i ero", + "▁dis cre", + "▁disc re", + "air y", + "ai ry", + "Data Source", + "q t", + "ic iones", + "ici ones", + "icio nes", + "icion es", + "▁res pected", + "▁respect ed", + "▁f ram", + "▁fr am", + "▁fra m", + "▁spec ialized", + "▁special ized", + "▁prés ent", + "▁pré sent", + "Tur n", + "T urn", + "▁compl aints", + "▁complain ts", + "▁complaint s", + "(\" ,", + "( \",", + "▁Rel ated", + "▁Set ting", + "▁ Setting", + "р ю", + "▁s ą", + "▁P le", + "▁Pl e", + "▁d isse", + "▁dis se", + "▁diss e", + "ca ps", + "cap s", + "c aps", + "▁C ash", + "▁Cas h", + "▁Ca sh", + "▁cons umed", + "▁consum ed", + "▁consume d", + "▁l b", + "▁ lb", + "Ad just", + "Ser ialize", + "Serial ize", + "S erialize", + "is y", + "i sy", + "▁pat ent", + "▁vis ibility", + "▁S ach", + "▁Sa ch", + "▁Sac h", + "ün st", + "▁cy ber", + "▁Bl ake", + "▁Bl oom", + "▁Blo om", + "▁Sh ah", + "▁Sha h", + "PO WER", + "▁in clusion", + "▁incl usion", + "se rie", + "ser ie", + "s erie", + "▁man era", + "sec onds", + "second s", + "is ches", + "isch es", + "ische s", + "isc hes", + "▁C andidate", + "W D", + "op ath", + "o path", + "▁про гра", + "▁efficient ly", + "ap ps", + "app s", + "tool bar", + "we nd", + "wen d", + "w end", + "▁Ne il", + "▁form ats", + "▁format s", + "▁forma ts", + "▁T emplate", + "▁Temp late", + "▁ Template", + "▁min istry", + "▁minist ry", + "▁Char acter", + "▁ Character", + "Un iform", + "▁fon ction", + "не м", + "н ем", + "Wh ile", + "к ва", + "рі я", + "▁D L", + "▁ DL", + "▁L ayout", + "▁La yout", + "▁Lay out", + "▁ Layout", + "не ние", + "▁c aval", + "▁ca val", + "▁cav al", + "▁H ob", + "▁Ho b", + "SP I", + "S PI", + "▁h ely", + "▁he ly", + "▁hel y", + "Dest ination", + "), \r", + ") ,\r", + "▁i OS", + "▁ad mission", + "▁adm ission", + "▁c ss", + "▁cs s", + "▁ css", + "user Id", + "um bling", + "umb ling", + "▁bo oking", + "▁book ing", + "▁COPY RIGHT", + "▁b land", + "▁bl and", + "output s", + "▁sub mission", + "▁subm ission", + "ti t", + "t it", + "fe ctions", + "fect ions", + "fection s", + "fr agment", + "frag ment", + "▁fa ç", + "▁Through out", + "▁distingu ished", + "▁distinguish ed", + "▁ar range", + "▁arr ange", + "▁arrang e", + "ume ric", + "umer ic", + "xf e", + "x fe", + "ip age", + "ipa ge", + "i page", + "ер жа", + "▁C ars", + "▁Car s", + "▁Ca rs", + "▁P AGE", + "▁PA GE", + "▁ PAGE", + "▁a unque", + "▁insert ed", + "smith y", + "AL LOC", + "ALL OC", + "RE C", + "R EC", + "▁B ak", + "▁Ba k", + "▁Str ong", + "ac hen", + "ach en", + "ache n", + "a chen", + "▁Spec ific", + "w q", + "▁Д у", + "MO VE", + "▁mús ica", + "▁C ris", + "▁Cr is", + "ea u", + "e au", + "▁F orum", + "▁For um", + "▁Fo rum", + "li sted", + "list ed", + "l isted", + ")\\ \\", + ") \\\\", + "▁X VI", + "▁XV I", + "▁м оло", + "▁мо ло", + "/ $", + "Be r", + "B er", + "▁tact ics", + "Form atter", + "Format ter", + "op ens", + "ope ns", + "open s", + "▁r h", + "▁ rh", + "▁t ram", + "▁tr am", + "▁tra m", + "V L", + "▁Pro file", + "▁Prof ile", + "▁ Profile", + "▁par ish", + "▁Ray mond", + "▁cont empor", + "▁Pl anning", + "▁Plan ning", + "▁Ч е", + "▁A RM", + "▁AR M", + "▁ ARM", + "▁des ires", + "▁desire s", + "k v", + "O s", + "▁m iner", + "▁min er", + "▁mi ner", + "▁mine r", + "▁qual ify", + "ik u", + "i ku", + "▁der ni", + "ol óg", + "▁K id", + "▁Ki d", + "ane an", + "▁Hol land", + "▁Holl and", + "Aut om", + "Auto m", + "▁Hamilton ian", + "St ation", + "Stat ion", + "js p", + "j sp", + "▁YO UR", + "▁YOU R", + "▁Th ailand", + "effect ive", + "п ло", + "▁relie ved", + "▁O klahoma", + "▁Jul ian", + "▁Juli an", + "▁Julia n", + "▁ind ent", + "▁inde nt", + "▁ indent", + "if r", + "i fr", + "пре де", + "▁fl ame", + "on io", + "oni o", + "o nio", + "As sign", + "Ass ign", + "▁sh ifts", + "▁shift s", + "▁car acter", + "▁caract er", + "if icates", + "ific ates", + "ificate s", + "ifica tes", + "X R", + "▁G FP", + "▁GF P", + "FE ATURE", + "▁M aine", + "▁Ma ine", + "▁Main e", + "▁Mai ne", + "▁f rank", + "▁fr ank", + "▁al igned", + "▁align ed", + "▁ aligned", + "▁p ří", + "▁př í", + "Code Attribute", + "▁M AC", + "▁MA C", + "▁ MAC", + "▁R oot", + "▁Ro ot", + "▁ Root", + "▁F M", + "▁ FM", + "erv ation", + "с лі", + "▁s hy", + "▁sh y", + "▁partic ul", + "▁parti cul", + "pl atz", + "▁hypothes is", + "at hol", + "ath ol", + "s With", + "J s", + "$ ^{-", + "▁#! /", + "▁l emon", + "▁le mon", + "▁a bol", + "▁ab ol", + "▁ abol", + "▁Mil an", + "▁Mi lan", + "an ten", + "ant en", + "ante n", + "a nten", + "▁s ia", + "▁si a", + "ri as", + "ria s", + "r ias", + "▁con sid", + "▁cons id", + "as so", + "ass o", + "ain ers", + "ai ners", + "ainer s", + "aine rs", + "▁cir ca", + "▁circ a", + "re try", + "ret ry", + "r etry", + "▁nue vo", + "const ants", + "constant s", + "▁Med iterr", + "▁Turk ish", + "ion en", + "io nen", + "ione n", + "i onen", + "c rypto", + "▁ev olved", + "▁\" ?", + "▁p úblic", + "▁comp rend", + "▁compre nd", + "▁compr end", + "al lo", + "all o", + "zo om", + "z oom", + "▁dat etime", + "▁date time", + "▁ datetime", + "▁mond iale", + "ма т", + "м ат", + "▁M ask", + "▁Ma sk", + "▁Mas k", + "▁ Mask", + "▁p row", + "▁pro w", + "▁pr ow", + "▁belong ing", + "+ '", + "OUT PUT", + "▁G rab", + "▁Gr ab", + "▁Gra b", + "M ir", + "▁accommod ate", + "▁$ ('#", + "▁ $('#", + "▁Lou ise", + "▁Louis e", + "▁da mit", + "▁dam it", + "}' ,", + "} ',", + "scri pts", + "script s", + "sn apshot", + "snap shot", + "▁sh itty", + "▁shit ty", + "▁y o", + "▁ yo", + "▁belie ving", + "▁inhabit ants", + "W P", + "▁Colomb ia", + "li sts", + "list s", + "l ists", + "▁Mur phy", + "Data set", + "Dat aset", + "▁(! $", + "▁tremend ous", + "▁se ñ", + "▁S ed", + "▁Se d", + "▁sw allowed", + "▁swallow ed", + "om p", + "o mp", + "▁L ate", + "▁La te", + "▁Lat e", + "▁an ys", + "▁any s", + "▁dead ly", + "fol low", + "f ollow", + "▁A nc", + "▁An c", + "▁h w", + "▁ hw", + "wik ipedia", + "ic ts", + "ict s", + "▁Al aska", + "▁sc ary", + "▁scar y", + "▁second o", + "▁sec ondo", + "▁her oes", + "▁hero es", + "▁veter an", + "▁behav iors", + "▁behavior s", + "▁behavi ors", + "- %", + "▁E z", + "▁с і", + "▁ сі", + "tik z", + "▁spect acular", + "▁Ch ron", + "▁( @", + "▁ (@", + "▁de mo", + "▁dem o", + "▁ demo", + "▁ser ialized", + "▁serial ized", + "▁In depend", + "▁Indep end", + "BU ILD", + "fail ure", + "▁P ORT", + "▁PO RT", + "▁ PORT", + "ю чи", + "▁med itation", + "sample s", + "sam ples", + "s amples", + "i ão", + "▁Ни кола", + "▁я зы", + "▁Tr uth", + "▁Tru th", + "▁co efficient", + "▁coeff icient", + "sl ug", + "▁XV III", + "▁XVI II", + "▁XVII I", + "ia o", + "i ao", + "de ck", + "dec k", + "▁раз ви", + "▁ad oles", + "ar ius", + "ari us", + "▁H az", + "▁Ha z", + "▁Pro test", + "▁Prote st", + "ra de", + "rad e", + "r ade", + "не ния", + "▁cl ause", + "conne ctor", + "connect or", + "conn ector", + "RA TE", + "R ATE", + "ц ю", + "▁Conne cticut", + "V S", + "abul ary", + "HO W", + "▁d elen", + "▁de len", + "▁del en", + "▁su ited", + "▁suit ed", + "▁suite d", + "▁Sur vey", + "ze c", + "z ec", + "ți i", + "ț ii", + "▁b acks", + "▁back s", + "▁ba cks", + "▁ backs", + "com merce", + "▁And rea", + "▁Andre a", + "▁Andr ea", + "▁propag anda", + "iz ioni", + "izi oni", + "izio ni", + "▁B il", + "▁Bi l", + "▁In nov", + "▁Inn ov", + "▁forg ive", + "▁oper ates", + "▁operate s", + "▁opera tes", + "ч ний", + "▁l ingu", + "▁lin gu", + "▁ling u", + "▁c ollar", + "▁col lar", + "▁coll ar", + "до л", + "сі й", + "zt en", + "zte n", + "z ten", + "im at", + "ima t", + "i mat", + "▁sh oe", + "ge nder", + "gen der", + "g ender", + "▁leg ally", + "▁legal ly", + "RO P", + "R OP", + "▁S leep", + "deleg ate", + "ID s", + "▁build s", + "▁qu er", + "▁que r", + "▁q uer", + "▁ quer", + "uls ion", + ". “", + "к ло", + "ri se", + "ris e", + "r ise", + "th ink", + "К о", + "▁bacter ia", + "▁magn ific", + "▁prison er", + "Cl ock", + "C lock", + "R B", + "ú t", + "▁L iz", + "▁Li z", + "gr a", + "g ra", + "▁And ré", + "▁Andr é", + "▁D ennis", + "▁Den nis", + "▁sur ge", + "▁surg e", + "ex isting", + "exist ing", + "▁W ald", + "▁Wal d", + "▁Wa ld", + "▁S chema", + "▁Sch ema", + "▁Sche ma", + "▁ Schema", + "▁war nings", + "▁warn ings", + "▁warning s", + "▁qu adr", + "▁quad r", + "at te", + "att e", + "▁E ins", + "▁Ein s", + "▁ad option", + "▁adopt ion", + "▁w anna", + "▁de rive", + "▁der ive", + "▁deriv e", + "▁ derive", + "▁a rena", + "▁are na", + "▁ar ena", + "▁aren a", + "▁Den ver", + "▁F i", + "▁ Fi", + "▁Jess ica", + "acy j", + "R atio", + "▁которы е", + "▁Act ivity", + "▁Activ ity", + "▁ Activity", + "em u", + "e mu", + "▁St alin", + "▁Sta lin", + "ag gi", + "agg i", + "a ggi", + "▁f ün", + "▁f ils", + "▁fil s", + "▁fi ls", + "aj u", + "a ju", + "card s", + "car ds", + "c ards", + "▁att raction", + "▁attract ion", + "▁attr action", + "▁attra ction", + "od ot", + "odo t", + "o dot", + "F at", + "▁H aven", + "▁Ha ven", + "▁Have n", + "▁Hav en", + "▁nine teenth", + "▁ninete enth", + "▁* *\"", + "▁** \"", + "▁m aggio", + "▁mag gio", + "ma ny", + "man y", + "m any", + "win ning", + "▁G A", + "▁ GA", + "▁d ummy", + "▁ dummy", + "Un able", + "en ci", + "enc i", + "ère nt", + "è rent", + "Im g", + "I mg", + "▁t ob", + "▁to b", + "DI P", + "D IP", + "S ince", + "▁Sa fe", + "▁Saf e", + "▁ Safe", + "Gu ard", + "is ure", + "i sure", + "port e", + "por te", + "p orte", + "▁stad ium", + "in di", + "ind i", + "▁App arently", + "ug no", + "▁w olf", + "▁ne ces", + "▁overse as", + "of s", + "o fs", + "ar el", + "are l", + "a rel", + "▁F ine", + "▁Fin e", + "▁Fi ne", + "▁cor rupt", + "▁n ovember", + "▁nov ember", + "▁nove mber", + "▁interpret ed", + "ib ile", + "ibil e", + "▁w ages", + "▁wa ges", + "▁wage s", + "▁Pre tty", + "▁Her bert", + "▁reg istr", + "вы м", + "an swer", + "ans wer", + "▁m orte", + "▁mor te", + "▁mort e", + "▁com posite", + "▁compos ite", + "Tool bar", + "▁iter ator", + "▁ iterator", + "ant ine", + "anti ne", + "▁init ialized", + "▁initial ized", + "▁initialize d", + "▁ initialized", + "▁poor ly", + "Access or", + "▁Han nah", + "▁Hann ah", + "▁то лько", + "ol an", + "ola n", + "o lan", + "▁o tto", + "▁ot to", + "▁ott o", + "▁ otto", + "▁str ikes", + "▁stri kes", + "▁strike s", + "▁conflict s", + "▁conflic ts", + "▁s urg", + "▁su rg", + "▁sur g", + "▁histor ian", + "▁historia n", + "wo man", + "w oman", + "▁l ibraries", + "be w", + "b ew", + ")- -(", + ")-- (", + "ga ther", + "g ather", + "▁L ip", + "▁Li p", + "▁f ict", + "▁fi ct", + "FIL TER", + "@ {", + "▁bl essed", + "▁bless ed", + "et ics", + "etic s", + "eti cs", + "▁f ork", + "▁for k", + "▁Me tal", + "▁Met al", + "▁Meta l", + "po lation", + "pol ation", + "p olation", + "▁negoti ations", + "▁gen us", + "▁genu s", + "▁cont rolling", + "▁control ling", + "VER T", + "VE RT", + "V ERT", + "▁P erry", + "▁Per ry", + "▁S PD", + "▁SP D", + "CA SE", + "C ASE", + "т вер", + "▁C rown", + "▁Cr own", + "▁Cro wn", + "▁Crow n", + "▁ind ul", + "▁indu l", + "▁e hemal", + "▁ampl itude", + "▁amplit ude", + "▁B ach", + "▁Ba ch", + "▁phot ographer", + "▁photograph er", + "n ý", + "▁inv ested", + "▁invest ed", + "▁P arte", + "▁Par te", + "▁Part e", + "▁pro long", + "C U", + "icht et", + "ichte t", + "res ume", + "▁c arb", + "▁car b", + "▁ca rb", + "ur st", + "urs t", + "▁N ixon", + "▁n eur", + "▁ne ur", + "▁neu r", + "▁ neur", + "▁corpor ations", + "▁corporation s", + "Op s", + "O ps", + "u u", + "l m", + "ap ple", + "app le", + "ch te", + "cht e", + "▁deliber ately", + "ber e", + "be re", + "b ere", + "▁fe br", + "▁provinc ia", + "▁provin cia", + "Over flow", + "▁E ight", + "▁ind ication", + "▁indic ation", + "▁pist ol", + "▁к ре", + "▁ кре", + "oc ial", + "oci al", + "o cial", + "▁r und", + "▁run d", + "▁ru nd", + "▁ rund", + "▁se hr", + "ok at", + "oka t", + "o kat", + "ül et", + "ü let", + "▁He at", + "Н а", + "▁о дин", + "▁од ин", + "IC S", + "I CS", + "ay e", + "a ye", + "▁eight een", + "▁t ug", + "▁tu g", + "LO T", + "L OT", + "▁L ar", + "▁La r", + "ning s", + "n ings", + "▁T odd", + "▁To dd", + "▁Tod d", + "▁organis ations", + "▁organisation s", + "▁g enes", + "▁gen es", + "▁ge nes", + "▁gene s", + "B ag", + "Ke ep", + "^{ +", + "Base d", + "Bas ed", + "B ased", + "sk in", + "ski n", + "s kin", + "▁to das", + "▁tod as", + "▁toda s", + "▁illustr ated", + "▁c f", + "▁ cf", + "▁ar riving", + "▁arriv ing", + "▁arr iving", + "▁excess ive", + "▁tra its", + "▁trait s", + "▁s ank", + "▁san k", + "▁Att ribute", + "▁ Attribute", + "▁G D", + "▁ GD", + "com par", + "comp ar", + "▁dent ro", + "br is", + "b ris", + "▁at oms", + "▁atom s", + "fr ed", + "fre d", + "f red", + "▁E val", + "▁Ev al", + "▁Eva l", + "▁ Eval", + "▁di stances", + "▁dist ances", + "▁distance s", + "st aw", + "sta w", + "краї н", + "vari ables", + "variable s", + "l c", + "на ли", + "нал и", + "н али", + "▁чемпи она", + "wi j", + "w ij", + "▁Sim ilar", + "je k", + "j ek", + "Pe t", + "P et", + "=\" $", + "ко то", + "▁R ang", + "▁Ra ng", + "▁Ran g", + "ion ato", + "iona to", + "▁bek annt", + "▁bekan nt", + "! *", + "Li m", + "L im", + "▁concl usions", + "▁conclusion s", + "ain te", + "ai nte", + "aint e", + "a inte", + "- ,", + "▁g ł", + "▁pass ive", + "▁Ga ussian", + "▁stag ione", + "ME DI", + "MED I", + "it ol", + "ito l", + "i tol", + "▁Jer emy", + "View s", + "class List", + "▁desper ately", + "▁desperate ly", + "▁ver l", + "▁ve rl", + "br ace", + "bra ce", + "N P", + "▁c ob", + "▁co b", + "▁A rist", + "▁Ar ist", + "▁Ari st", + "da p", + "d ap", + "Fil ters", + "Filter s", + "'=> '", + "ul tan", + "ult an", + "▁F actory", + "▁ Factory", + "è le", + "▁l asting", + "▁last ing", + "▁las ting", + "▁element ary", + "▁C M", + "▁ CM", + "▁Louis iana", + "▁p ov", + "▁po v", + "PC I", + "P CI", + "è de", + "▁P ink", + "▁Pin k", + "▁Br uno", + "▁Bru no", + "▁Brun o", + "▁Y ellow", + "▁ev angel", + "▁lik elihood", + "WID TH", + "▁$ -", + "▁ $-", + "ni co", + "nic o", + "n ico", + "hu i", + "h ui", + "ak ter", + "akt er", + "akte r", + "ne urs", + "neur s", + "n eurs", + "▁bre eze", + "▁bree ze", + "▁со ста", + "▁He ader", + "▁Head er", + "▁ Header", + "om rå", + "▁D ylan", + "▁Dy lan", + "▁Bi ographie", + "▁Univers ität", + "on so", + "ons o", + "HAND LE", + "J ournal", + "ea st", + "e ast", + "▁sup pliers", + "▁supplier s", + "▁table t", + "▁tab let", + "LI C", + "L IC", + "PER TY", + "ї в", + "▁z aw", + "▁za w", + "▁su bm", + "▁sub m", + "▁Fern ando", + "▁nou velle", + "▁nouve lle", + "▁Point s", + "▁ Points", + "▁str angers", + "▁strange rs", + "▁stranger s", + "▁strang ers", + "Component Model", + "ist ro", + "istr o", + "au rus", + "aur us", + "▁san ct", + "▁о дна", + "▁од на", + "▁В ы", + "▁о на", + "▁он а", + "▁ она", + "vert ical", + "Sp ring", + "▁Har old", + "▁Back ground", + "▁ Background", + "Bal ance", + "Key word", + "~$ \\", + "~ $\\", + "mal loc", + "m alloc", + "ORM AL", + "Sk ip", + "▁Mu ham", + "▁back wards", + "▁backward s", + "c ów", + "по зи", + "▁back end", + "▁ backend", + "▁de emed", + "▁accur ately", + "▁accurate ly", + "▁trans c", + "▁Broad way", + "▁g rud", + "▁gr ud", + "▁gru d", + "▁N amen", + "▁Name n", + "▁Na men", + "▁Nam en", + "▁sh ifting", + "▁shift ing", + "▁ment ally", + "▁mental ly", + "▁cal ories", + "▁cons ensus", + "Perm issions", + "Permission s", + "▁ob jet", + "▁obj et", + "▁elabor ate", + "at ts", + "att s", + "▁sn ake", + "▁ref res", + "▁refr es", + "ar u", + "a ru", + "▁reflect s", + "oun ge", + "o unge", + "R ank", + "▁K urt", + "▁Kur t", + "▁Ku rt", + "▁p ied", + "▁pie d", + "▁pi ed", + "▁exped ition", + "V el", + "▁O wen", + "Le ad", + "L ead", + "▁utter ly", + "▁Ar be", + "▁bre asts", + "▁breast s", + "IP S", + "I PS", + "▁hung er", + "▁hun ger", + "at em", + "ate m", + "a tem", + "▁vers chied", + "▁versch ied", + "▁Cam era", + "▁ Camera", + "▁Mün chen", + "iv als", + "ival s", + "iva ls", + "i vals", + "▁sp raw", + "▁spr aw", + "▁S ü", + "▁Was ser", + "▁mechan ics", + "Load ed", + "Lo aded", + "db c", + "d bc", + "▁re marks", + "▁rem arks", + "▁remark s", + "▁ remarks", + "▁} ).", + "▁}) .", + "▁ }).", + "▁pain ter", + "▁pa inter", + "▁paint er", + "▁h aut", + "▁ha ut", + "Mar shal", + "IS D", + "I SD", + "▁ve loc", + "▁vel oc", + "▁In cre", + "▁Inc re", + "W ar", + "▁ру с", + "▁com pte", + "▁comp te", + "▁compt e", + "ü g", + "▁Def inition", + "▁ Definition", + "▁G am", + "▁Ga m", + "▁H ir", + "▁Hi r", + "▁witness ed", + "▁g ren", + "▁gr en", + "▁gre n", + "▁ gren", + "▁hur ry", + "ch et", + "che t", + "c het", + "re verse", + "G F", + "▁Qu arter", + "п ла", + "▁s ar", + "▁sa r", + "sb urg", + "sbur g", + "s burg", + "▁D it", + "▁Di t", + "▁ Dit", + "▁Arn old", + "j k", + "▁l ambda", + "▁ lambda", + "è ge", + "▁o z", + "▁ oz", + "▁h ans", + "▁ha ns", + "▁han s", + "▁answ ering", + "▁answer ing", + "▁o live", + "▁ol ive", + "▁sp ont", + "▁spo nt", + "▁inter vals", + "▁interval s", + "> @", + "▁т ран", + "▁тра н", + "▁F ocus", + "▁ Focus", + "ч них", + "▁д ви", + "▁tri angle", + "▁r ally", + "▁P unk", + "▁Pun k", + "▁G and", + "▁Ga nd", + "se ctions", + "section s", + "sect ions", + "сси й", + "AC CESS", + "A CCESS", + "ha rm", + "har m", + "h arm", + "▁Sk ip", + "▁ Skip", + "▁D river", + "▁Dr iver", + "▁Drive r", + "▁ Driver", + "▁Sant iago", + "it ung", + "itu ng", + "▁B arr", + "▁Bar r", + "▁Ba rr", + "process or", + "▁real ised", + "▁realise d", + "ą z", + "le ave", + "▁C omo", + "▁Com o", + "▁Co mo", + "▁Re views", + "▁Review s", + "▁и зда", + "▁из да", + "▁earn ings", + "▁ear nings", + "▁earning s", + "▁S creen", + "▁Sc reen", + "▁Scre en", + "▁ Screen", + "gr and", + "gra nd", + "g rand", + "▁ap ril", + "▁apr il", + "▁sil ently", + "▁silent ly", + "ed o", + "e do", + "ue st", + "ues t", + "u est", + "oo oo", + "▁Исто рия", + "ра з", + "MAGE S", + "MAG ES", + "▁Sing h", + "▁Sin gh", + "▁Per fect", + "▁revolution ary", + "▁н і", + "▁ ні", + "▁Sch ools", + "▁School s", + "R ich", + "▁ch rom", + "▁chr om", + "▁an terior", + "▁ante rior", + "▁Indones ia", + "Con straints", + "Constraint s", + "▁\" __", + "▁\"_ _", + "▁six teen", + "▁sixt een", + "ér e", + "é re", + "мен та", + "мент а", + "N il", + "je l", + "j el", + "че ские", + "чески е", + "▁thr one", + "▁thro ne", + "▁aud iences", + "▁audience s", + "▁i hren", + "▁ih ren", + "▁ihr en", + "▁ihre n", + "ра б", + "Qu ick", + "in burgh", + "fi co", + "fic o", + "f ico", + "▁kid n", + "▁ki dn", + "irm ingham", + "is le", + "isl e", + "iz ación", + "iza ción", + "▁Ch ampions", + "▁Champion s", + "▁вы со", + "ol er", + "ole r", + "o ler", + "▁z ak", + "▁za k", + "▁p lat", + "▁pl at", + "▁V III", + "▁VI II", + "▁VII I", + "at ique", + "ati que", + "li ter", + "lit er", + "l iter", + "▁P rest", + "▁Pr est", + "▁Pres t", + "▁Pre st", + "in is", + "ini s", + "i nis", + "▁scient ist", + "▁m ån", + "▁må n", + "ke ley", + "kel ey", + "▁h yd", + "▁hy d", + "grad uate", + "of t", + "o ft", + "▁N GC", + "on gs", + "ong s", + "▁t ier", + "▁tie r", + "▁ti er", + "▁Sh aw", + "▁Sha w", + "un ächst", + "▁establish ing", + "▁ind icator", + "▁indic ator", + "▁Par ad", + "▁Pa rad", + "▁Para d", + "▁Tr ail", + "▁Tra il", + "UM N", + "▁sp ine", + "▁spin e", + "▁Vis ual", + "▁ Visual", + ":: $", + "▁t eles", + "▁te les", + "▁tele s", + "▁tel es", + "OP ER", + "O PER", + "▁pack aging", + "to ire", + "t oire", + "▁не ско", + "▁product ivity", + "A f", + "ні ї", + "▁de gener", + "▁deg ener", + "br it", + "b rit", + "U i", + "▁Y am", + "▁Ya m", + "▁d ough", + "▁do ugh", + "▁dou gh", + "os ph", + "osp h", + "▁cl ue", + "▁ре ги", + "▁me ille", + "▁tend ency", + "▁re lay", + "▁rel ay", + "▁design ers", + "▁designer s", + "▁Т у", + "Sh are", + "▁b icy", + "▁bi cy", + "▁M asters", + "▁Ma sters", + "▁Mas ters", + "▁Master s", + "▁м но", + "▁altern atives", + "▁alternative s", + "ет о", + "е то", + "▁coun tr", + "▁count r", + "▁W ow", + "▁Wo w", + "LO CAL", + "LOC AL", + "en ue", + "enu e", + "▁s lim", + "▁sl im", + "к ви", + "▁t ir", + "▁ti r", + "▁do it", + "lic a", + "li ca", + "l ica", + "ci pe", + "cip e", + "c ipe", + "iz ia", + "izi a", + "▁A ires", + "▁Air es", + "▁F alls", + "▁Fall s", + "▁Fal ls", + "▁concent rate", + "▁concentr ate", + "▁ne gl", + "▁neg l", + "▁Re in", + "? ,", + "▁G ott", + "▁Go tt", + "▁Got t", + "▁Ver ify", + "▁ Verify", + "▁Stud ios", + "▁Studio s", + "$ ('#", + "ow ym", + "owy m", + "я в", + "Prim itive", + "▁tax i", + "▁ta xi", + "▁Com mercial", + "▁Ч ер", + "▁Че р", + "place holder", + "se au", + "sea u", + "s eau", + "cor rect", + "he imer", + "heim er", + "▁H of", + "▁Ho f", + "▁d ia", + "▁di a", + "▁i rr", + "▁ir r", + "▁ur ged", + "▁urg ed", + "▁urge d", + "▁a nom", + "▁an om", + "▁ano m", + "▁t arde", + "▁tar de", + "▁tard e", + "ur m", + "u rm", + "▁se ized", + "▁sei zed", + "▁seiz ed", + "DO T", + "D OT", + "op acity", + "St rings", + "String s", + "Str ings", + "▁dec iding", + "▁decid ing", + "▁listen ers", + "▁listener s", + "ár a", + "á ra", + "▁pl anted", + "▁plan ted", + "▁plant ed", + "▁é taient", + "▁ét aient", + "Z oom", + "st ví", + "ng th", + "ä ude", + "▁C av", + "▁Ca v", + "▁v endor", + "▁vend or", + "▁ vendor", + "▁ ż", + "▁meas uring", + "▁necess ity", + "▁r ivers", + "▁ri vers", + "▁river s", + "▁riv ers", + "▁labor atory", + "▁E ff", + "▁reprodu ce", + "▁S ak", + "▁Sa k", + "▁not ebook", + "▁note book", + "▁reason ably", + "isecond s", + "i seconds", + "▁Part ial", + "▁ Partial", + "GUI D", + "GU ID", + "G UID", + "▁Per iod", + "▁ Period", + "▁reve aling", + "▁reveal ing", + "▁conv iction", + "▁ н", + "▁бу ли", + "▁altern ate", + "▁alter nate", + "cc iones", + "▁N AT", + "▁NA T", + "▁can onical", + "▁canon ical", + "mo z", + "m oz", + "▁Mé xico", + "M o", + "▁ш а", + "▁ ша", + "lim inary", + "f é", + "чно й", + "ч ной", + "▁Ham burg", + "▁Hamb urg", + "▁influ ential", + "▁b olt", + "▁bo lt", + "▁bol t", + "az zo", + "azz o", + "PH P", + "P HP", + "▁Sa udi", + "▁Saud i", + "▁Sau di", + "▁r m", + "▁ rm", + "▁cer ca", + "▁decor ated", + "▁st aat", + "▁sta at", + "Lo u", + "L ou", + "▁compet itors", + "во ї", + "▁diam ond", + "▁dia mond", + "▁m obil", + "▁mo bil", + "▁mob il", + "Click Listener", + "set State", + "▁s üd", + "; \"", + "œ ur", + "▁Lud wig", + "▁clin ic", + "▁e go", + "▁eg o", + "▁ ego", + "Thread ing", + "▁f ract", + "▁fr act", + "▁fra ct", + "Ref lection", + "oss ip", + "\"] [\"", + "▁L ov", + "▁Lo v", + "Ex press", + "Exp ress", + "Expr ess", + "д ри", + "if acts", + "ifact s", + "▁O ften", + "▁Of ten", + "▁ лу", + "▁p ets", + "▁pe ts", + "▁pet s", + "▁address ing", + "▁m ens", + "▁me ns", + "▁men s", + "▁ED IT", + "▁ EDIT", + "ud der", + "udd er", + "Vert ical", + "ка та", + "Cap t", + "C apt", + "verb ose", + "▁вой ны", + "UNK NOWN", + "un its", + "unit s", + "uni ts", + "per mission", + "perm ission", + "[ _", + "▁er sch", + "▁ers ch", + "▁comm unes", + "▁commun es", + "▁commune s", + "Un ityEngine", + "▁com mut", + "▁comm ut", + "kl ass", + "k lass", + "▁volt age", + "▁volta ge", + "re zent", + "rez ent", + "pe rf", + "per f", + "DR V", + "D RV", + "▁f ame", + "▁fam e", + "▁fa me", + "▁S pot", + "▁Sp ot", + "▁Л ю", + "▁c asting", + "▁cas ting", + "▁cast ing", + "hi m", + "h im", + "▁en gl", + "▁eng l", + "▁int ro", + "▁intr o", + "▁Г у", + "Comp any", + "some thing", + "som ething", + "▁cl icking", + "▁click ing", + "жи ва", + "▁fl ames", + "▁flame s", + "▁random ly", + "ex tr", + "ext r", + "Equal To", + "an ners", + "ann ers", + "anner s", + "anne rs", + "▁p arks", + "▁par ks", + "▁park s", + "▁murm ured", + "ми я", + "▁reason ing", + "сле д", + "▁n er", + "▁ne r", + "▁ ner", + "▁é c", + "▁ éc", + "ow ners", + "own ers", + "owner s", + "▁Д же", + "▁Дж е", + "▁me er", + "▁typ ing", + "▁ty ping", + "▁happ ily", + ".. ...", + "... ..", + ".... .", + ". ....", + "▁Ч а", + "be cca", + "bec ca", + "▁P apers", + "▁Pa pers", + "▁Pap ers", + "▁Paper s", + "▁Or acle", + "▁equ ilibrium", + "man agement", + "Li te", + "L ite", + "▁des ktop", + "▁desk top", + "ă r", + "▁G ill", + "▁Gi ll", + "▁Gil l", + "do rf", + "d orf", + "ig g", + "i gg", + "▁qu esta", + "▁que sta", + "▁quest a", + "▁ questa", + "Warning s", + "Warn ings", + "War nings", + "over flow", + "▁V T", + "▁ VT", + "▁cons isted", + "▁consist ed", + "▁A bu", + "▁Ab u", + "v scale", + "J O", + "ah o", + "a ho", + "▁T ensor", + "▁Ten sor", + "▁ Tensor", + "▁hes itated", + "▁w enn", + "▁we nn", + "▁wen n", + "map sto", + "maps to", + "▁controvers ial", + "M F", + "▁l ac", + "▁la c", + "▁an ch", + "▁anc h", + "▁ anch", + "▁A A", + "▁ AA", + "it ta", + "itt a", + "i tta", + "ul in", + "uli n", + "u lin", + "▁c ler", + "▁cl er", + "▁cle r", + "▁D iana", + "▁Di ana", + "▁Fre ud", + "▁challeng ed", + "▁challenge d", + "лё н", + "л ён", + "▁se ated", + "▁sea ted", + "▁seat ed", + "▁sm iles", + "▁smile s", + "▁cr acked", + "▁crack ed", + "▁а ктив", + "ско ј", + "dict ion", + "di ction", + "d iction", + "ex press", + "exp ress", + "expr ess", + "▁im posed", + "▁imp osed", + "▁pro tests", + "▁prote sts", + "▁protest s", + "▁prot ests", + "▁w ounds", + "▁wound s", + "C ulture", + "N Y", + "prevent Default", + "ad io", + "adi o", + "▁NE W", + "▁ NEW", + "B attle", + "▁se colo", + "▁sec olo", + "▁A x", + "▁found ing", + "(\" -", + "▁ret ro", + "▁retr o", + "▁pot atoes", + "import ant", + "ie me", + "iem e", + "i eme", + "ys ide", + "y side", + "d ummy", + "▁t ilt", + "▁til t", + "▁ti lt", + "▁R ules", + "▁Ru les", + "▁Rule s", + "▁ Rules", + "▁un ters", + "▁unt ers", + "▁unter s", + "A ud", + "V ENDOR", + "ud ge", + "un al", + "una l", + "u nal", + "▁Ad ult", + "▁im pat", + "▁imp at", + "▁rep airs", + "▁repair s", + "▁F erd", + "▁Fe rd", + "▁Fer d", + "▁Az ure", + "▁ Azure", + ")) :", + ") ):", + "▁pag ina", + "▁E pisode", + "File name", + "Fil ename", + "▁j á", + "▁oblig ation", + "ig hed", + "igh ed", + "▁pers istent", + "Mus ic", + "▁C ele", + "▁Ce le", + "▁Cel e", + "▁r y", + "▁ ry", + "▁cert ification", + "ul d", + "u ld", + "▁T L", + "▁ TL", + "▁sk irt", + "▁ski rt", + "▁M ini", + "▁Min i", + "▁Mi ni", + "▁B ring", + "▁Br ing", + ">< ?", + "> %", + "▁P and", + "▁Pan d", + "▁Pa nd", + "▁S UB", + "▁SU B", + "▁ SUB", + "▁compan ions", + "▁companion s", + "▁RE AD", + "▁ READ", + "▁S olutions", + "▁Solution s", + "▁acc essed", + "▁access ed", + "▁p osto", + "▁pos to", + "▁po sto", + "▁post o", + "▁pur suit", + "▁purs uit", + "ow i", + "o wi", + "▁gro cery", + "Sp e", + "S pe", + "haus en", + "▁normal ized", + "▁tra uma", + "gg i", + "g gi", + "ie nia", + "ien ia", + "▁aut umn", + "▁so vere", + "▁sov ere", + "▁Men schen", + "▁Mens chen", + "▁D AG", + "▁DA G", + "▁S ort", + "▁So rt", + "▁Sor t", + "▁ Sort", + "| ---", + "▁l iver", + "▁li ver", + "▁live r", + "▁liv er", + "▁ liver", + "env iron", + "DE CL", + "DEC L", + "▁ма й", + "▁N ak", + "▁Na k", + "bet ween", + "▁gentle man", + "in ging", + "ing ing", + "▁su bur", + "▁sub ur", + "ST O", + "S TO", + "ace ut", + "\\ !", + "▁Fuß ball", + "na r", + "n ar", + "▁b og", + "▁bo g", + "Token s", + "Tok ens", + "▁cer emon", + "▁cere mon", + "DA Y", + "D AY", + "▁out fit", + "▁agric ulture", + "ди и", + "▁N in", + "▁Ni n", + "▁Sp rings", + "▁Spring s", + "▁Spr ings", + "▁Co ach", + "▁d jango", + "▁ django", + "▁C rim", + "▁Cr im", + "▁te cn", + "Th ree", + "em os", + "e mos", + "▁be an", + "▁ bean", + "pi eler", + "pie ler", + "p ieler", + "ri tz", + "rit z", + "r itz", + "ta bs", + "tab s", + "t abs", + "▁Pro blem", + "in and", + "ina nd", + "oc on", + "oco n", + "o con", + "њ и", + "▁bu yer", + "▁buy er", + "us ement", + "use ment", + "▁b or", + "▁bo r", + "▁ bor", + "▁sett embre", + "pp e", + "p pe", + "▁D eg", + "▁De g", + "▁W a", + "▁w ives", + "▁fr anzös", + "▁mar ca", + "▁des cent", + "▁desc ent", + "▁S ha", + "▁Sh a", + "ver ts", + "vert s", + "v erts", + "▁Sh adow", + "▁ Shadow", + "▁Hug o", + "▁Hu go", + "▁A ppe", + "▁App e", + "▁Ap pe", + "▁ Appe", + "▁L ac", + "▁La c", + "al len", + "all en", + "alle n", + "os ity", + "osi ty", + "▁consult ation", + "▁T i", + "▁er ano", + "▁era no", + "▁eran o", + "▁lo vers", + "▁love rs", + "▁lov ers", + "▁lover s", + "▁уни версите", + "▁virt ue", + "▁view ers", + "M u", + "c ategories", + "▁о пера", + "▁over look", + "▁overl ook", + "▁тер рито", + "▁Oper ations", + "▁Operation s", + "▁ Operations", + "è ve", + "- (", + "▁ Ż", + "je v", + "j ev", + "▁c rist", + "▁cr ist", + "▁cris t", + "▁cri st", + "▁мар та", + "▁pro vin", + "▁prov in", + "product ion", + "produ ction", + "prod uction", + "p roduction", + "▁T all", + "▁Tal l", + "▁Ta ll", + "Requ ests", + "Request s", + "▁t iles", + "▁til es", + "▁tile s", + "▁ti les", + "ref lect", + "▁ar gc", + "▁arg c", + "▁ argc", + "▁t emplates", + "▁templ ates", + "▁template s", + "▁ templates", + "AR B", + "A RB", + "▁weiter e", + "▁weit ere", + ")? ;", + ") ?;", + "▁t oll", + "▁to ll", + "▁correspond ence", + "$ ;", + "L T", + "▁t am", + "▁ta m", + "de cess", + "dec ess", + "built in", + "da sh", + "das h", + "d ash", + "ze nie", + "zen ie", + "▁mole cular", + "▁chem icals", + "▁chemical s", + "▁rend ering", + "▁render ing", + "▁Sing les", + "▁Sin gles", + "▁Single s", + "Init ialized", + "Initial ized", + "Initialize d", + "▁Mar tha", + "▁Mart ha", + "ri ere", + "rie re", + "rier e", + "r iere", + "par agraph", + "para graph", + "as ters", + "ast ers", + "aster s", + "aste rs", + "a sters", + "▁dec ides", + "▁decide s", + "▁decid es", + "▁Flor ence", + "▁Flo rence", + "▁Floren ce", + "▁And ers", + "▁An ders", + "мо й", + "▁a pt", + "▁ap t", + "▁ apt", + "▁affili ate", + "ch el", + "che l", + "c hel", + "▁re vision", + "▁rev ision", + "Pat ch", + "P atch", + "▁fi scal", + "▁fis cal", + "wi ę", + "w ię", + "N ational", + "▁depend encies", + "TRAN S", + "TRA NS", + "▁r ack", + "▁rac k", + "▁ra ck", + "sel ling", + "s elling", + "na issance", + "c atalog", + "Sh ip", + "S hip", + "IM AGE", + "I MAGE", + "'] [", + "' ][", + "▁p rv", + "▁pr v", + "▁F en", + "▁Fe n", + "▁rad ar", + "▁ra dar", + "cond itions", + "condition s", + "▁Quest ions", + "▁Question s", + "▁v ivid", + "▁vi vid", + "▁viv id", + "op f", + "o pf", + "FA CE", + "F ACE", + "ry s", + "r ys", + "Ex tract", + "Ext ract", + "Extra ct", + "il ians", + "ili ans", + "ilia ns", + "pl ug", + "▁a té", + "▁at é", + "и л", + "▁like wise", + "▁L il", + "▁Li l", + "▁Cam peonato", + "AUT O", + "AU TO", + "▁M eta", + "▁Me ta", + "▁Met a", + "▁ Meta", + "re no", + "ren o", + "r eno", + "▁Trans fer", + "▁ Transfer", + "▁Mich elle", + "▁Michel le", + "▁Miche lle", + "bi s", + "b is", + "ń st", + "зо н", + "з он", + "▁C ultural", + "com pass", + "comp ass", + "▁my sql", + "▁ mysql", + "▁cancel led", + "▁cancell ed", + "▁ ’", + "to o", + "t oo", + "▁re bell", + "▁reb ell", + "▁rebel l", + "ég e", + "é ge", + "os z", + "o sz", + "▁com poser", + "▁comp oser", + "▁compos er", + "}\" )", + "} \")", + "▁des erves", + "▁deserve s", + "▁oh ne", + "▁J ed", + "▁Je d", + "K ernel", + "▁pract ition", + "▁in door", + "▁ind oor", + "▁config urations", + "▁configuration s", + "▁m eth", + "▁me th", + "▁met h", + "+ (", + "Quest ion", + "▁bl own", + "▁blo wn", + "▁blow n", + ") '", + "▁Ar gs", + "▁Arg s", + "▁ Args", + "F ake", + "▁d even", + "▁de ven", + "▁dev en", + "istrz ost", + "na io", + "▁\" {", + "▁L it", + "▁Li t", + "com ed", + "co med", + "come d", + "c omed", + "▁st am", + "▁sta m", + "▁pl ugins", + "▁plugin s", + "▁plug ins", + "▁ plugins", + "▁travel ling", + "▁trav elling", + "na ire", + "n aire", + "▁aut onom", + "▁auto nom", + "STRU CT", + "n h", + "né es", + "née s", + "n ées", + "▁consider ably", + "ко р", + "к ор", + "B G", + "▁lad der", + "▁h ast", + "▁has t", + "▁ha st", + "iz ado", + "iza do", + "▁s ele", + "▁se le", + "▁sel e", + "▁W ere", + "▁We re", + "▁Wer e", + "ar don", + "ard on", + "ardo n", + "B ank", + "bund le", + "b undle", + "▁anticip ated", + "▁C ot", + "▁Co t", + "▁else if", + "▁ elseif", + "▁Bl ues", + "▁Blue s", + "▁fil tered", + "▁filter ed", + "▁a uction", + "▁au ction", + "ed uc", + "edu c", + "e duc", + "▁Ex pression", + "▁Express ion", + "▁Exp ression", + "▁ Expression", + "in x", + "i nx", + "▁s ucks", + "▁su cks", + "▁suc ks", + "▁suck s", + "▁ма я", + "EL L", + "E LL", + "ющи й", + "▁Hud son", + "it ä", + "на ми", + "▁fem me", + "in ho", + "▁e vt", + "▁ev t", + "▁ evt", + "istribut ions", + "istribution s", + "▁r uss", + "▁ru ss", + "▁rus s", + "▁pet ition", + "▁petit ion", + "▁г ла", + "▁ гла", + "Si g", + "S ig", + "▁T ut", + "▁Tu t", + "Part ial", + "Ent ities", + "▁b ears", + "▁be ars", + "▁bear s", + "▁h ollow", + "▁hol low", + "__ [\"", + "▁R is", + "ț ă", + "dim s", + "di ms", + "d ims", + "▁compl ained", + "▁complain ed", + "▁m apped", + "▁map ped", + "▁ma pped", + "▁авгу ста", + "▁initi atives", + "▁initiative s", + "▁own s", + "ch ez", + "che z", + "▁dis pon", + "▁disp on", + "▁m ush", + "▁mus h", + "▁mu sh", + "q s", + "▁er folg", + "▁Nor weg", + "▁c et", + "▁ce t", + "im ag", + "ima g", + "i mag", + "▁исто ри", + "▁ни х", + "▁ них", + "Un til", + "U ntil", + "▁s talk", + "▁st alk", + "▁П ра", + "uv o", + "u vo", + "ie rz", + "ier z", + "ri eben", + "rie ben", + "rieb en", + "X T", + "ic als", + "ical s", + "ica ls", + "std out", + "▁extra cted", + "▁extract ed", + "▁Im ages", + "▁Image s", + "▁ Images", + "un def", + "und ef", + "unde f", + "u ndef", + "▁L é", + "▁accommod ation", + "▁T ouch", + "▁To uch", + "▁ Touch", + "▁intent ions", + "▁intention s", + "▁concent rated", + "▁concentr ated", + "▁concentrate d", + "▁Насе ление", + "▁ut ilis", + "▁util is", + "▁сле д", + "▁ след", + "li f", + "l if", + "▁comp ris", + "▁compr is", + "▁с бор", + "med ium", + "medi um", + "St ates", + "State s", + "Stat es", + "▁Би ография", + "▁Fa ith", + "U A", + "ADD RESS", + "▁r ated", + "▁rate d", + "▁rat ed", + "▁ra ted", + "▁ rated", + "▁R ena", + "▁Re na", + "▁Ren a", + "▁C ache", + "▁Ca che", + "▁ Cache", + "▁pe que", + "▁un used", + "▁unus ed", + "▁ unused", + "ni m", + "n im", + "ol ding", + "old ing", + "▁N r", + "R ay", + "ur ls", + "url s", + "▁em issions", + "▁emission s", + "I r", + "▁m å", + "be ar", + "b ear", + "▁L ub", + "▁Lu b", + "▁Out side", + "min ded", + "mind ed", + "▁PRO VID", + "▁s ó", + "▁civil ian", + "Find er", + "Fin der", + "Fi nder", + "F inder", + "▁achie ving", + "mod ified", + "la ne", + "lan e", + "l ane", + "Se nder", + "Send er", + "S ender", + "▁Cr ime", + "▁Crim e", + "REQ UI", + "▁open ly", + "▁Belg ium", + "ic ity", + "ici ty", + "icit y", + "i city", + "▁M az", + "▁Ma z", + "▁st agger", + "▁stag ger", + "}} $,", + "}}$ ,", + "} }$,", + "na te", + "nat e", + "n ate", + "'' '", + "' ''", + "▁Ge off", + "ll i", + "l li", + "Su ite", + "▁D istribution", + "▁я кий", + "Com bo", + "Comb o", + "ho oks", + "hook s", + "▁F ight", + "▁Fig ht", + "▁Fi ght", + "Set s", + "Se ts", + "S ets", + "▁m k", + "▁ mk", + "▁gu ides", + "▁guide s", + "▁guid es", + "▁princip ale", + "▁principal e", + "Pre ferences", + "ti ny", + "t iny", + "ap pen", + "app en", + "appe n", + "a ppen", + "▁ru ined", + "▁ruin ed", + "▁sl iding", + "▁slid ing", + "▁Z en", + "▁Ze n", + "▁oct ubre", + "pos er", + "po ser", + "pose r", + "p oser", + "▁F lag", + "▁Fl ag", + "▁ Flag", + "▁b oom", + "▁bo om", + "▁Det ect", + "▁activ ation", + "▁обра зова", + "▁entertain ing", + "▁entert aining", + "▁protect ive", + "ál l", + "á ll", + "▁Fl ash", + "▁mid st", + "▁mi dst", + "ствен ной", + "▁Ph D", + "ij ing", + "iji ng", + "cl ub", + "get C", + "▁tro uve", + "▁trou ve", + "am bers", + "amb ers", + "amber s", + "▁g reed", + "▁gr eed", + "▁gre ed", + "am arin", + "ama rin", + "amar in", + "▁suspic ious", + "▁susp icious", + "▁dep uty", + "▁deput y", + "as per", + "asp er", + "▁fun ded", + "▁fund ed", + "al one", + "alo ne", + "▁t ract", + "▁tr act", + "▁tra ct", + "▁ tract", + "▁R ating", + "▁Ra ting", + "▁Rat ing", + "ad ays", + "ada ys", + "a days", + "▁st att", + "▁stat t", + "▁sta tt", + "▁Priv acy", + "▁_ _(", + "▁__ (", + "▁ __(", + "▁f ights", + "▁fight s", + "á j", + "\\ ]", + "ag h", + "a gh", + "or na", + "orn a", + "▁Diam ond", + "▁pro totype", + "▁proto type", + "▁prot otype", + "▁ prototype", + "▁Str ateg", + "ha do", + "had o", + "h ado", + "▁l ungs", + "▁lung s", + "▁lun gs", + "Pro totype", + "Proto type", + "ließ lich", + "▁d ive", + "▁di ve", + "▁div e", + "co v", + "c ov", + "▁M ist", + "▁Mi st", + "▁Mis t", + "▁T ypes", + "▁Type s", + "▁Ty pes", + "▁Typ es", + "▁ Types", + "▁di agonal", + "▁p review", + "▁pre view", + "▁prev iew", + "▁ preview", + "▁Cont ainer", + "▁ Container", + "DESC RIP", + "▁brit ann", + "▁C ord", + "▁Co rd", + "▁Cor d", + "ak ov", + "ako v", + "a kov", + "▁far ming", + "▁farm ing", + "▁p ère", + "▁k ills", + "▁kill s", + "▁kil ls", + "▁Car ib", + "▁Ca rib", + "ћ и", + "▁А л", + "? ;", + "▁пи са", + "▁ писа", + "▁En sure", + "par sed", + "parse d", + "pars ed", + "än ge", + "äng e", + "▁D elta", + "▁Del ta", + "▁ Delta", + "▁g aining", + "▁gain ing", + "▁ga ining", + "▁n oting", + "▁not ing", + "▁no ting", + "▁B arb", + "▁Bar b", + "▁Ba rb", + "▁фев ра", + "▁фе вра", + "Em p", + "E mp", + "▁{ })", + "▁{} )", + "▁ {})", + "▁sy ntax", + "▁syn tax", + "▁synt ax", + "W alk", + "▁P ere", + "▁Per e", + "▁Pe re", + "Is Null", + "▁U V", + "▁ UV", + "▁ret val", + "▁ retval", + "▁sim plicity", + "▁simpl icity", + "▁rein force", + "Lin q", + "▁diff usion", + "▁dis orders", + "▁disorder s", + "ât re", + "â tre", + "ui ty", + "uit y", + "u ity", + "▁hel pless", + "▁help less", + "Me asure", + "▁com pression", + "▁comp ression", + "▁compr ession", + "▁compress ion", + "▁Co al", + "olut ely", + "olute ly", + "og ue", + "o gue", + "▁up ward", + "▁Block ly", + "▁b ride", + "▁br ide", + "parse Int", + "▁is olation", + "▁isol ation", + "▁regul atory", + "ș ti", + "ric ane", + "м б", + "▁с ло", + "▁ сло", + "▁sa lad", + "▁sal ad", + "we i", + "w ei", + "▁B asket", + "▁Bas ket", + "▁M ON", + "▁MO N", + "▁ MON", + "\"> &", + "\" >&", + "do ors", + "door s", + "▁K ill", + "▁Kil l", + "▁Ki ll", + "▁conspir acy", + "▁M iles", + "▁Mil es", + "▁Mi les", + "wa nt", + "wan t", + "w ant", + "Mod ifier", + "▁batter ies", + "▁batt eries", + "iv as", + "iva s", + "i vas", + "▁att endance", + "▁attend ance", + "▁AUT H", + "▁AU TH", + "▁ AUTH", + "▁с ві", + ".. .,", + "... ,", + "▁aggreg ate", + "▁de struct", + "▁dest ruct", + "▁four teen", + "▁м ет", + "▁ме т", + "▁ мет", + "▁both ered", + "▁bother ed", + "el te", + "elt e", + "e lte", + "▁m ism", + "▁mis m", + "▁mi sm", + "▁res ting", + "▁rest ing", + "▁P ars", + "▁Par s", + "▁Pa rs", + "▁ Pars", + "▁id le", + "▁ idle", + "▁d eren", + "▁de ren", + "▁der en", + "▁dere n", + "▁di ary", + "▁dia ry", + "▁v ague", + "▁va gue", + "▁vag ue", + "▁margin al", + "▁marg inal", + "Wr it", + "W rit", + "Bo t", + "B ot", + "▁Met ro", + "▁e arning", + "▁earn ing", + "▁ear ning", + "hist oire", + "his toire", + "▁end orse", + "▁be ard", + "▁bear d", + "▁Chair man", + "ie b", + "i eb", + "▁neut r", + "▁neu tr", + "▁am bit", + "▁amb it", + "▁Leon ard", + "ban ds", + "band s", + "b ands", + "▁D ale", + "▁Da le", + "▁Dal e", + "▁ver ified", + "Al gorithm", + "Enumer able", + "op code", + "cast le", + "cas tle", + "š e", + "▁Venez uela", + "▁de scriptions", + "▁des criptions", + "▁description s", + "▁value d", + "▁val ued", + "▁chapter s", + "▁chap ters", + "▁I ls", + "▁Il s", + "▁cl arity", + "▁clar ity", + "▁tour ists", + "▁tourist s", + "Da n", + "D an", + "▁t ribe", + "▁tr ibe", + "▁tri be", + "▁trib e", + "▁г и", + "▁ ги", + "fol k", + "f olk", + "ac cur", + "acc ur", + "▁St ack", + "▁Sta ck", + "▁ Stack", + "▁adv ocate", + "▁advoc ate", + "▁G ene", + "▁Ge ne", + "▁Gen e", + "Im ages", + "Image s", + "▁rig id", + "▁con greg", + "▁congr eg", + "▁start up", + "▁dead line", + "co uld", + "cou ld", + "c ould", + "▁beg ann", + "▁began n", + "▁cal ci", + "▁calc i", + "▁Cir cle", + "▁Circ le", + "▁in cons", + "▁inc ons", + "▁incon s", + "aaaa aaaa", + "▁rub bed", + "ape ut", + "ua rio", + "uar io", + "u ario", + "worth y", + "wor thy", + "wort hy", + "▁уча сти", + "▁участ и", + "▁fam ília", + "▁synchron ized", + "▁unf air", + "rs p", + "r sp", + "▁soc ieties", + "▁societ ies", + "bo at", + "gr o", + "g ro", + "▁k at", + "▁ka t", + "▁ kat", + "▁p oker", + "▁po ker", + "▁pok er", + "▁l ocks", + "▁loc ks", + "▁lo cks", + "▁lock s", + "▁G F", + "▁ GF", + "▁re conc", + "▁recon c", + "▁Maur ice", + "▁Mau rice", + "__( /*!", + "▁ble eding", + "äs ident", + "▁по след", + "▁после д", + "▁deriv ative", + "ша я", + "cc ió", + "c ció", + "▁cr ushed", + "▁crush ed", + "▁tempor arily", + "▁co aches", + "▁coach es", + "▁Mo vement", + "▁Move ment", + "▁Mov ement", + "}} $.", + "}}$ .", + "} }$.", + "▁K yle", + "▁Ky le", + "▁S ohn", + "▁So hn", + "▁cre ator", + "▁creat or", + "ind ust", + "▁E rik", + "▁Er ik", + "▁se iz", + "▁sei z", + "▁dim ensional", + "▁dimension al", + "▁ dimensional", + "▁I st", + "▁Is t", + "▁pre val", + "▁pr eval", + "▁prev al", + "he ads", + "head s", + "▁про ти", + "▁determ ines", + "▁determine s", + "▁determin es", + "eg y", + "e gy", + "▁U INT", + "▁UI NT", + "▁ UINT", + "▁V olk", + "▁Vol k", + "pa wn", + "p awn", + "Ph oto", + "▁C olin", + "▁Col in", + "▁Co lin", + "ap propri", + "app ropri", + "ort ion", + "st eller", + "stell er", + "É tat", + "▁im ply", + "▁imp ly", + "▁impl y", + "▁t outes", + "▁to utes", + "▁tou tes", + "▁tout es", + "▁toute s", + "VO L", + "V OL", + "an ing", + "ani ng", + "a ning", + "Tool tip", + "ig ious", + "igi ous", + "▁e ternal", + "▁etern al", + "▁P oz", + "▁Po z", + "▁bank rupt", + "▁fail ures", + "▁failure s", + "uer te", + "▁вре ме", + "zu ng", + "z ung", + "▁t cp", + "▁tc p", + "▁ tcp", + "▁cont ainers", + "▁contain ers", + "▁container s", + "ou sel", + "ous el", + "ouse l", + "▁H IV", + "▁con ced", + "▁conc ed", + "▁conce d", + "▁sept iembre", + "gi rl", + "g irl", + "▁C ho", + "▁Ch o", + "▁f az", + "▁fa z", + "▁Up per", + "▁ Upper", + "▁For ces", + "▁Force s", + "äh lt", + "in ject", + "Re ceived", + "MA T", + "M AT", + "ag lia", + "ów nie", + "ówn ie", + "/ '", + "▁p ip", + "▁pi p", + "▁G est", + "▁Ge st", + "▁Ges t", + "▁l ado", + "▁la do", + "▁lad o", + "▁compat ibility", + "▁m are", + "▁mar e", + "▁ma re", + "▁ mare", + "▁Cle arly", + "▁Clear ly", + "vers ation", + "Ver s", + "V ers", + "▁ch ick", + "▁chi ck", + "▁organ ize", + "▁organiz e", + "▁econom ics", + "▁economic s", + "▁ancest ors", + "ME D", + "M ED", + "▁sc rub", + "▁scr ub", + "▁label ed", + "▁lab eled", + "▁п р", + "▁S uz", + "▁Su z", + "▁A str", + "▁As tr", + "▁Ast r", + "allow een", + "allo ween", + "rh s", + "r hs", + "as ci", + "asc i", + "▁C ancer", + "▁Can cer", + "▁H unt", + "▁Hun t", + "▁Hu nt", + "▁switch ing", + "▁R idge", + "Se q", + "S eq", + "▁gi ugno", + "bus iness", + "▁char ming", + "▁charm ing", + "▁I o", + "▁ Io", + "▁prés ident", + "ek ing", + "e king", + "í l", + "en h", + "e nh", + "pr it", + "p rit", + "erc ise", + "án ak", + "á nak", + "▁х ра", + "▁ хра", + "▁b ugs", + "▁bu gs", + "▁bug s", + "▁жи во", + "▁light ning", + "▁never theless", + "▁length s", + "G U", + "H idden", + "Act or", + "Ac tor", + "A ctor", + "To pic", + "Top ic", + "T opic", + "▁H orse", + "▁Hor se", + "ћ е", + "el ines", + "eline s", + "eli nes", + "elin es", + "e lines", + "▁trag edy", + "▁traged y", + "int endo", + "▁abund ance", + "▁ev ac", + "it ably", + "+\\ _\\", + "▁rec ib", + "ua ted", + "uate d", + "u ated", + "рі ї", + "▁fool ish", + "▁foo lish", + "▁t m", + "▁ tm", + "▁des pair", + "▁desp air", + "TO KEN", + "▁comp romise", + "▁comprom ise", + "▁Person en", + "▁Pers onen", + "▁investig ated", + "▁investigate d", + "▁ex clude", + "▁excl ude", + "▁telev is", + "▁tele vis", + "▁pull s", + "▁pul ls", + "▁according ly", + "▁accord ingly", + "▁f ő", + "▁Le ave", + "▁ Leave", + "oper ations", + "operation s", + "cri m", + "cr im", + "c rim", + "▁r hs", + "▁rh s", + "▁ rhs", + "▁form ally", + "▁formal ly", + "▁L ily", + "▁Li ly", + "▁Lil y", + "▁Com ments", + "▁Comm ents", + "▁Comment s", + "▁se ptember", + "▁sept ember", + "ie fs", + "ief s", + "▁tre asure", + "Http Servlet", + "ді в", + "д ів", + "▁dis claimer", + "lu ss", + "l uss", + "▁ка о", + "ro gen", + "rog en", + "r ogen", + "▁Start ing", + "▁Star ting", + "▁d ém", + "▁dé m", + "▁select ing", + "▁ ↘", + "▁О н", + "▁Pract ice", + "▁p orte", + "▁por te", + "▁port e", + "▁ porte", + "▁as sure", + "▁ass ure", + "▁frustr ated", + "S ink", + "▁A ri", + "▁Ar i", + "▁esc ort", + "ais es", + "ai ses", + "aise s", + "a ises", + "▁b ush", + "▁bu sh", + "▁bus h", + "▁Se ine", + "▁F ill", + "▁Fil l", + "▁Fi ll", + "▁ Fill", + "▁S ull", + "▁Su ll", + "▁Sul l", + "Do t", + "D ot", + "vi l", + "v il", + "un ing", + "uni ng", + "u ning", + "Render ing", + "Rend ering", + "sh ake", + "sha ke", + "пи си", + "пис и", + "pt e", + "p te", + "▁b end", + "▁be nd", + "▁ben d", + "▁jewel ry", + "▁Stock holm", + "▁Hon estly", + "! [", + "▁array s", + "▁arr ays", + "▁War ner", + "▁sh aft", + "▁sha ft", + "▁C ann", + "▁Can n", + "▁Ca nn", + "▁Pitt sburgh", + "ir ical", + "iri cal", + "i rical", + "au tre", + "aut re", + "▁R ück", + "▁gen naio", + "▁Ш а", + "an nte", + "ann te", + "annt e", + "ps hire", + "p shire", + "но логи", + "н ологи", + "ét a", + "é ta", + "▁pr inter", + "▁print er", + "▁prin ter", + "▁dam ages", + "▁damage s", + "▁Isa ac", + "▁Famil ie", + "Code s", + "Co des", + "C odes", + "th rift", + "no b", + "n ob", + "▁c av", + "▁ca v", + "▁techn ically", + "▁technical ly", + "▁I mm", + "▁Im m", + "▁tr icks", + "▁tri cks", + "▁trick s", + "EA R", + "E AR", + "▁Sub ject", + "▁ Subject", + "▁ne eding", + "▁need ing", + "▁G ir", + "▁Gi r", + "Bo ard", + "B oard", + "▁re he", + "▁rem inder", + "▁remind er", + "▁sh iver", + "K it", + "▁strugg les", + "▁struggle s", + "▁gen om", + "▁ge nom", + "im il", + "imi l", + "i mil", + "Reg istration", + "▁gl oves", + "▁glo ves", + "▁Z ur", + "▁Zu r", + "▁B eg", + "▁Be g", + "▁in clusive", + "▁incl usive", + "/ ,", + "og an", + "oga n", + "o gan", + "po que", + "cont rib", + "contr ib", + "ши н", + "ш ин", + "▁M ama", + "▁Ma ma", + "▁Mam a", + "print s", + "▁re named", + "▁ren amed", + "ють ся", + "ю ться", + "net dev", + "▁comp ile", + "▁ compile", + "▁ §", + "M UL", + "▁dr aws", + "▁draw s", + "co ck", + "c ock", + "▁сво и", + "▁M um", + "▁Mu m", + "sp ieler", + "spi eler", + "s pieler", + "▁n ail", + "▁na il", + "▁ nail", + "▁trans it", + "▁S aw", + "▁Sa w", + "▁com press", + "▁comp ress", + "▁compre ss", + "▁compr ess", + "▁ compress", + "▁purch ases", + "▁purchase s", + "▁per forms", + "▁perform s", + "▁dem ol", + "▁demo l", + "▁comm ence", + "▁C B", + "▁ CB", + "▁A ber", + "▁Ab er", + "▁c ush", + "▁cu sh", + "▁ком п", + "▁ру ко", + "▁Muham mad", + "▁Net flix", + "▁Environment al", + "No rm", + "N orm", + "▁w ir", + "null ptr", + "▁refuge es", + "до н", + "д он", + "▁B irmingham", + "New s", + "Ne ws", + "▁В се", + "Or ient", + "O rient", + "As sembly", + "▁introdu cing", + "fin der", + "find er", + "fi nder", + "f inder", + "▁scholar ship", + "▁scholars hip", + "▁ос нова", + "▁основ а", + "if a", + "i fa", + "Si ng", + "S ing", + "ib lic", + "ibli c", + "i blic", + "istribut ed", + "istribute d", + "▁depart ments", + "▁department s", + "CR EF", + "CRE F", + "C REF", + "▁Malays ia", + "CO NF", + "CON F", + "▁Cl aud", + "▁Bu ilt", + "▁ Built", + "RAN GE", + "Re direct", + "Red irect", + "LE ASE", + "-- -------", + "---- -----", + "-------- -", + "--- ------", + "----- ----", + "------ ---", + "------- --", + "- --------", + "▁П у", + "▁n umpy", + "▁num py", + "▁project ed", + "▁remind s", + "▁- *-", + "ib ling", + "ibli ng", + "i bling", + "▁s lower", + "▁sl ower", + "▁slow er", + "op p", + "o pp", + "ro pic", + "rop ic", + "r opic", + "▁Mont real", + "▁detect ive", + "TH READ", + "▁qu é", + "▁R osa", + "▁Ro sa", + "▁Ros a", + "▁seven th", + "▁sevent h", + "Col ors", + "Color s", + "de mo", + "dem o", + "▁E sta", + "▁Est a", + "▁Es ta", + "ff f", + "f ff", + "ick ets", + "icket s", + "Gr e", + "G re", + "á b", + "bo ost", + "▁Go ing", + "▁Su ite", + "▁ Suite", + "▁adapt ation", + "▁j ours", + "▁jour s", + "▁jo urs", + "▁jou rs", + "▁ jours", + "▁Or th", + "▁Ort h", + "х і", + "Fig ure", + "▁su pers", + "▁sup ers", + "▁super s", + "▁access ories", + "we ak", + "▁dist ress", + "fr ied", + "f ried", + "▁go og", + "ка з", + "▁far mer", + "▁farm er", + "it ational", + "itation al", + "itat ional", + "Go ld", + "G old", + "▁ass hole", + "▁assh ole", + "▁Cont roller", + "▁Control ler", + "▁ Controller", + "▁ар хи", + "To o", + "T oo", + "▁mol to", + "▁p ropri", + "▁prop ri", + "▁ propri", + "▁al go", + "▁alg o", + "Af f", + "A ff", + "re sc", + "res c", + "r esc", + "▁D y", + "▁con gr", + "▁T es", + "▁Te s", + "▁W IN", + "▁ WIN", + "de serialize", + "des erialize", + "sy n", + "s yn", + "▁chem istry", + "m iddle", + "▁an imated", + "▁anim ated", + "▁K um", + "▁Ku m", + "file Name", + "Amer ica", + "▁dr ums", + "▁dru ms", + "▁drum s", + "▁program a", + "▁n ej", + "▁ne j", + "▁ nej", + "Read Only", + "▁Б ра", + "-- -----", + "---- ---", + "--- ----", + "----- --", + "------ -", + "- ------", + "Mut ex", + "Mu tex", + "un ned", + "unn ed", + "ynam ics", + "ynamic s", + "co system", + "cos ystem", + "▁R ect", + "▁Re ct", + "▁Rec t", + "▁ Rect", + "▁an ime", + "▁anim e", + "▁I BM", + "▁need le", + "es ser", + "ess er", + "esse r", + "▁incl u", + "▁inc lu", + "Le an", + "tr aining", + "tra ining", + "train ing", + "▁b our", + "▁bo ur", + "▁bou r", + "▁ bour", + "ab ases", + "abase s", + "aba ses", + "▁tak że", + "wa rz", + "war z", + "w arz", + "▁ste pping", + "▁step ping", + "▁T IME", + "▁ TIME", + "▁Ein stein", + "▁Log in", + "▁Lo gin", + "▁ Login", + "pon ential", + "ponent ial", + "De ad", + "D ead", + "in str", + "ins tr", + "inst r", + "▁ne ural", + "▁neu ral", + "▁neur al", + "▁ub ic", + "▁Init ialized", + "▁Initialize d", + "▁Initial ized", + "▁ Initialized", + "▁facil itate", + "G D", + "}{ (", + "} {(", + "D ark", + "▁n ag", + "▁na g", + "min ipage", + "Size s", + "Si zes", + "S izes", + "▁w orm", + "▁wor m", + "▁wo rm", + "bi as", + "bia s", + "b ias", + "Su ch", + "S uch", + "wick lung", + "▁sp ouse", + "▁spo use", + "▁surviv ors", + "er st", + "ers t", + "at ype", + "aty pe", + "a type", + "}) $,", + "})$ ,", + "} )$,", + "▁n l", + "▁ nl", + "▁cogn itive", + "▁o nde", + "▁on de", + "▁ onde", + "▁en abling", + "▁soc iet", + "▁soci et", + "▁c lan", + "▁cl an", + "▁ex cluded", + "▁excl uded", + "▁exclude d", + "▁th under", + "▁ent ropy", + "▁entr opy", + "▁fast est", + "RE EN", + "REE N", + "▁Vien na", + "▁fl owing", + "▁flo wing", + "▁flow ing", + "▁aff irm", + "al om", + "alo m", + "▁h ips", + "▁hi ps", + "▁hip s", + "▁can nab", + "▁st icks", + "▁stick s", + "▁cur riculum", + "▁ret ained", + "▁retain ed", + "▁ext ending", + "▁extend ing", + "ó z", + "he aded", + "head ed", + "ex c", + "e xc", + "▁je ho", + "▁for ests", + "▁fore sts", + "▁forest s", + "ma nia", + "man ia", + "m ania", + "▁C anal", + "▁Can al", + "▁Ca nal", + "▁S out", + "▁So ut", + "▁Sou t", + "▁B ahn", + "▁Ba hn", + "▁Bah n", + "▁T EXT", + "▁TE XT", + "▁ TEXT", + "▁др жа", + "▁User s", + "▁Us ers", + "▁Use rs", + "▁ Users", + "▁G EN", + "▁ GEN", + "sl ash", + "ben falls", + "Text Field", + "▁r av", + "▁ra v", + "▁ rav", + "▁continu ously", + "▁continuous ly", + "IT ER", + "ITE R", + "I TER", + "▁Jen ny", + "▁Jenn y", + "ch os", + "cho s", + "c hos", + "▁am big", + "▁amb ig", + "▁ж ур", + "Aut ow", + "Auto w", + "▁V eter", + "▁Ve ter", + "▁dest in", + "H om", + "au ge", + "aug e", + "a uge", + "▁com mod", + "▁comm od", + "▁gar lic", + "< =", + "▁dram atically", + "▁dramatic ally", + "CA N", + "C AN", + "an cers", + "ance rs", + "anc ers", + "ancer s", + "() }", + "( )}", + "gh ai", + "▁tw ee", + "▁twe e", + "▁сент ября", + "GP U", + "G PU", + "▁B omb", + "▁Bo mb", + "▁young est", + "▁c age", + "▁ca ge", + "ok s", + "o ks", + "ic hes", + "ich es", + "iche s", + "i ches", + "▁T ests", + "▁Te sts", + "▁Test s", + "▁Tes ts", + "▁ Tests", + "sk ý", + "cur y", + "cu ry", + "c ury", + "na ls", + "nal s", + "n als", + "ț a", + "▁V oice", + "▁Vo ice", + "Depend ency", + "v f", + "e ous", + "▁Z a", + "▁am ateur", + "▁G host", + "▁Gh ost", + "▁dis ability", + "▁Вла ди", + "▁rev enge", + "▁reven ge", + "Trans lation", + "▁cour tesy", + "ски я", + "▁bl ob", + "▁blo b", + "▁ blob", + "ä ß", + "ó j", + "▁print s", + "▁prin ts", + "▁ prints", + "▁pro ves", + "▁pr oves", + "▁prov es", + "▁prove s", + ">? [<", + "▁ut ils", + "▁util s", + "▁ utils", + "ty pen", + "type n", + "typ en", + "▁t erra", + "▁ter ra", + "▁terr a", + "▁ terra", + "▁min eral", + "▁mine ral", + "▁miner al", + "▁war rior", + "▁ме ст", + "▁D S", + "▁ DS", + "Em b", + "E mb", + "get Data", + "ли чи", + "лич и", + "▁sa fer", + "▁saf er", + "▁safe r", + "▁com une", + "▁comun e", + "▁hier archy", + "Cred entials", + "res se", + "ress e", + "r esse", + "gr av", + "gra v", + "g rav", + "lo gs", + "log s", + "l ogs", + "br os", + "bro s", + "b ros", + "BUT TON", + "lit eral", + "liter al", + "l iteral", + "▁S r", + "an tal", + "ant al", + "anta l", + "▁mer cy", + "▁merc y", + "DA P", + "D AP", + "▁Mag gie", + "▁sust ained", + "▁sustain ed", + "N M", + "Re view", + "Rev iew", + "▁Buen os", + "▁de aler", + "▁deal er", + "en es", + "ene s", + "e nes", + "▁file Name", + "▁ fileName", + "bb ra", + "b bra", + "ро ма", + "ром а", + "Inst all", + "▁Mor ning", + "LE T", + "L ET", + "ip a", + "i pa", + "G a", + "го в", + "г ов", + "▁Sche dule", + "▁ Schedule", + "▁rep orters", + "▁report ers", + "▁reporter s", + "▁pecul iar", + "▁sup plier", + ")$ -", + ") $-", + "ë l", + "▁roll s", + "▁né cess", + "▁p reg", + "▁pre g", + "▁pr eg", + "▁Re yn", + "▁sur render", + "▁contribut ing", + ")+ \\", + ") +\\", + "PRO P", + "PR OP", + "P ROP", + "▁dec imal", + "▁Town ship", + "gr p", + "g rp", + "▁terror ist", + "pt o", + "p to", + "on en", + "one n", + "o nen", + "▁Polit ics", + "▁Pe arl", + "▁Pear l", + "▁pil low", + "▁pill ow", + "▁gr ades", + "▁grad es", + "▁grade s", + "▁gra des", + "▁ grades", + "TH E", + "T HE", + "▁num ero", + "▁numer o", + "▁nu mero", + "i NdEx", + "M igration", + "PE ND", + "P END", + "ph oto", + "▁cent ered", + "▁center ed", + "▁r het", + "▁rh et", + "egr ünd", + "▁laund ry", + "get Node", + "▁est imation", + "▁estim ation", + "▁I v", + "▁wh oles", + "▁who les", + "▁whole s", + "ше ния", + "▁const itutional", + "▁constitution al", + "am ination", + "amin ation", + "▁Municip al", + "ad t", + "a dt", + "th y", + "t hy", + "▁pub li", + "▁di cembre", + "▁dic embre", + "▁dice mbre", + "` )", + "▁Ch rome", + "ef e", + "e fe", + "con g", + "co ng", + "c ong", + "bre aking", + "break ing", + "at ched", + "atch ed", + "es tr", + "est r", + "e str", + "▁i di", + "▁id i", + "▁ idi", + "VER Y", + "V ERY", + "▁app el", + "▁ap pel", + "▁appe l", + "▁Techn ical", + "tc x", + "t cx", + "DO UBLE", + "se k", + "s ek", + "hu ng", + "h ung", + "▁A ur", + "▁Au r", + "coll apse", + "▁adv ise", + "▁advis e", + "▁Pr imary", + "▁Pri mary", + "▁Prim ary", + "▁ Primary", + "ia z", + "i az", + "▁a nten", + "▁an ten", + "▁ant en", + "▁ante n", + "▁ anten", + "▁bro ader", + "▁broad er", + "▁ju nio", + "▁jun io", + "▁juni o", + "▁w ool", + "▁wo ol", + "▁hat red", + "▁ex agger", + "Con v", + "Co nv", + "kt ur", + "▁em peror", + "▁Pack age", + "▁ Package", + "TD M", + "T DM", + "\\{ \\", + "\\ {\\", + "whe el", + "▁fe as", + "▁js ou", + "", + "< ?>", + "INST ANCE", + "▁ch ant", + "▁cha nt", + "▁ chant", + "▁Re fer", + "▁Ref er", + "▁S hir", + "▁Sh ir", + "▁ве ка", + "▁Me eting", + "▁Meet ing", + "▁n v", + "▁ nv", + "▁associ ations", + "▁association s", + "it ations", + "itation s", + "itat ions", + "or um", + "o rum", + "▁t ires", + "▁ti res", + "▁tire s", + "▁tir es", + "▁d ash", + "▁da sh", + "▁das h", + "▁ dash", + "▁} ));", + "▁}) );", + "ar to", + "art o", + "▁Ed inburgh", + "W T", + "▁inv ented", + "▁invent ed", + "ve h", + "v eh", + "▁Hind u", + "▁Насе лення", + "▁ur gent", + "▁urg ent", + "▁urge nt", + "text color", + "we rp", + "wer p", + "▁det ector", + "▁detect or", + "▁al tered", + "▁alt ered", + "▁alter ed", + "▁t b", + "▁ tb", + "▁N aval", + "▁Na val", + "▁Nav al", + "▁mem br", + "style sheet", + "styles heet", + "un ts", + "unt s", + "▁nut rition", + "▁S ylv", + "▁Sy lv", + "▁e numer", + "▁en umer", + "▁enum er", + "▁m ines", + "▁min es", + "▁mi nes", + "▁mine s", + "▁l itter", + "▁lit ter", + "▁litt er", + "ž í", + "con current", + "▁sw allow", + "Si r", + "S ir", + "tal k", + "t alk", + "▁de utschen", + "▁deutsch en", + "re peat", + "▁dom ains", + "▁domain s", + "▁Mc Donald", + "▁cand le", + "▁pl ural", + "▁sharp ly", + "▁shar ply", + "orig ine", + "origin e", + "▁c andy", + "▁can dy", + "▁cand y", + "▁kilomet res", + "▁power ed", + "▁pow ered", + "▁ powered", + "▁s ep", + "▁se p", + "▁ sep", + "▁S oci", + "▁So ci", + "▁Soc i", + "▁Ber nie", + "▁Bern ie", + "GE NER", + "GEN ER", + "Ex per", + "Exp er", + "▁Al low", + "▁All ow", + "▁ Allow", + "▁Ern st", + "▁Re becca", + "▁Cont ribut", + "ro utes", + "rou tes", + "route s", + "r outes", + "▁s uffix", + "▁suff ix", + "▁ju lio", + "▁jul io", + "▁juli o", + "▁provinc ial", + "▁provincia l", + "▁provin cial", + "▁appreci ation", + "Us ing", + "U sing", + "abs olute", + "▁cr icket", + "W ould", + "▁Equip ment", + "▁tort ure", + "на х", + "ut ton", + "utt on", + "че ство", + "▁out break", + "▁prevent ing", + "▁mad re", + "▁ret ire", + "end region", + "▁f ais", + "▁fa is", + "▁remember ing", + "▁Al ban", + "▁Alb an", + "▁a rist", + "▁ar ist", + "▁work out", + "▁u z", + "▁ uz", + "as to", + "ast o", + "a sto", + "fort unate", + "fortun ate", + "▁p aste", + "▁past e", + "▁pas te", + "▁pa ste", + "▁M R", + "▁ MR", + "▁o tra", + "▁ot ra", + "S v", + "an gen", + "ang en", + "ange n", + "▁S ierra", + "▁Si erra", + "▁n au", + "▁na u", + "▁s era", + "▁se ra", + "▁ser a", + "$ ~", + "▁cos ì", + ")( (", + ") ((", + "▁propos als", + "▁proposal s", + "it te", + "itt e", + "▁P ero", + "▁Per o", + "▁Pe ro", + "▁te nant", + "▁ten ant", + "▁ tenant", + "Y P", + "▁Param eter", + "▁ Parameter", + "sp ell", + "spe ll", + "▁e merge", + "▁emer ge", + "▁g ek", + "▁ge k", + "ol ence", + "olen ce", + "ot os", + "oto s", + "o tos", + "▁witness es", + "▁watch es", + "▁wat ches", + "▁A ch", + "▁Ac h", + "Cr oss", + "C ross", + "▁янва ря", + "; }", + "▁O NE", + "▁ON E", + "▁ ONE", + "▁care ers", + "▁career s", + "▁faith ful", + "▁J our", + "▁Jo ur", + "▁Gener ate", + "▁Gene rate", + "▁ Generate", + "▁ию ля", + "▁recommend ation", + "w b", + "sk ich", + "ski ch", + "bold math", + "▁orig ins", + "▁origin s", + "▁spin ning", + "▁// \r", + "▁bomb s", + "▁bom bs", + "min ister", + "I o", + "öl ker", + "Autow ired", + "um per", + "ump er", + "ich ael", + "▁contribut ors", + "▁contributor s", + "▁n asty", + "▁na sty", + "▁nas ty", + "▁nast y", + "▁d rap", + "▁dr ap", + "▁Bud apest", + "ur ious", + "uri ous", + "hi d", + "h id", + "▁wel comed", + "▁welcome d", + "▁w agon", + "▁wa gon", + "▁Ва си", + "▁embarrass ed", + "▁Har vey", + "Lo s", + "L os", + "▁S ter", + "▁St er", + "▁Ste r", + "▁enjoy able", + "ör t", + "ö rt", + "Mill is", + "-- )", + "- -)", + "▁d ashed", + "▁das hed", + "▁dash ed", + "\"> < ?", + "\" >' ,", + "> ',", + "▁all iance", + "ic ism", + "ici sm", + "▁NAS A", + "▁NA SA", + "▁p ode", + "▁po de", + "▁pod e", + "č ní", + "▁respon ding", + "▁respond ing", + "▁bl owing", + "▁blo wing", + "▁blow ing", + "ic ké", + "ick é", + "ва но", + "ван о", + "▁H off", + "▁Ho ff", + "▁Hof f", + "MB ER", + "M BER", + "▁civil ization", + "ar ía", + "a ría", + "Un lock", + "ge ts", + "get s", + "g ets", + "no d", + "n od", + "▁S TE", + "▁ST E", + "▁con science", + "PE G", + "ch anging", + "chan ging", + "▁Rich mond", + "ling ton", + "l ington", + "ocr atic", + "▁trav és", + "▁ф ран", + "▁up dating", + "process ing", + "Al ex", + "A lex", + "▁mil itar", + "▁milit ar", + "▁pse udo", + "▁pseud o", + "str len", + "▁be have", + "▁beh ave", + "▁behav e", + "▁distinct ive", + "▁E c", + "▁c x", + "▁ cx", + "▁journal ists", + "▁journalist s", + "vo lt", + "vol t", + "v olt", + "▁sp un", + "▁d urable", + "▁dur able", + "▁pro position", + "▁propos ition", + "▁ proposition", + "thread s", + "▁tw entieth", + "▁ф і", + "▁ фі", + "en son", + "ens on", + "enso n", + "▁self ish", + "▁sel fish", + "ar ium", + "ari um", + "a rium", + "▁de cid", + "▁dec id", + "▁ха рак", + "▁psy chiat", + "▁psych iat", + "g d", + "Z Z", + "ug u", + "u gu", + "▁i ds", + "▁id s", + "▁ ids", + "Man aged", + "▁Leg isl", + "ancell ationToken", + "▁gr ants", + "▁gran ts", + "▁grant s", + "▁lie utenant", + "▁lieu tenant", + "▁Fle et", + "** /", + "* */", + "▁T ig", + "▁Ti g", + "▁accept s", + "▁system atic", + ", {\\", + "▁У кра", + "▁aus ge", + "▁dial ect", + "▁dia lect", + "st ri", + "str i", + "s tri", + "er me", + "erm e", + "▁B esch", + "▁Be sch", + "▁Bes ch", + "lo ve", + "lov e", + "l ove", + "S ensor", + "▁B IT", + "▁ BIT", + "▁т ру", + "▁mist aken", + "▁mistake n", + "p v", + "▁u tf", + "▁ut f", + "▁ utf", + "▁[ \\", + "▁ [\\", + "▁Geb iet", + "▁Mann schaft", + "PAR AMETER", + "▁u rb", + "▁ur b", + "▁ urb", + "▁R eed", + "▁Re ed", + "▁c ough", + "▁co ugh", + "▁cou gh", + "wa ld", + "wal d", + "w ald", + "▁L amb", + "▁La mb", + "▁Lam b", + "▁surv iving", + "▁surviv ing", + "▁s way", + "▁sw ay", + "▁с ве", + "WI SE", + "ä ger", + "f y", + "sk e", + "s ke", + "▁s og", + "▁so g", + "▁Im plement", + "▁Imp lement", + "▁ Implement", + "获 取", + "▁T ools", + "▁To ols", + "▁Tool s", + "▁Too ls", + "▁ Tools", + "▁ne wer", + "▁new er", + "▁exempl e", + "▁exem ple", + "▁l itt", + "▁li tt", + "▁lit t", + "▁вы пу", + "▁у прав", + "Em itter", + "Emit ter", + "IS ING", + "I SING", + "▁органи за", + "▁М і", + "▁Ex amples", + "▁Example s", + "▁I con", + "▁ Icon", + "Get ter", + "▁L ay", + "▁La y", + "▁Col lect", + "▁Coll ect", + "▁ Collect", + "Sa int", + "S aint", + "or able", + "ora ble", + "▁f ick", + "▁fi ck", + "ik h", + "i kh", + "sl ave", + "▁c lay", + "▁cl ay", + "▁W A", + "▁ WA", + "Re po", + "Rep o", + "▁Java Script", + "it r", + "i tr", + "pa id", + "p aid", + "▁home work", + "M iddleware", + "▁r éal", + "▁ré al", + "▁при зна", + "ê m", + "ès e", + "è se", + "▁W ells", + "▁Well s", + "▁Wel ls", + "▁e nero", + "▁en ero", + "▁ener o", + "emperature n", + "▁N aj", + "▁Na j", + "▁Re agan", + "▁comp elling", + "▁tri bes", + "▁trib es", + "▁tribe s", + "▁to String", + "▁ toString", + "pace s", + "pa ces", + "p aces", + "▁harm ful", + "▁Con se", + "▁Cons e", + "od io", + "odi o", + "▁m im", + "▁mi m", + "get Item", + "▁script s", + "▁ scripts", + "ra is", + "rai s", + "r ais", + "▁Ph ase", + "▁ Phase", + "▁An swer", + "▁$ |\\", + "▁$| \\", + "▁as sembled", + "el in", + "eli n", + "e lin", + "ph abet", + "pha bet", + "▁to ast", + "▁tut ti", + "▁tu tti", + "▁be zeichnet", + "Gre at", + "G reat", + "et tes", + "ett es", + "ette s", + "e ttes", + "▁дека бря", + "F ULL", + "▁re gener", + "▁reg ener", + "▁któ re", + "го р", + "г ор", + "is ce", + "isc e", + "▁t oda", + "▁to da", + "▁tod a", + "▁eth ical", + "i q", + "P t", + "ar in", + "ari n", + "a rin", + "ig os", + "igo s", + "i gos", + "▁work shops", + "▁workshop s", + "▁R oche", + "▁Ro che", + "▁Roc he", + "Get String", + "мини стратив", + "m ême", + "▁D aw", + "▁Da w", + "ar ians", + "ari ans", + "aria ns", + "arian s", + "▁imp acts", + "▁impact s", + "▁por table", + "▁port able", + ")- \\", + ") -\\", + "sh ots", + "shot s", + "▁re lev", + "▁rel ev", + "▁rele v", + "PR IV", + "PRI V", + "▁бу ла", + "ard less", + "ul ously", + "ulous ly", + "-- >", + "- ->", + "ol ent", + "ole nt", + "olen t", + "▁э того", + "▁это го", + "▁Gener ic", + "▁Gene ric", + "▁ Generic", + "▁* /,", + "▁*/ ,", + "▁comb inations", + "▁combination s", + "▁re jo", + "с публи", + "cap acity", + "▁tr aces", + "▁tra ces", + "▁trace s", + "▁op acity", + "▁ opacity", + "▁Off icial", + "ic ion", + "ici on", + "icio n", + "▁emotional ly", + "▁emotion ally", + "▁Jo el", + "▁Joe l", + "сько му", + "▁legend ary", + "▁p am", + "▁pa m", + "▁Tamb ién", + ". <", + "ib a", + "i ba", + "mi dt", + "mid t", + "бо м", + "▁en suite", + "Author ization", + "P ag", + "▁hel met", + "▁ter rito", + "▁terr ito", + "second ary", + "▁seg unda", + "▁W ire", + "▁Wi re", + "rec ated", + "▁inv oked", + "▁invoke d", + "▁Value Error", + "▁ф о", + "▁ фо", + "AL IGN", + "CUR RENT", + "\\ +\\_\\", + "▁comp ilation", + "æ r", + "▁Pal mar", + "▁Palm ar", + "▁influ ences", + "▁influence s", + "/ :", + "M ix", + "NO P", + "N OP", + "ec onom", + "e conom", + "▁t ucked", + "▁} );\r", + "▁}); \r", + "▁}) ;\r", + "▁ });\r", + "AN K", + "re ject", + "▁p ension", + "▁pens ion", + "▁gener ates", + "▁generate s", + "ч ё", + "▁in cap", + "▁inc ap", + "▁cl icked", + "▁click ed", + "▁f us", + "▁fu s", + "our ses", + "ours es", + "ourse s", + "▁E aster", + "▁East er", + "% ;", + "zi n", + "z in", + "▁oblig ations", + "▁obligation s", + "▁T ips", + "▁Tip s", + "▁Ti ps", + "}; \r", + "} ;\r", + ".\" _", + "▁B SD", + "▁BS D", + "át ica", + "▁ex pose", + "▁exp ose", + "▁expos e", + "Par s", + "P ars", + "▁Am anda", + "ку п", + "▁gu essed", + "▁guess ed", + "ds i", + "d si", + "▁Le ip", + "Br oad", + "Bro ad", + "B road", + "▁Hug hes", + "▁Hugh es", + "i é", + "▁W ahl", + "▁Wa hl", + "▁former ly", + "Rel ative", + "▁Y u", + "▁Mount ains", + "▁Mountain s", + "▁E num", + "▁En um", + "▁ Enum", + "▁str ang", + "▁stra ng", + "_ -", + "re cht", + "rec ht", + "vi v", + "v iv", + "pa use", + "p ause", + "▁Lond res", + "▁el bow", + "▁Hawai i", + "▁Cas ino", + "Th reshold", + "Un its", + "Unit s", + "In clude", + "ит о", + "и то", + "as ury", + "▁ste ht", + "▁dam ned", + "▁damn ed", + "▁pack ets", + "▁packet s", + "▁W erk", + "▁Wer k", + "▁elev ator", + "ied ad", + "go vern", + "gov ern", + "g overn", + "▁CONTR ACT", + "ma ls", + "mal s", + "m als", + "▁re mem", + "▁rem em", + "▁ent onces", + "▁v as", + "▁va s", + "▁ vas", + "▁sym pathy", + "▁befind et", + "in cing", + "inc ing", + "Data Set", + "▁add itionally", + "▁addition ally", + "▁additional ly", + "▁mus ician", + "▁music ian", + "ше го", + "▁li stop", + "▁list op", + ">\" )", + "> \")", + "Print f", + "▁Fel ix", + "▁car ved", + "▁nice ly", + "▁nic ely", + "го м", + "ch ap", + "cha p", + "▁N ieder", + "▁Ni eder", + "▁Nie der", + "▁L av", + "▁La v", + "▁mod ifications", + "▁modification s", + "mo ment", + "m oment", + "▁bal con", + "▁depend ency", + "CK ET", + "▁van ished", + "▁f ighters", + "▁fight ers", + "▁fighter s", + "▁z unächst", + "io ctl", + "ioc tl", + "▁def ens", + "▁defe ns", + "▁N em", + "▁Ne m", + "Util ity", + "Ut ility", + "▁cur v", + "▁cu rv", + "▁DA MAGES", + "▁Ro gers", + "▁Rog ers", + "▁Roger s", + "▁grat itude", + "▁Den mark", + "ра я", + "gr pc", + "grp c", + "g rpc", + "▁j uni", + "▁ju ni", + "▁jun i", + "▁окт ября", + "▁imm ense", + "▁prevent ed", + "▁prev ented", + "▁fo am", + "▁Ex tra", + "▁Ext ra", + "▁ Extra", + "ai med", + "aim ed", + "▁C riteria", + "▁Crit eria", + "▁ Criteria", + "▁Sim ply", + "box es", + "▁Leg end", + "▁P layers", + "▁Play ers", + "▁Player s", + "▁Mer cedes", + "▁Merc edes", + "▁Br anch", + "▁ Branch", + "TER N", + "T ERN", + "om ena", + "ome na", + "omen a", + "▁incorpor ate", + "con de", + "co nde", + "cond e", + "c onde", + "▁Est ado", + "▁Esta do", + "▁w asted", + "▁was ted", + "▁wa sted", + "▁waste d", + "▁compl aining", + "▁complain ing", + "▁war riors", + "▁warrior s", + "ot er", + "ote r", + "o ter", + "▁э том", + "▁это м", + "▁con ten", + "▁cont en", + "▁co nten", + "▁machine ry", + "▁mach inery", + "▁techn ological", + "▁T D", + "▁ TD", + "▁g ras", + "▁gr as", + "▁gra s", + "▁minim ize", + "▁D oor", + "▁Do or", + "▁b zw", + "▁p rac", + "▁pr ac", + "▁pra c", + "TR EE", + "T REE", + "▁W ing", + "▁Win g", + "▁Wi ng", + "▁Trans action", + "▁ Transaction", + "▁M VT", + "▁Kle in", + "com mons", + "comm ons", + "common s", + "▁} {", + "▁ }{", + "▁Her itage", + "▁f ade", + "▁fa de", + "ро к", + "set Value", + "▁Wal lace", + "▁Wall ace", + "M X", + "▁A CT", + "▁AC T", + "▁ ACT", + "▁foot age", + "▁ent stand", + "ar ga", + "arg a", + "▁n ails", + "▁na ils", + "▁nail s", + "▁capital ism", + "▁G arc", + "▁Gar c", + "▁Ga rc", + "▁susp ension", + "il is", + "ili s", + "▁M ov", + "▁Mo v", + "uff led", + "uffle d", + "Ar c", + "A rc", + "▁Beaut iful", + "WA Y", + "W AY", + "Par allel", + "XX XX", + "di ag", + "▁D T", + "▁ DT", + "m q", + "Text View", + "ML E", + "M LE", + "en nen", + "enn en", + "enne n", + "▁infect ed", + "▁therap ist", + "IN GS", + "ING S", + "▁c idade", + "ъ н", + "▁p df", + "▁pd f", + "▁ pdf", + "▁b ump", + "▁bu mp", + "CT X", + "C TX", + "▁IN CLUDING", + "▁ INCLUDING", + "▁G ef", + "▁Ge f", + "ENT IAL", + "▁h andy", + "▁hand y", + "▁han dy", + "▁tempor al", + "▁temp oral", + "▁tempo ral", + "At A", + "IS H", + "I SH", + "▁Pat tern", + "▁ Pattern", + "▁l an", + "▁la n", + "▁ lan", + "ep endant", + "▁sh ining", + "id y", + "i dy", + "▁N T", + "▁ NT", + "▁F ran", + "▁Fr an", + "▁Fra n", + "▁nur ses", + "▁nurs es", + "▁nurse s", + "▁bet ray", + "▁sens ible", + "▁апре ля", + "▁' [", + "▁th irteen", + ")} _{", + ") }_{", + "▁No ah", + "INS ERT", + "ist ically", + "istic ally", + "▁Append ix", + "▁re cher", + "▁rec her", + "Re ceiver", + "▁der nier", + "▁derni er", + "л ла", + "ли за", + "▁Part ido", + "▁max imal", + "▁maxim al", + "sn ap", + "▁ча сть", + "▁част ь", + "▁час ть", + "ST OP", + "STO P", + "S TOP", + "▁ult ra", + "▁ul tra", + "▁dévelop p", + "▁t egen", + "▁te gen", + "▁Ч и", + "LI B", + "L IB", + "▁bas eline", + "▁base line", + "re load", + "rel oad", + "▁Ar bitro", + "▁k all", + "▁ka ll", + "c apture", + "Ar m", + "A rm", + "qu in", + "im pse", + "imp se", + "za s", + "z as", + "▁C and", + "▁Can d", + "▁Ca nd", + "▁br ains", + "▁brain s", + "▁bra ins", + "▁host ile", + "▁mar ble", + "oo ns", + "oon s", + "o ons", + "▁L oss", + "▁Los s", + "▁Lo ss", + "Meta Data", + "▁Rep ública", + "▁and ra", + "▁ andra", + "od en", + "ode n", + "o den", + "▁document ed", + "▁M oses", + "▁Mo ses", + "▁Mos es", + "od d", + "o dd", + "▁w ax", + "▁wa x", + "us ch", + "usc h", + "u sch", + "▁diagn osed", + "in kle", + "ink le", + "▁X box", + "▁seven ty", + "▁sevent y", + "ci as", + "cia s", + "c ias", + "▁nov iembre", + "Com pute", + "Comp ute", + "Comput e", + "}) ;\r", + "}); \r", + "} );\r", + "▁Philip pe", + "▁Philipp e", + "▁F ör", + "Le ave", + "▁s age", + "▁sa ge", + "▁sag e", + "▁un pre", + "▁Fort unately", + "▁a post", + "▁ap ost", + "ent ities", + "enti ties", + "▁el los", + "▁ell os", + "author ized", + "GB T", + "G BT", + "▁ins ist", + "▁insp ire", + "▁inspir e", + "Ma ss", + "M ass", + "▁r ôle", + "fe e", + "f ee", + "ip art", + "ipa rt", + "i part", + "це р", + "ц ер", + "un ate", + "una te", + "u nate", + "▁C NN", + ": }", + "▁unh appy", + "▁import ed", + "▁imp orted", + "H IGH", + "ring s", + "rin gs", + "r ings", + "▁In stance", + "▁Inst ance", + "▁ Instance", + "B ay", + "ag les", + "agle s", + "a gles", + "me e", + "m ee", + "ber y", + "be ry", + "b ery", + "▁St ories", + "▁Sto ries", + "▁Ch ase", + "▁Cha se", + "▁car riage", + "▁mis under", + "▁imag in", + "p w", + "▁M eter", + "▁Me ter", + "▁Met er", + "▁crow ds", + "▁crowd s", + "▁F ame", + "▁Fa me", + "sk ill", + "ski ll", + "s kill", + "▁c omed", + "▁com ed", + "▁co med", + "▁come d", + "▁ comed", + "▁r anch", + "▁ran ch", + "▁l acking", + "▁lack ing", + "▁lac king", + "▁sub mar", + "▁subm ar", + "ia nte", + "ian te", + "iant e", + "i ante", + "▁l anz", + "▁lan z", + "▁слу ж", + "-- ---------", + "---- -------", + "-------- ---", + "--- --------", + "----- ------", + "---------- -", + "------ -----", + "--------- --", + "------- ----", + "- ----------", + "▁ob ten", + "▁obt en", + "▁down stairs", + "Y N", + "rot ation", + "▁J esse", + "▁Jes se", + "▁Jess e", + "$ (\"#", + "▁p uls", + "▁pu ls", + "▁pul s", + "ir ling", + "irl ing", + "▁Sch aus", + "▁Sc haus", + "▁de ployed", + "▁deploy ed", + "▁{ }\",", + "▁{} \",", + "▁Mar vel", + "EN UM", + "E NUM", + "▁Mat hemat", + "▁Math emat", + "▁n n", + "▁ nn", + "com pet", + "comp et", + "k ów", + "bi l", + "b il", + "Wh ich", + "is ine", + "isi ne", + "▁r ude", + "▁ru de", + "▁n iveau", + "▁á rea", + "▁p rès", + "▁pr ès", + "at is", + "ati s", + "▁[... ]", + "fu r", + "f ur", + "om m", + "o mm", + "pack ed", + "p acked", + "ме не", + "мен е", + "м ене", + "script style", + "▁A th", + "▁At h", + "▁d esp", + "▁de sp", + "▁des p", + "elt emperaturen", + "▁tal ents", + "▁talent s", + "oc y", + "o cy", + "▁r aises", + "▁rais es", + "▁raise s", + "▁ra ises", + "LI MIT", + "L IMIT", + "▁editor ial", + "▁edit orial", + "▁An imal", + "▁Anim al", + "dr ive", + "d rive", + "▁рабо та", + "bs s", + "b ss", + "▁S ev", + "▁Se v", + "ep och", + "e poch", + "▁R C", + "▁ RC", + "UN USED", + "▁mand atory", + "( ?:", + "▁B in", + "▁Bi n", + "▁ Bin", + "▁synt hetic", + "▁g own", + "▁go wn", + "▁D ob", + "▁Do b", + "ka p", + "k ap", + "▁har mon", + "▁harm on", + "▁liber ty", + "▁libert y", + "▁R ice", + "▁Ric e", + "▁pray ers", + "▁pra yers", + "▁prayer s", + "▁m ise", + "▁mis e", + "▁mi se", + "▁conf using", + "▁le ap", + "▁arriv es", + "▁arr ives", + "▁arrive s", + "ka mp", + "k amp", + "▁th ats", + "▁that s", + "AC C", + "A CC", + "▁Param eters", + "▁Parameter s", + "▁ Parameters", + "▁о дно", + "▁од но", + "▁B io", + "▁Bi o", + "d ensity", + "▁gl impse", + "FO RE", + "FOR E", + "▁L isten", + "▁List en", + "▁Li sten", + "▁Liste n", + "▁Lis ten", + "▁ Listen", + "Pr ev", + "Pre v", + "P rev", + "}\\ ,\\", + "}\\, \\", + "} \\,\\", + "ку ль", + "▁S EC", + "▁SE C", + "▁ SEC", + "▁expl ored", + "▁explore d", + "▁explo red", + "▁mean time", + "▁meant ime", + "AI L", + "A IL", + "▁W P", + "▁ WP", + "▁r aison", + "▁rais on", + "▁ra ison", + "▁ex iste", + "▁exist e", + "▁l esser", + "▁les ser", + "▁less er", + "▁Valid ate", + "▁ Validate", + "▁ca ution", + "▁caut ion", + "us ta", + "ust a", + "u sta", + "he ading", + "head ing", + "EF F", + "E FF", + ".' \"", + ". '\"", + "▁Gil bert", + "▁lim itation", + "▁limit ation", + "▁ret our", + "▁Common wealth", + "▁gew ann", + "▁miser able", + "▁net working", + "▁network ing", + "▁ott obre", + "▁otto bre", + "▁D ise", + "▁Dis e", + "▁Di se", + "ed ges", + "edge s", + "▁s ede", + "▁se de", + "▁sed e", + "ви ча", + "вич а", + "un iform", + "uni form", + "▁дея тель", + "ir os", + "iro s", + "i ros", + "▁d esen", + "▁de sen", + "▁des en", + "▁p arc", + "▁par c", + "▁pa rc", + "▁R ico", + "▁Ric o", + "N s", + "gu id", + "gui d", + "g uid", + "or io", + "ori o", + "o rio", + "ave length", + "avel ength", + "▁G le", + "▁Gl e", + "ince ton", + "inc eton", + "Am az", + "A maz", + "Con struct", + "Const ruct", + "▁m x", + "▁ mx", + "▁V ern", + "▁Ver n", + "▁Ve rn", + "▁Gener ation", + "▁ Generation", + "J ack", + "ro mag", + "rom ag", + "▁vi agra", + "▁via gra", + "▁P eg", + "▁Pe g", + "▁Up dated", + "▁Update d", + "▁ Updated", + "▁over lap", + "▁overl ap", + "Event Args", + "к ро", + "▁* «", + "▁quest ioned", + "▁question ed", + "So uth", + "S outh", + "not ice", + "▁perman ently", + "▁permanent ly", + "ls t", + "l st", + "fi cie", + "fic ie", + "▁qu ella", + "▁que lla", + "▁quel la", + "▁college s", + "▁colle ges", + "▁colleg es", + "▁disappoint ment", + "▁Lu ft", + "img ur", + "▁trans itions", + "▁transition s", + "▁transit ions", + "▁s eller", + "▁sell er", + "▁sel ler", + "▁ию ня", + "▁O g", + "▁A DD", + "▁AD D", + "▁ ADD", + "▁P ays", + "▁Pa ys", + "▁Pay s", + "COMM AND", + "gr ades", + "grad es", + "grade s", + "gra des", + "▁fe bbra", + "▁C yr", + "▁Cy r", + "▁febbra io", + "et i", + "e ti", + "▁a rom", + "▁ar om", + "▁Cl aude", + "▁Claud e", + "▁UE FA", + "▁жи ве", + "▁Victor ian", + "▁Victoria n", + "ke eping", + "keep ing", + "kee ping", + "ê n", + "▁FIX ME", + "it ime", + "iti me", + "i time", + "ch estr", + "che str", + "ches tr", + "▁Sam sung", + "▁do ctrine", + "▁p ear", + "▁pe ar", + "▁Mediterr anean", + "▁Y a", + "▁v ault", + "▁va ult", + "▁Hist oric", + "▁Histor ic", + "▁se dan", + "▁sed an", + "▁he ated", + "▁heat ed", + "▁polít ica", + "Pro of", + ": {", + "fe m", + "f em", + "▁Frank furt", + "pect ives", + "pective s", + "M G", + "▁E ye", + "da i", + "d ai", + "▁res erves", + "▁reserv es", + "▁reserve s", + "NE R", + "N ER", + "▁tob acco", + "▁frag ments", + "▁fragment s", + "ic c", + "i cc", + "▁b ooth", + "▁bo oth", + "▁boot h", + "▁cru ise", + "▁Test ament", + "co la", + "col a", + "c ola", + "▁Le op", + "▁Leo p", + "▁n oon", + "▁no on", + "▁ noon", + "▁terr ified", + "v b", + "int el", + "inte l", + "al ie", + "ali e", + "a lie", + "▁ver ification", + "yst er", + "ys ter", + "y ster", + "AD ER", + "A DER", + "ch ied", + "chie d", + "chi ed", + "▁data sets", + "▁dat asets", + "▁dataset s", + "▁з і", + "▁ зі", + "▁m iem", + "▁mi em", + "▁mie m", + "ul ates", + "ula tes", + "ulate s", + "▁u uid", + "▁ uuid", + "▁Pict ures", + "▁Picture s", + "▁B rend", + "▁Br end", + "▁Bre nd", + "▁Bren d", + "Bill board", + "▁s tern", + "▁st ern", + "▁ste rn", + "▁ster n", + "▁de nom", + "▁den om", + "▁acc idents", + "▁accident s", + "с ня", + "▁p acking", + "▁pack ing", + "▁pac king", + "ци ја", + "ibli cal", + "iblic al", + "▁Та к", + "▁wh isk", + "▁whis k", + "▁l uego", + "▁lu ego", + "▁rect angle", + "▁ho oks", + "▁hook s", + "▁ hooks", + "▁neg lect", + "▁negl ect", + "▁s ober", + "▁so ber", + "▁sob er", + "pro position", + "Mult iple", + "Multi ple", + ":\" ,", + ": \",", + "▁b apt", + "▁ba pt", + "Par ts", + "Part s", + "P arts", + "▁S election", + "▁Se lection", + "▁Sel ection", + "▁Select ion", + "▁ Selection", + "▁Al pha", + "▁ Alpha", + "we ights", + "weight s", + "ha ll", + "hal l", + "h all", + "со б", + "с об", + "▁l ur", + "▁lu r", + "▁ép oca", + "▁re sted", + "▁r ested", + "▁res ted", + "▁rest ed", + "▁reste d", + "amb igu", + "▁taste s", + "▁tast es", + "amazon aws", + "▁conf ess", + "▁dic iembre", + "▁dici embre", + "im plement", + "impl ement", + "imp lement", + "▁absor ption", + "Ha l", + "H al", + "LE AN", + "▁Z ach", + "▁Za ch", + "▁free ze", + "▁fre eze", + "L BL", + "ST M", + "S TM", + "▁cal c", + "▁ca lc", + "▁ calc", + "={ ()", + "= */", + "▁b t", + "▁ bt", + "Re b", + "R eb", + "▁W ien", + "▁Wi en", + "an ska", + "ans ka", + "ansk a", + "▁s urn", + "▁su rn", + "▁sur n", + "iat ive", + "i ative", + "▁inv ån", + "C Y", + "▁l à", + "am ba", + "amb a", + "le en", + "lee n", + "l een", + "wa hl", + "w ahl", + "▁function ing", + "ți a", + "ț ia", + "get Context", + "ga rt", + "gar t", + "g art", + "▁о бе", + "▁об е", + "Pe n", + "P en", + "vi k", + "v ik", + "Sl ider", + "▁Ac cept", + "▁ Accept", + "Ga p", + "G ap", + "▁J orge", + "SI G", + "S IG", + "▁во с", + "▁го ло", + "▁г оло", + "▁period o", + "ш та", + "▁pat ches", + "▁patch es", + "ко ї", + "är e", + "ä re", + "eng ono", + "li sta", + "list a", + "l ista", + "hor n", + "ho rn", + "h orn", + "▁Com plex", + "▁Comp lex", + "▁ Complex", + "Se nt", + "S ent", + "tr fs", + "▁conv ex", + "▁conve x", + "Gener ation", + "▁міс це", + "com press", + "comp ress", + "▁S ax", + "▁Sa x", + "▁u id", + "▁ui d", + "▁ uid", + "▁Leb ens", + "▁Leben s", + "Com pletion", + "\\| _{", + "\\ |_{", + "in sky", + "ins ky", + "▁sc hon", + "▁sch on", + "▁m asters", + "▁ma sters", + "▁master s", + "▁mas ters", + "▁mast ers", + "in depend", + "inde pend", + "ne ys", + "ney s", + "▁l ied", + "▁li ed", + "▁lie d", + "▁a spir", + "▁asp ir", + "ч ні", + "▁break down", + "▁H arm", + "▁Har m", + "▁Ha rm", + "▁design ing", + "h f", + "▁Ang ela", + "▁Angel a", + "▁con fer", + "▁conf er", + "▁part ido", + "▁parti do", + "▁inter ference", + "ma o", + "m ao", + "▁absor bed", + "▁absorb ed", + "▁V all", + "▁Val l", + "▁Va ll", + "Error Code", + "▁Publish ing", + "va no", + "van o", + "v ano", + "BIT S", + "BI TS", + "B ITS", + "▁de er", + "▁Camp aign", + "▁g raz", + "▁gr az", + "▁gra z", + "CHAN GE", + "▁f eder", + "▁fe der", + "▁fed er", + "if fe", + "iff e", + "hand ed", + "han ded", + "h anded", + "c q", + "um bing", + "umb ing", + "▁un re", + "▁s iendo", + "▁si endo", + "▁sim pler", + "▁simple r", + "▁simpl er", + "wh y", + "w hy", + "ar ettes", + "are ttes", + "aret tes", + "arette s", + "an st", + "ans t", + "▁h ass", + "▁has s", + "▁ha ss", + "▁Enter prise", + "▁m ois", + "▁mo is", + "▁F o", + "▁уча ст", + "ff en", + "f fen", + "▁MOD ULE", + "▁ MODULE", + "▁activ ated", + "▁activate d", + "▁intern acional", + "▁M ittel", + "deg ree", + "▁от кры", + "▁& (", + "get Property", + "is z", + "i sz", + "ced ure", + "▁en ters", + "▁ent ers", + "▁enter s", + "▁S ally", + "▁Sal ly", + "▁Tr ain", + "▁Tra in", + "▁lo gged", + "▁log ged", + "▁R av", + "▁Ra v", + "▁A void", + "▁Av oid", + "▁K aiser", + "▁Ka iser", + "▁ex pend", + "▁exp end", + "ap hor", + "aph or", + "▁b rass", + "▁br ass", + "▁bra ss", + "▁bras s", + "▁mel od", + "▁att itudes", + "▁attitude s", + "* \"", + "W all", + "▁o we", + "▁ owe", + "▁b amb", + "▁ba mb", + "sh ader", + "sha der", + "ce ster", + "ces ter", + "c ester", + "▁P P", + "▁ PP", + "▁migr ations", + "▁migration s", + "ent ric", + "entr ic", + "▁Set up", + "▁ Setup", + "▁Art ist", + "hr e", + "h re", + "▁pol ite", + "▁polit e", + "ah an", + "aha n", + "a han", + "▁lug lio", + "▁pre decess", + "▁S IG", + "▁SI G", + "▁ SIG", + "ті в", + "т ів", + "▁R F", + "▁ RF", + "▁D ry", + "▁Dr y", + "▁m aker", + "▁make r", + "▁ma ker", + "▁ maker", + "ши м", + "ш им", + "▁S ounds", + "▁Sound s", + "▁implement ing", + "▁a h", + "▁ ah", + "▁g ev", + "▁ge v", + "▁du plicate", + "▁L ogan", + "▁Log an", + "▁Lo gan", + "▁G rade", + "▁Gr ade", + "▁Grad e", + "▁Gra de", + "DU CT", + "ís es", + "í ses", + "ér t", + "é rt", + "▁nons ense", + "back up", + "Att achment", + "▁e cc", + "▁ec c", + "▁Squad ron", + "le arn", + "lear n", + "de precated", + "dep recated", + "▁A ub", + "▁Au b", + "▁G ol", + "▁Go l", + "▁over l", + "SER VICE", + "▁beautiful ly", + "RE L", + "R EL", + "▁G ian", + "▁Gi an", + "▁P apa", + "▁Pa pa", + "▁Pap a", + "res pond", + "respon d", + "resp ond", + "▁Carib bean", + "r n", + "▁худо ж", + "C fg", + "ra i", + "r ai", + "▁sn iff", + "tt o", + "t to", + "оло ги", + "о логи", + "▁r b", + "▁ rb", + "▁inc idents", + "▁incident s", + "▁d uck", + "▁du ck", + "▁PROVID ED", + "Source s", + "S ources", + "▁Chel sea", + "▁t ek", + "▁te k", + "▁ tek", + "▁на лази", + "▁pil ots", + "▁pilot s", + "т ки", + "▁tr aded", + "▁trad ed", + "▁tra ded", + "▁trade d", + "▁Be ijing", + "▁Greg ory", + "scal ar", + "▁incl ined", + "▁inc lined", + "▁K amp", + "▁Kam p", + "▁Ka mp", + "▁M arian", + "▁Mar ian", + "▁Ma rian", + "▁Maria n", + "▁fier ce", + "▁the ft", + "▁th eft", + "ющи х", + "▁In to", + "▁Int o", + "▁ Into", + "con straint", + "parent Node", + "id ental", + "ident al", + "iden tal", + "▁gouver nement", + "▁S ND", + "▁SN D", + "▁Rub y", + "▁Ru by", + "▁mon aster", + "Rec ords", + "Record s", + "▁K ab", + "▁Ka b", + "▁Un iverse", + "▁Univers e", + "▁approxim ate", + "▁approx imate", + "W ater", + "▁Phys ical", + "ap pers", + "app ers", + "appe rs", + "oubt edly", + "ло жен", + "ложе н", + "▁tow el", + "▁sib lings", + "ep h", + "e ph", + "ic ios", + "ici os", + "icio s", + "ра ми", + "▁out rage", + "▁tamb é", + "SR C", + "S RC", + "те лем", + "тел ем", + "V i", + ".' );", + ". ');", + "L M", + "▁m itt", + "▁mit t", + "▁mi tt", + "▁ mitt", + "▁w eed", + "▁we ed", + "▁cr ops", + "▁cro ps", + "▁crop s", + "im an", + "ima n", + "i man", + "Cl aim", + "ins ula", + "▁( “", + "▁Ch anges", + "▁Change s", + "▁ Changes", + "▁invån are", + "ag ain", + "aga in", + "a gain", + "▁c nt", + "▁ cnt", + "▁G az", + "▁Ga z", + "▁a ustral", + "over lay", + "▁Me chan", + "▁sl ammed", + "▁tr ailing", + "▁tra iling", + "▁trail ing", + "▁Bi ography", + "▁appe aling", + "▁appeal ing", + "IV ER", + "IVE R", + "I VER", + "▁A ve", + "▁Av e", + "▁P lot", + "▁Pl ot", + "vo j", + "v oj", + "▁s ung", + "▁su ng", + "▁sun g", + "▁ sung", + "▁u nos", + "▁un os", + "▁uno s", + "Effect s", + "v v", + "co ok", + "c ook", + "But tons", + "Button s", + "▁trans m", + "ier to", + "iert o", + "CON TEXT", + "CONT EXT", + "▁dign ity", + "air ed", + "ai red", + "aire d", + "a ired", + "java x", + "jav ax", + "j avax", + "▁Albert o", + "▁Alber to", + "▁Rec ently", + "▁Recent ly", + "▁fac ial", + "▁fa cial", + "math op", + "mat hop", + "ał o", + "a ło", + "ви д", + "co tt", + "c ott", + "Vari ables", + "Variable s", + "▁R an", + "▁Ra n", + "▁b unk", + "am iliar", + "amil iar", + "CA ST", + "C AST", + "▁fr ü", + "VE D", + "V ED", + "▁NOT ICE", + "▁turn o", + "▁tur no", + "valid ator", + "▁Portug uese", + "▁question ing", + "}} )", + "} })", + "▁l ear", + "▁le ar", + "▁ lear", + "X amarin", + "▁dis adv", + "enc oded", + "encode d", + "▁K ot", + "▁Ko t", + "ra ted", + "rat ed", + "rate d", + "r ated", + "▁The ory", + "ci us", + "c ius", + "▁Dar win", + "ђ е", + "▁dé cl", + "▁déc l", + "▁обла сть", + "ро вич", + "▁mob ility", + "▁mobil ity", + "V F", + "▁х и", + "▁ хи", + "un til", + "unt il", + "u ntil", + "▁bar riers", + "▁barrier s", + "▁barr iers", + "gi f", + "g if", + "▁R oh", + "▁Ro h", + "▁a ging", + "▁ag ing", + "▁ aging", + "▁W idget", + "▁ Widget", + "ol k", + "▁f arms", + "▁far ms", + "▁farm s", + "Check er", + "Che cker", + "Int roduction", + "с мо", + "▁Russ ians", + "▁Russian s", + "▁Russia ns", + "na ments", + "nam ents", + "nament s", + "n aments", + "▁In sert", + "▁Ins ert", + "▁ Insert", + "▁When ever", + "▁Whe never", + "er set", + "ers et", + "it ori", + "itor i", + "ito ri", + "▁D ort", + "▁Do rt", + "▁Dor t", + "▁cost ume", + "▁mathemat ical", + "▁B ast", + "▁Bas t", + "▁Ba st", + "▁nom inated", + "▁nomin ated", + "▁rest oration", + "pos al", + "po sal", + "▁un fortunate", + "P s", + "LI N", + "L IN", + "▁int act", + "▁prov oc", + "▁situ ée", + "▁но ября", + "er mo", + "erm o", + "▁f isher", + "▁fish er", + "▁fis her", + "г ля", + "▁con ting", + "▁cont ing", + "▁contin g", + "▁Do ug", + "▁Dou g", + "\" ?", + "▁E va", + "▁Ev a", + "▁t ops", + "▁to ps", + "▁top s", + "▁Rem ote", + "▁ Remote", + "▁art work", + "▁art illery", + "qu ick", + "▁Arab ia", + "▁SD Value", + "▁Dak ota", + "ia ted", + "iat ed", + "iate d", + "i ated", + "▁Op tim", + "▁Opt im", + "but tons", + "button s", + "▁c ottage", + "▁where in", + "▁tut orial", + "▁S cre", + "▁Sc re", + "▁swe ep", + "▁Coff ee", + "}) }", + "} )}", + "▁му зы", + "host name", + "▁T emp", + "▁Te mp", + "▁Tem p", + "▁ Temp", + "▁F ut", + "▁Fu t", + "re spect", + "res pect", + "resp ect", + "oc z", + "o cz", + "▁pre domin", + "▁pred omin", + "Ind icator", + "en cial", + "enc ial", + "encia l", + "enci al", + "UM ENT", + "U MENT", + "▁SH ALL", + "▁SHA LL", + "▁comm anded", + "▁command ed", + "▁withdraw al", + "io ur", + "i our", + "REG ION", + "s printf", + "▁в ме", + "▁Pay ment", + "▁ Payment", + "▁A nim", + "▁An im", + "▁ Anim", + "pub lish", + "▁se eks", + "▁see ks", + "▁seek s", + "ou w", + "o uw", + "▁G M", + "▁ GM", + "ru gu", + "rug u", + "r ugu", + "us tain", + "ust ain", + "usta in", + "▁) )", + "▁ ))", + "▁consult ing", + "▁D ialog", + "▁ Dialog", + "▁L ars", + "▁La rs", + "▁Lar s", + "▁crit ique", + "▁circ ulation", + "▁circul ation", + "▁land sc", + "▁lands c", + "man aged", + "▁C raft", + "▁Cr aft", + "▁Cra ft", + "▁h erman", + "▁her man", + "af i", + "a fi", + "am y", + "a my", + "▁disc our", + "▁disco ur", + "<> (", + "< >(", + "▁St eph", + "▁Ste ph", + "▁Step h", + "▁toler ance", + "type name", + "typ ename", + "typen ame", + "vent ions", + "vention s", + "zi ał", + "z iał", + "ст ов", + "сто в", + "с тов", + "▁st icking", + "▁stick ing", + "AS C", + "A SC", + "IS O", + "I SO", + "▁Sp encer", + "▁Di dn", + "▁Did n", + "gom ery", + "im iter", + "imit er", + "imi ter", + "dr u", + "d ru", + "Cl ause", + "▁sl ides", + "▁slide s", + "▁slid es", + "## #", + "# ##", + "▁S ugar", + "▁Su gar", + "H Y", + "▁э ти", + "▁Ed wards", + "▁Edward s", + "▁c ents", + "▁cent s", + "oy a", + "o ya", + "ser ts", + "sert s", + "s erts", + "▁H ass", + "▁Ha ss", + "▁Has s", + "▁in gen", + "▁ing en", + "▁ ingen", + "ст ри", + "с три", + "▁s addle", + "sol id", + "s olid", + "▁ch ampions", + "▁champion s", + "▁champ ions", + "- )", + "▁S lov", + "▁Sl ov", + "▁sh iny", + "▁* )&", + "▁*) &", + "▁Def ine", + "č e", + "▁scr ut", + "on den", + "ond en", + "onde n", + "'\" ,", + "' \",", + "uf fs", + "uff s", + "▁o lymp", + "id ential", + "ident ial", + "wa nd", + "wan d", + "w and", + "▁ann ually", + "▁annual ly", + "▁Ark ansas", + "▁s aint", + "▁sa int", + "▁gle ich", + "▁per fection", + "▁perfect ion", + "▁perf ection", + ") >", + "▁sh orts", + "▁short s", + "▁just ified", + "pe ated", + "peat ed", + "pack ages", + "package s", + "dr iven", + "drive n", + "d riven", + "▁Liber ty", + "▁str ipped", + "▁stri pped", + "▁strip ped", + "ше ние", + "▁fün f", + "▁e cosystem", + "ix a", + "i xa", + "▁F resh", + "▁Fr esh", + "▁Fre sh", + "var t", + "va rt", + "v art", + "▁tre ats", + "▁treat s", + "▁st ance", + "▁stan ce", + "▁ stance", + "чё т", + "ч ёт", + "▁p ity", + "▁pi ty", + "▁pit y", + "ad ém", + "▁о кон", + "▁ок он", + "▁C hand", + "▁Ch and", + "▁Cha nd", + "ra b", + "r ab", + "вши й", + "в ший", + "in ski", + "ins ki", + "▁contin ually", + "▁continu ally", + "▁D addy", + "▁Dad dy", + "▁night mare", + "ic ional", + "ici onal", + "icio nal", + "icion al", + "▁e fect", + "ue blo", + "▁l anç", + "▁lan ç", + "▁Col lections", + "▁Collection s", + "▁Collect ions", + "▁ Collections", + "du e", + "d ue", + "am pton", + "amp ton", + "▁mem cpy", + "▁ memcpy", + "▁* *(", + "▁** (", + "is sent", + "iss ent", + "isse nt", + "issen t", + "▁In sp", + "▁Ins p", + "▁Glas gow", + "▁fur ono", + "▁kind ness", + "B i", + "▁comp eted", + "▁compet ed", + "▁compete d", + "▁o ak", + "L arge", + "▁dis gu", + "▁disg u", + "▁k ings", + "▁king s", + "▁kin gs", + "та ми", + "▁st uffed", + "▁stuff ed", + "▁h ilar", + "▁hi lar", + "pub lished", + "publish ed", + "▁st ressed", + "▁str essed", + "▁stress ed", + "▁Pe ak", + "▁lo ader", + "▁load er", + "▁ loader", + "Key board", + "▁re construction", + "▁v od", + "▁vo d", + "▁ vod", + "▁d un", + "▁du n", + "▁understand s", + "te nant", + "ten ant", + "▁ch aque", + "▁cha que", + "▁pre jud", + "ut at", + "uta t", + "u tat", + "▁u so", + "▁us o", + "▁ uso", + "▁He avy", + "▁cu atro", + "▁side walk", + "▁B ug", + "▁Bu g", + "▁mån aden", + "ge o", + "▁un ited", + "▁unit ed", + "▁F iles", + "▁Fil es", + "▁File s", + "▁Fi les", + "▁ Files", + "▁А ль", + "▁Ал ь", + "▁rug by", + "▁fin ancing", + "▁financ ing", + "▁com ply", + "▁comp ly", + "▁compl y", + "& #", + "▁r ushing", + "▁rush ing", + "▁rus hing", + "▁f en", + "▁fe n", + "▁ fen", + "mon g", + "mo ng", + "m ong", + "▁sp é", + "▁present ing", + "IN CLUDING", + "ě l", + "zeich nung", + "Back up", + "▁pe tit", + "▁pet it", + "▁all erg", + "▁alle rg", + "▁aller g", + "ну т", + "н ут", + "▁wor rying", + "▁worry ing", + "▁m amm", + "▁ma mm", + "▁oper and", + "▁opera nd", + ":%.* ]]", + "▁real ise", + "Comm ands", + "Command s", + "▁B ew", + "▁Be w", + "▁ass umes", + "▁assum es", + "▁assume s", + "▁Co vid", + "▁Cov id", + "▁qu and", + "ty ard", + "t yard", + "▁M ono", + "▁Mon o", + "▁Mo no", + "lin ked", + "link ed", + "M ARK", + "Es p", + "E sp", + "▁bless ing", + "▁eyeb rows", + "▁N V", + "▁ NV", + "▁ст ру", + "▁ стру", + "▁mod eling", + "▁model ing", + "▁mode ling", + "▁gre eted", + "Work space", + "▁pe dest", + "▁ped est", + "▁не за", + "lem agne", + "Stat istics", + "▁a ument", + "▁au ment", + "▁spe eds", + "▁speed s", + "▁synd rome", + "CONNE CT", + "za hl", + "z ahl", + "ver so", + "vers o", + "érc ito", + "▁astr onom", + "▁ap rile", + "▁apr ile", + "▁april e", + "že n", + "ž en", + "ве ро", + "вер о", + "dr aft", + "d raft", + "▁g ioc", + "▁gi oc", + "▁com port", + "▁comp ort", + "▁var iance", + "▁vari ance", + "▁real izing", + "▁realiz ing", + "ED IT", + "оло ві", + "▁e star", + "▁est ar", + "▁es tar", + "▁esta r", + "▁s ost", + "▁so st", + "N ORMAL", + "▁ ó", + "▁And r", + "▁An dr", + "ATTR IB", + "▁re de", + "▁r ede", + "▁red e", + "▁t oes", + "▁to es", + "▁toe s", + "▁adv ances", + "▁advance s", + "▁Again st", + "TO M", + "T OM", + "rs s", + "r ss", + "MM MM", + "▁ne west", + "▁new est", + "▁V ER", + "▁ VER", + "▁phrase s", + "▁phr ases", + "an ter", + "ant er", + "ante r", + "La unch", + "▁c hr", + "▁ch r", + "▁ chr", + "▁manufact ured", + "$) ,", + "$ ),", + "roll ment", + "es ton", + "est on", + "esto n", + "e ston", + "▁pe int", + "” )", + "en det", + "end et", + "ende t", + "▁H air", + "▁Ha ir", + "ival ent", + "▁up right", + "gr en", + "gre n", + "g ren", + "an ked", + "ank ed", + "wr ight", + "w right", + "▁m ast", + "▁ma st", + "▁mas t", + "▁on Change", + "▁de bris", + "▁deb ris", + "▁g rap", + "▁gr ap", + "▁gra p", + "et ry", + "etr y", + "e try", + "▁( __", + "▁(_ _", + "▁ (__", + "▁Com merce", + "BO X", + "T ax", + "▁о три", + "▁от ри", + "▁pre vention", + "▁prevent ion", + "▁prev ention", + "▁Fe el", + "▁ex otic", + "▁B ark", + "▁Bar k", + "▁S team", + "▁Ste am", + "fo n", + "f on", + "ol in", + "oli n", + "o lin", + "▁elim inated", + "▁eliminate d", + "▁b c", + "▁ bc", + "▁C ycl", + "▁Cy cl", + "▁$ (\"#", + "▁ $(\"#", + "▁P arl", + "▁Par l", + "▁Pa rl", + "man uel", + "os pher", + "osp her", + "osph er", + "W F", + "An aly", + "Anal y", + "▁nav ig", + "▁re nown", + "▁ren own", + "R x", + "▁W alt", + "▁Wal t", + "▁Wa lt", + "uf fed", + "uff ed", + "▁f oster", + "▁fo ster", + "▁fost er", + "▁fos ter", + "$ :", + "sh ore", + "Conne ctor", + "Conn ector", + "Connect or", + "фи ка", + "▁real ization", + "▁realiz ation", + "L i", + "ct xt", + "ctx t", + "c txt", + "ah oo", + "aho o", + "▁mir acle", + "▁E T", + "▁ ET", + "▁G PS", + "▁GP S", + "▁Observ able", + "▁h f", + "▁ hf", + "▁magnific ent", + "не го", + "BI N", + "B IN", + "▁D orf", + "▁Do rf", + "▁Dor f", + "ie ck", + "ve e", + "v ee", + "▁C raw", + "▁Cr aw", + "▁Cra w", + "/ #", + "▁p ci", + "▁pc i", + "▁ pci", + "ip pet", + "ipp et", + "▁Hill ary", + "▁g ir", + "▁gi r", + "▁r and", + "▁ran d", + "▁ra nd", + "▁ rand", + "▁la ying", + "▁lay ing", + "▁D ifferent", + "bo ys", + "boy s", + "vi rt", + "vir t", + "v irt", + "▁enc ryption", + "ás z", + "á sz", + "по р", + "п ор", + "▁sm elled", + "▁smell ed", + "▁sus cept", + "cl uded", + "clude d", + "▁C arn", + "▁Car n", + "▁Ca rn", + "ig ten", + "igt en", + "igte n", + "▁Ch uck", + "▁Prov inc", + "▁per í", + "▁Mar shal", + "▁Mars hal", + "▁ Marshal", + "мо ж", + "g fx", + "os hi", + "osh i", + "▁W HE", + "▁WH E", + "▁relax ation", + ", .", + "we re", + "wer e", + "w ere", + "▁var ieties", + "▁W on", + "▁Wo n", + "▁g aps", + "▁gap s", + "▁ga ps", + "▁st ole", + "▁sto le", + "ig ua", + "igu a", + "ющи е", + "▁Ham pshire", + "ph rase", + "▁pel ícula", + "Process ing", + "▁initial ization", + "oust ic", + "▁Jose f", + "▁Jos ef", + "ic ating", + "ica ting", + "▁good ness", + "TE S", + "T ES", + "▁c ope", + "▁co pe", + "▁cop e", + "▁ cope", + "▁ignor ance", + "▁B rist", + "▁Br ist", + "▁par as", + "▁para s", + "▁pa ras", + "▁accident ally", + "▁t and", + "▁tan d", + "▁ta nd", + "it test", + "itt est", + "itte st", + "▁у ли", + "▁sh ipped", + "▁ship ped", + "▁о ст", + "▁ос т", + "else if", + "▁u size", + "▁us ize", + "hor izontal", + "▁C arr", + "▁Car r", + "▁Ca rr", + "▁pre cip", + "▁prec ip", + "ro z", + "r oz", + "path etic", + "pat hetic", + "ri ved", + "riv ed", + "rive d", + "r ived", + "ro k", + "r ok", + "▁dig ging", + "мо м", + "▁M ull", + "▁Mu ll", + "▁Mul l", + "▁X III", + "▁XII I", + "▁XI II", + "▁pe as", + "▁f oul", + "▁fo ul", + "▁fou l", + "▁travel s", + "▁trav els", + "▁N g", + "▁состав е", + "▁соста ве", + "Mon t", + "Mo nt", + "M ont", + "ar de", + "ard e", + "▁Ste fan", + "^^ ^^", + "▁K iss", + "▁Ki ss", + "▁E k", + "▁ok tober", + "▁mem orable", + "▁memor able", + "') ).", + "')) .", + "' )).", + "▁V ision", + "▁Vis ion", + "▁N ina", + "▁Ni na", + "▁Nin a", + "▁S olar", + "▁So lar", + "▁Sol ar", + "▁highlight ed", + "▁me mo", + "▁mem o", + "me isterschaft", + "side bar", + "SE E", + "S EE", + "▁Nev ada", + "D a", + "▁draw er", + "ast ically", + "astic ally", + "el de", + "eld e", + "sc ribed", + "scri bed", + "scribe d", + "scrib ed", + "▁pri ests", + "▁priest s", + "▁hom mes", + "▁homme s", + "▁in structor", + "▁instruct or", + "кла д", + "▁sp ett", + "▁spe tt", + "\\ -", + "▁ми ра", + "▁ мира", + "▁Look s", + "▁Lo oks", + "▁sle eve", + "▁strong est", + "▁t ête", + "▁Nic ole", + "▁Ni cole", + "▁Nicol e", + "im per", + "imp er", + "на ча", + "ip per", + "ipp er", + "▁in won", + "il ers", + "ile rs", + "iler s", + "i lers", + "▁Dep uty", + "og e", + "o ge", + "▁de pressed", + "▁dep ressed", + "▁depress ed", + "▁ar te", + "▁art e", + "▁ arte", + "▁comb ining", + "LA ST", + "L AST", + "in ted", + "int ed", + "inte d", + "▁A verage", + "▁Ave rage", + "▁poll ution", + "▁Phill ips", + "▁W M", + "▁ WM", + "}} }\\", + "}}} \\", + "} }}\\", + "Add ed", + "Ad ded", + "▁per ipher", + "Creat ion", + "C reation", + "▁ital ien", + "▁Ch oice", + "▁Cho ice", + "▁ Choice", + "▁EX PRESS", + "▁St ruct", + "▁Str uct", + "▁ Struct", + "ys z", + "y sz", + "Res ize", + "Re size", + "AR GS", + "ARG S", + "▁re po", + "▁rep o", + "▁ repo", + "▁что бы", + "▁p ref", + "▁pre f", + "▁pr ef", + "▁ pref", + "▁earth qu", + "▁Ме кси", + "▁F inale", + "▁Fin ale", + "▁Final e", + "▁h echo", + "▁he cho", + "requ ests", + "request s", + "C ut", + "▁des erved", + "▁deserve d", + "го во", + "гов о", + "▁Re cent", + "▁Rec ent", + "▁ди визи", + "▁support ive", + "пра ви", + "прав и", + "▁irre levant", + "' \r", + "▁c trl", + "▁ ctrl", + "▁De al", + "iz ada", + "iza da", + "u o", + "▁n ort", + "▁no rt", + "▁nor t", + "ge ometry", + "geo metry", + "▁Individ ual", + "er eg", + "ere g", + "e reg", + "▁при ня", + "cre f", + "cr ef", + "c ref", + "═ ═", + "▁com erc", + "▁come rc", + "= _", + "bu nd", + "b und", + "та х", + "il en", + "ile n", + "i len", + "чи та", + "▁corpor ation", + "es z", + "e sz", + "▁= =>", + "▁== >", + "ab lish", + "abl ish", + "Ap r", + "A pr", + "▁r ipped", + "▁ri pped", + "▁rip ped", + "Var s", + "V ars", + "st ret", + "str et", + "stre t", + "▁Frances co", + "Na N", + "▁any time", + "▁autom ated", + "ost ream", + "o stream", + "▁draw ings", + "▁drawing s", + "▁enhance ment", + "ok rat", + "▁Iss ue", + "в ра", + "Cur rency", + "▁w yn", + "▁wy n", + "izar re", + "ét ico", + "mult iple", + "multi ple", + "multip le", + "▁R ate", + "▁Ra te", + "▁Rat e", + "▁ Rate", + "▁I ch", + "▁A uss", + "▁Aus s", + "▁Au ss", + "▁For mer", + "▁Form er", + "Cur ve", + "▁mar vel", + "att ro", + "attr o", + "▁с п", + "BO OL", + "си я", + "go ld", + "g old", + "▁N intendo", + "▁Salv ador", + "▁S olution", + "▁Sol ution", + "AD C", + "A DC", + "бо ра", + "бор а", + "▁Ben nett", + "▁F R", + "▁ FR", + "▁pu eden", + "▁pued en", + "▁puede n", + "pat ient", + "▁P G", + "▁ PG", + "▁J in", + "▁Ji n", + "▁cr ashed", + "▁crash ed", + "▁d enen", + "▁de nen", + "▁den en", + "▁S ample", + "▁Sam ple", + "▁ Sample", + "▁Que bec", + "it ories", + "itor ies", + "ito ries", + "itori es", + "▁b linked", + "▁blink ed", + "▁l ion", + "▁li on", + "▁vo ce", + "▁voc e", + "▁Imp act", + "▁M au", + "▁Ma u", + "▁N ie", + "▁Ni e", + "▁l ob", + "▁lo b", + "▁д ве", + "or neys", + "orney s", + "orne ys", + "▁coast al", + "▁s ensors", + "▁sens ors", + "▁sensor s", + "▁X II", + "▁XI I", + "▁ill usion", + "oj i", + "o ji", + "▁I NC", + "▁IN C", + "▁Dun can", + "y k", + "▁affect ing", + "pu l", + "p ul", + "▁Napole on", + "▁а каде", + "▁com pt", + "▁comp t", + "▁prof itable", + "▁profit able", + "lo e", + "l oe", + "▁deux ième", + "▁W C", + "▁ WC", + "▁v iable", + "▁vi able", + "▁via ble", + "▁D rug", + "▁Dr ug", + "▁Dru g", + "Text Box", + "▁lum inos", + "au té", + "aut é", + "y c", + "št ě", + "▁affili ates", + "▁affiliate s", + "il da", + "ild a", + "con duct", + "cond uct", + "▁e benfalls", + "▁A MD", + "▁AM D", + "▁Mon itor", + "▁ Monitor", + "▁Compan ies", + "▁correct ed", + "▁corre cted", + "ä ck", + "SY STEM", + "other apy", + "▁п еред", + "▁пере д", + "▁пе ред", + "▁bl ues", + "▁blue s", + "at isf", + "ati sf", + "atis f", + "al though", + "alth ough", + "ro st", + "ros t", + "r ost", + "SC AN", + "S CAN", + "▁R AM", + "ці ональ", + "▁vend ors", + "▁vendor s", + "▁custom s", + "▁cust oms", + "▁activ ate", + "▁ activate", + "▁b logs", + "▁bl ogs", + "▁blo gs", + "▁blog s", + "▁br ace", + "▁bra ce", + "▁ brace", + "▁st rat", + "▁str at", + "▁stra t", + "an je", + "anj e", + "щ ё", + "▁t ide", + "▁tid e", + "▁ti de", + "▁Brig ade", + "get Operand", + "▁al iment", + "▁ali ment", + "▁achieve ments", + "▁achievement s", + "▁suspic ion", + "▁susp icion", + "▁touch down", + "br oad", + "bro ad", + "b road", + "io re", + "ior e", + "i ore", + "Compar ison", + "▁m um", + "▁mu m", + "Eng lish", + "▁P icture", + "▁Pict ure", + "▁M ouse", + "▁Mo use", + "▁ Mouse", + "am d", + "a md", + "▁[ `", + "▁den omin", + "▁denom in", + "▁Al eks", + "▁Ale ks", + "▁pr events", + "▁prevent s", + "▁prev ents", + "ó b", + "fe d", + "f ed", + "▁P ray", + "▁Pr ay", + "▁Pra y", + "▁sh ine", + "▁ shine", + "▁cl utch", + "mu x", + "m ux", + "App ro", + "Ap pro", + "▁not ably", + "ch io", + "chi o", + "na ge", + "n age", + "HA S", + "H AS", + "▁' )", + "▁ ')", + "▁M iche", + "▁Mich e", + "▁Mic he", + "▁Mi che", + "t g", + ":: ~", + "▁am ely", + "▁ro dz", + "▁rod z", + "z s", + "tr ait", + "tra it", + "t rait", + "▁k lass", + "▁kl ass", + "▁ klass", + "f ö", + "▁dest ac", + "▁Cl ara", + "▁Clar a", + "f requency", + "▁G it", + "▁Gi t", + "▁по ль", + "▁пол ь", + "▁frequ encies", + "▁febr ero", + "▁st umbled", + "ко ю", + "▁N ames", + "▁Name s", + "▁Na mes", + "▁Nam es", + "▁ Names", + "▁F light", + "▁Fl ight", + "▁p rey", + "▁pre y", + "▁pr ey", + "▁med io", + "▁medi o", + "▁V AR", + "▁VA R", + "▁ VAR", + "▁F loat", + "▁Flo at", + "▁ Float", + "▁Ern est", + "▁Marc atori", + "op ort", + "o port", + "▁cancel lation", + "▁cancell ation", + "▁Br yan", + "▁Bry an", + "—— ——", + "Lu c", + "L uc", + "▁li bre", + "▁lib re", + "▁t ítulo", + "* >", + "▁S andy", + "▁San dy", + "▁Sand y", + "▁Mar ina", + "Be en", + "B een", + "▁w al", + "▁wa l", + "▁ wal", + "▁K ultur", + "▁expl ode", + "▁explo de", + "▁lim iting", + "▁limit ing", + "▁presum ably", + "▁p b", + "▁ pb", + "▁M erc", + "▁Me rc", + "▁Mer c", + "▁ре ки", + "le arning", + "lear ning", + "learn ing", + "C atalog", + "▁C ensus", + "lt e", + "l te", + "▁N ET", + "▁NE T", + "▁ NET", + "ra ising", + "rais ing", + "rai sing", + "сь ке", + "st aff", + "sta ff", + "▁Qu inn", + "▁mem orial", + "▁memor ial", + "▁memo rial", + "п ня", + "▁cu enta", + "▁X I", + "lb l", + "l bl", + "▁v aries", + "▁var ies", + "▁vari es", + "▁va ries", + "▁fluct uations", + "▁дол ж", + "▁осо би", + "▁ware house", + "How ever", + "▁correct ions", + "▁corre ctions", + "▁correction s", + "dh d", + "d hd", + "▁f als", + "▁fa ls", + "▁fal s", + "▁controvers y", + "▁cur se", + "▁t élé", + "▁té lé", + "ře d", + "ř ed", + "▁A U", + "▁ AU", + "▁т ор", + "▁то р", + "▁ тор", + "▁cr ít", + "id an", + "ida n", + "i dan", + "ili ary", + "iliar y", + "ilia ry", + "▁P anel", + "▁Pan el", + "▁Pa nel", + "▁ Panel", + "cul e", + "cu le", + "c ule", + "▁P oor", + "▁Po or", + "▁B A", + "▁ BA", + "▁ignor ant", + "ème s", + "è mes", + "▁aest hetic", + "Link ed", + "Lin ked", + "get Int", + "Un icode", + "[ @", + "▁Z ent", + "▁Ze nt", + "▁Zen t", + "Man ifest", + "▁v ars", + "▁var s", + "▁va rs", + "▁ vars", + "P B", + "▁в у", + "▁ ву", + "▁De scribe", + "▁Desc ribe", + "▁ Describe", + "▁Any thing", + "oi rs", + "oir s", + "o irs", + "▁s ocks", + "▁so cks", + "▁soc ks", + "▁sock s", + "▁im ped", + "▁imp ed", + "▁ne ue", + "▁neu e", + "▁dis pers", + "▁disp ers", + "Col lect", + "Coll ect", + "file r", + "fil er", + "fi ler", + "f iler", + "▁Fr au", + "▁Fra u", + "▁H ockey", + "▁te ens", + "▁teen s", + "▁Rober to", + "▁Robert o", + "la uf", + "l auf", + "ва ть", + "ват ь", + "▁с ко", + "▁ ско", + "is Array", + "▁teen ager", + "Bu ilt", + "▁loud ly", + "Cap acity", + "▁advent ures", + "▁adventure s", + "▁M olly", + "▁Mol ly", + "rec ogn", + "bar s", + "ba rs", + "b ars", + "▁L or", + "▁Lo r", + "▁pu ò", + "▁m ong", + "▁mon g", + "▁mo ng", + "▁ mong", + "in ement", + "ine ment", + "i nement", + "Ass ignment", + "Assign ment", + "▁d iz", + "▁di z", + "less ness", + "▁H alloween", + "▁bit map", + "▁ bitmap", + "Ro m", + "R om", + "на р", + "н ар", + "▁re bel", + "▁reb el", + "▁rad ial", + "▁radi al", + "me asure", + "ni t", + "n it", + "▁Ass ume", + "▁assign ments", + "▁assignment s", + "▁I sn", + "▁Is n", + "▁al tre", + "▁alt re", + "ße r", + "ß er", + "на ль", + "нал ь", + "н аль", + "▁fl ies", + "▁d roit", + "▁dro it", + "▁thick ness", + "▁en jo", + "▁d well", + "▁dw ell", + "▁hom osexual", + "▁e val", + "▁ev al", + "▁ eval", + "$_ {", + "$ _{", + "as ia", + "asi a", + "▁phil os", + "get Current", + "▁veter ans", + "▁veteran s", + "▁Ber keley", + "▁wild life", + "Co p", + "C op", + "ve rn", + "ver n", + "v ern", + "▁ Ú", + "to s", + "t os", + "▁L ed", + "▁Le d", + "▁key words", + "▁keyword s", + "▁med ications", + "▁medic ations", + "▁medication s", + "ne um", + "▁jam ais", + "▁B uc", + "▁Bu c", + "▁P D", + "▁ PD", + "▁State ment", + "▁Stat ement", + "▁ Statement", + "▁P I", + "▁ PI", + "▁Jack ie", + "▁Jac kie", + "▁ord in", + "▁k ör", + "▁kö r", + "en ze", + "enz e", + "▁util ized", + "▁utiliz ed", + "▁utilize d", + "á ct", + "az ed", + "aze d", + "a zed", + "▁sever ely", + "▁severe ly", + "▁ä ven", + "▁li bro", + "▁lib ro", + "▁E u", + "äs t", + "ä st", + "PAR T", + "PA RT", + "P ART", + "▁But ler", + "▁puzz le", + "F all", + "Count ry", + "C ountry", + "pf n", + "p fn", + "▁у країн", + "▁Or chestra", + "▁al to", + "▁alt o", + "▁anc ora", + "▁decom position", + "▁ م", + "▁appet ite", + "ad u", + "a du", + "▁TH AT", + "▁com enz", + "min a", + "mi na", + "m ina", + "▁init iated", + "▁initi ated", + "▁T at", + "▁Ta t", + "▁some time", + "▁som etime", + "▁somet ime", + "re k", + "r ek", + "br ead", + "bre ad", + "b read", + "▁Stat istics", + "▁ Statistics", + "▁C ob", + "▁Co b", + "F ollow", + "▁ge ometric", + "ш ла", + "▁proceed ings", + "D lg", + "se ven", + "s even", + "▁[ -", + "▁ [-", + "▁Buff alo", + "▁bl acks", + "▁black s", + "▁s ov", + "▁so v", + "▁cust ody", + "▁r as", + "▁ra s", + "▁ ras", + "▁tatto o", + "öffent licht", + "Bl o", + "B lo", + "A ustral", + "▁rec uper", + "ле в", + "л ев", + "▁b em", + "▁be m", + "▁t hou", + "▁th ou", + "ori ented", + "orient ed", + "vi r", + "v ir", + "▁col ony", + "▁colon y", + "▁Stan ford", + "Abs olute", + "ad rat", + "adr at", + "▁S itu", + "▁Si tu", + "▁sou vent", + "EX EC", + "▁m ű", + "▁apart ments", + "▁apartment s", + "▁слу ча", + "▁a no", + "▁an o", + "▁ ano", + "WIN DO", + "ac ci", + "acc i", + "▁L au", + "▁La u", + "co urt", + "cou rt", + "c ourt", + "▁manif old", + "▁coal ition", + "▁X IV", + "▁XI V", + "Att rib", + "Attr ib", + "asc ade", + "▁whe at", + "▁strength s", + "FR EE", + "F REE", + "EMP TY", + "▁h ey", + "▁he y", + "as cular", + "asc ular", + "▁pl asma", + "▁b ob", + "▁bo b", + "Sep arator", + "=\" ${", + "=\"$ {", + "▁Z ag", + "▁Za g", + "▁pro jet", + "▁smooth ly", + "SE QU", + "an aly", + "ana ly", + "anal y", + "att achment", + "attach ment", + "▁E S", + "▁ ES", + "▁po pped", + "▁pop ped", + "ő s", + "to m", + "t om", + "▁s ón", + "▁só n", + "▁r ott", + "▁ro tt", + "▁rot t", + "▁ rott", + "Util ities", + "Ut ilities", + "had oop", + "hado op", + "▁s otto", + "▁so tto", + "au tor", + "aut or", + "auto r", + "▁George s", + "▁Georg es", + "▁kter ý", + "▁gru ppo", + "▁ко гда", + "▁ме да", + "▁instrument al", + "▁W riter", + "▁Write r", + "▁Writ er", + "▁Wr iter", + "▁ Writer", + "▁set Timeout", + "ik k", + "i kk", + "▁Do po", + "▁Dop o", + "]) ;\r", + "]); \r", + "] );\r", + "▁pract icing", + "▁Ron ald", + "▁у би", + "▁ag rees", + "▁agree s", + "▁agre es", + "▁den oted", + "▁denote d", + "is miss", + "ism iss", + "▁interview ed", + "template s", + "t emplates", + "ř i", + "ad ministr", + "admin istr", + "▁B utter", + "▁But ter", + "▁XV II", + "▁XVI I", + "▁position ed", + "▁posit ioned", + "▁Four th", + "▁overwhel med", + "▁Reg ular", + "▁rep rezent", + "коно ми", + "▁expect s", + "Ind ices", + "▁mar ijuana", + "▁z aj", + "▁za j", + "▁B ren", + "▁Br en", + "▁Bre n", + "▁be gg", + "▁beg g", + "▁na hm", + "▁nah m", + "▁inter rog", + "ти е", + "▁B un", + "▁Bu n", + "▁с еред", + "▁се ред", + "▁shel ves", + "▁которы х", + "▁Fra uen", + "▁Frau en", + "▁Serge ant", + "▁у спе", + "mat ched", + "match ed", + "m atched", + "▁d onne", + "▁don ne", + "▁touch es", + "▁tou ches", + "ab ort", + "abor t", + "▁v ale", + "▁val e", + "▁va le", + "▁inst itutional", + "▁institut ional", + "▁institution al", + "▁M ons", + "▁Mon s", + "▁Mo ns", + "▁ambit ious", + "▁non etheless", + "▁none theless", + "j d", + "пе й", + "п ей", + "▁back pack", + "da o", + "d ao", + "ви я", + "▁surround ings", + "▁surrounding s", + "| _{", + "▁g egründ", + "dis p", + "di sp", + "d isp", + "▁moist ure", + "▁w yd", + "▁wy d", + "▁tr aders", + "▁trad ers", + "▁tra ders", + "▁trade rs", + "▁Er st", + "▁Gal axy", + "▁в оло", + "▁во ло", + "▁Per u", + "▁Pe ru", + "▁prior ities", + "▁pron ounced", + "▁C BS", + "▁CB S", + "▁Pal m", + "▁Pa lm", + "▁exp ans", + "▁ener get", + "▁energ et", + "▁Cond ition", + "▁ Condition", + "▁S ver", + "▁Sv er", + "ne sted", + "nes ted", + "n ested", + "▁февра ля", + "he ro", + "her o", + "h ero", + "▁ко ло", + "▁к оло", + "▁ коло", + "▁Fil ms", + "▁Film s", + "Bo n", + "B on", + "é al", + "ploy ed", + "tr ained", + "tra ined", + "train ed", + "▁els ő", + "▁l ust", + "▁lu st", + "ati num", + "atin um", + "oy le", + "o yle", + "▁J et", + "▁Je t", + "жде ния", + "▁survey s", + "▁surve ys", + "be e", + "b ee", + "work ers", + "worker s", + "wor kers", + "rec ords", + "record s", + "cal endar", + "bb ing", + "b bing", + "reg ation", + "dash board", + "d ashboard", + "K ing", + "▁v ista", + "▁vis ta", + "▁vi sta", + "▁dep icted", + "▁occur ring", + "▁о фи", + "▁sand wich", + "rc u", + "r cu", + "ke rn", + "ker n", + "k ern", + "▁min ut", + "▁mi nut", + "▁с мер", + "▁t d", + "▁ td", + "so lete", + "sole te", + "sol ete", + "Com plex", + "Comp lex", + "▁t unn", + "▁tu nn", + "▁tun n", + "▁sc arc", + "▁scar c", + "st ead", + "ste ad", + "▁F ail", + "▁Fa il", + "▁ Fail", + "▁R s", + "▁tr ails", + "▁tra ils", + "▁trail s", + "ke m", + "k em", + "▁Rom ans", + "▁Ro mans", + "▁Roman s", + "▁Roma ns", + "at ivity", + "ativ ity", + "Pre vious", + "Prev ious", + "▁de press", + "▁dep ress", + "▁re signed", + "▁res igned", + "▁resign ed", + "get Default", + "▁Tib et", + "▁Ti bet", + "▁Fr anco", + "▁Franc o", + "▁Fran co", + "\") ));", + "\")) );", + "\" )));", + "▁in jection", + "▁inj ection", + "▁inject ion", + "rem oved", + "remove d", + "▁pra ised", + "▁praise d", + "▁A sc", + "▁As c", + "er ase", + "era se", + "eras e", + "e rase", + "▁commission ed", + "MA IL", + "M AIL", + "▁B oh", + "▁Bo h", + "Pol y", + "Po ly", + "P oly", + "▁cin q", + "▁Ab ove", + "▁Josh ua", + "ZE RO", + "Z ERO", + "▁sum mit", + "▁U rs", + "▁Ur s", + "▁c url", + "▁cur l", + "▁cu rl", + "▁v isa", + "▁vis a", + "▁vi sa", + "▁re sur", + "▁res ur", + "={ '", + "= {'", + "fe at", + "▁abs orb", + "▁absor b", + "▁plan ets", + "▁plane ts", + "▁planet s", + "▁prin cess", + "▁prince ss", + "▁Jahrhund erts", + "▁Jahrhundert s", + "x p", + "▁N BC", + "▁ко ми", + "▁ком и", + "▁F UN", + "▁ FUN", + "▁ne uen", + "▁neu en", + "▁neue n", + "▁dé jà", + "▁O z", + "bb en", + "b ben", + "VID EO", + "▁ej empl", + "▁cons iders", + "▁consider s", + "▁consid ers", + "at ri", + "atr i", + "a tri", + "▁ar rog", + "▁arr og", + "io so", + "ios o", + "i oso", + "▁h ace", + "▁ha ce", + "▁contact ed", + "▁un ple", + "▁spons ored", + "▁tr ainer", + "▁tra iner", + "▁train er", + "sb i", + "s bi", + "▁за нима", + "C riterion", + "но то", + "sch eme", + "sche me", + "enn ial", + "per form", + "perf orm", + "▁fix ing", + "▁по стро", + "▁пос тро", + "ar b", + "a rb", + "EX IT", + "▁ca fé", + "▁caf é", + "itut ed", + "itute d", + "itu ted", + "ri ages", + "ria ges", + "riage s", + "T ur", + "▁hab er", + "▁ha ber", + "el asticsearch", + "▁а л", + "▁ ал", + "r h", + "▁v oll", + "▁vo ll", + "▁vol l", + "CL U", + "M il", + "▁mem bres", + "▁membr es", + "▁membre s", + "▁remark ed", + "ва на", + "ван а", + "в ана", + "=\" _", + "Le ss", + "Les s", + "L ess", + "(\" \");", + "▁Y ale", + "▁Ya le", + "ber ries", + "▁rele asing", + "▁im ports", + "▁import s", + "▁imp orts", + "id ea", + "ide a", + "▁( +", + "▁ar qu", + "ific ación", + "ifica ción", + "▁па ра", + "▁пар а", + "▁R angers", + "▁Range rs", + "▁Rang ers", + "▁Ran gers", + "M ic", + "▁n ederbörd", + "▁imag inary", + "▁imagin ary", + "▁special ists", + "▁specialist s", + "▁ho of", + "Mod ules", + "Module s", + "▁sad ly", + "ce il", + "Tab Index", + "at ionale", + "ation ale", + "ational e", + "▁Part ner", + "tb ody", + "t body", + "▁le verage", + "▁lever age", + "D N", + "▁P rec", + "▁Pr ec", + "▁Pre c", + "▁S é", + "▁M am", + "▁Ma m", + "▁a fin", + "▁af in", + "is Valid", + "Ps e", + "P se", + "▁сто ро", + "▁cho pped", + "▁chop ped", + "▁Min or", + "▁Mi nor", + "▁d abei", + "Da vid", + "D avid", + "uss ia", + "▁дере вня", + "▁Id entity", + "▁Ident ity", + "▁ Identity", + "▁L GBT", + "ци је", + "▁Or ts", + "▁Ort s", + "▁part i", + "▁par ti", + "▁B achelor", + "ug a", + "u ga", + "▁O PT", + "▁OP T", + "▁ OPT", + "▁S eth", + "▁Se th", + "▁Set h", + "▁LI ABLE", + "▁inaug ur", + "▁Shan ghai", + "▁relax ing", + "ци она", + "цион а", + "\" %", + "▁o bey", + "▁ob ey", + "▁A irlines", + "▁Air lines", + "Link s", + "Lin ks", + "L inks", + "▁C elt", + "▁Ce lt", + "▁Cel t", + "▁Ad min", + "▁Adm in", + "▁ Admin", + "ag ation", + "▁wor ries", + "IN TE", + "INT E", + "ar ith", + "ari th", + "Fat alf", + "]] )", + "] ])", + "co lm", + "col m", + "▁arch ae", + "▁br ushed", + "▁brush ed", + "▁t ät", + "▁struct ured", + "▁structure d", + "ти и", + "▁home m", + "▁hom em", + "▁ho mem", + "[: ,", + "▁n avy", + "▁na vy", + "▁nav y", + "get Key", + "power ed", + "pow ered", + "▁s ucked", + "▁suc ked", + "▁suck ed", + "▁z omb", + "▁zo mb", + "iss ant", + "issa nt", + "▁M ight", + "▁Mi ght", + "▁Mig ht", + "▁P ull", + "▁Pu ll", + "▁Pul l", + "ri r", + "r ir", + "▁п і", + "▁ пі", + "▁se as", + "▁sea s", + "▁W rest", + "▁Wr est", + "▁t ense", + "▁ten se", + "▁tens e", + "▁a tm", + "▁at m", + "▁have t", + "▁ha vet", + "▁hav et", + "▁pier ws", + "▁trag ic", + "▁D iff", + "▁Di ff", + "▁ Diff", + "▁conf idential", + "▁confident ial", + "success ful", + "ę ż", + "▁Ch ain", + "▁Cha in", + "▁ Chain", + "▁Ken ya", + "Ch oice", + "oc ur", + "o cur", + "an iu", + "ani u", + "▁consult ant", + "▁Ad vis", + "▁Adv is", + "Li f", + "L if", + "▁L ors", + "▁Lo rs", + "▁Lor s", + "avor ite", + "avo rite", + "▁util izing", + "▁utiliz ing", + "▁v intage", + "Mat cher", + "Match er", + "▁m embre", + "▁me mbre", + "▁mem bre", + "▁membr e", + "▁Ex pect", + "▁Exp ect", + "▁ Expect", + "▁tr acing", + "▁tra cing", + "no g", + "n og", + "▁d ej", + "▁de j", + "▁у че", + "▁lo ops", + "▁loop s", + "▁on click", + "▁G PU", + "▁GP U", + "▁ GPU", + "▁Album s", + "▁Alb ums", + "▁Arch ives", + "ва та", + "ват а", + "▁st ove", + "▁sto ve", + "ш ли", + "an cies", + "anc ies", + "▁geme ente", + "mo b", + "m ob", + "PD F", + "P DF", + "es o", + "e so", + "▁v ég", + "▁vé g", + "Res olve", + "▁te aches", + "▁teach es", + "▁tea ches", + "ло же", + "▁с тво", + "▁ст во", + "▁ ство", + "▁О дна", + "▁f id", + "▁fi d", + "Some thing", + "Som ething", + "▁ne bo", + "▁Valent ine", + "row ning", + "rown ing", + "▁а ле", + "▁ал е", + "aw i", + "a wi", + "is hi", + "ish i", + "▁S PI", + "▁SP I", + "▁ SPI", + "▁s pel", + "▁sp el", + "▁spe l", + "▁б іль", + "▁бі ль", + "▁particip ant", + "▁N ed", + "▁Ne d", + "▁G ast", + "▁Ga st", + "▁Gas t", + "▁bl ond", + "▁blo nd", + "▁s aves", + "▁sa ves", + "▁save s", + "▁sav es", + "col ored", + "color ed", + "colo red", + "▁A CTION", + "▁AC TION", + "▁ACT ION", + "▁ ACTION", + "▁Polit iker", + "}$ )", + "} $)", + "▁D um", + "▁Du m", + "den try", + "d entry", + "Stud ent", + "▁~ =", + "lo ads", + "load s", + "▁F oster", + "▁Fo ster", + "一 个", + "▁P K", + "▁ PK", + "▁S B", + "▁ SB", + "▁H ern", + "▁He rn", + "▁Her n", + "▁Ex hib", + "Listener s", + "Listen ers", + "Su n", + "S un", + "pl ac", + "▁B ever", + "▁Be ver", + "▁Bev er", + "▁incl uy", + "▁inclu y", + "▁d c", + "▁ dc", + "ar gc", + "arg c", + "▁g ed", + "▁ge d", + "▁ ged", + "с па", + "▁Form ula", + "▁с ем", + "▁се м", + "▁em pt", + "▁emp t", + "▁ empt", + "un register", + "▁Queens land", + "ánd ez", + "ot ive", + "oti ve", + "▁al ley", + "▁all ey", + "▁alle y", + "▁Democr at", + "▁trav ail", + "▁$ ,", + "▁ $,", + "R P", + "ро е", + "pers onal", + "person al", + "▁péri ode", + "HO ME", + "om es", + "ome s", + "o mes", + "▁recogn ised", + "he ng", + "hen g", + "h eng", + "▁J ung", + "▁Jun g", + "▁Ju ng", + "▁Ro land", + "▁Rol and", + "▁conv icted", + "Loc ked", + "Lock ed", + "L ocked", + "▁m ari", + "▁mar i", + "▁ma ri", + "▁Lux em", + "refer to", + "De leted", + "Dele ted", + "Delete d", + "Del eted", + "int ent", + "inte nt", + "▁St aats", + "▁Sta ats", + "▁обла сті", + "и т", + "▁са ве", + "▁Pro tocol", + "▁ Protocol", + "ają c", + "ch k", + "Type Info", + "▁p kt", + "▁ pkt", + "▁sc andal", + "▁scan dal", + "▁individ ually", + "▁individual ly", + "FM T", + "F MT", + "▁n j", + "ab ile", + "abil e", + "abi le", + "▁R ivers", + "▁River s", + "PRO PERTY", + "V B", + "wo rt", + "wor t", + "w ort", + "▁split ting", + "▁spl itting", + "ach ten", + "acht en", + "achte n", + "a chten", + "▁AR ISING", + "▁s ip", + "▁si p", + "▁f res", + "▁fr es", + "▁fre s", + "▁g room", + "▁gr oom", + "▁gro om", + "H ol", + "▁c anon", + "▁can on", + "▁ca non", + "▁abrupt ly", + "▁after ward", + "▁R unning", + "▁Run ning", + "▁ Running", + "▁j i", + "▁ ji", + "▁% ,", + "▁ %,", + "▁Palest inian", + "R W", + "pgf scope", + "▁country side", + "▁countr yside", + "▁fort unate", + "▁ fortunate", + "▁c él", + "▁Po inter", + "▁Point er", + "▁ Pointer", + "ens ors", + "ensor s", + "enso rs", + "ra ting", + "rat ing", + "r ating", + "▁buff ers", + "▁buffer s", + "▁buf fers", + "▁re mot", + "▁rem ot", + "▁Prop Types", + "▁N ah", + "▁Na h", + "al tern", + "alt ern", + "alter n", + "▁eas iest", + "▁in vas", + "▁inv as", + "▁cl k", + "▁ clk", + "copy right", + "c opyright", + "▁bl anc", + "SA MP", + "S AMP", + "▁Co hen", + "▁S hell", + "▁She ll", + "▁Sh ell", + "▁Shel l", + "▁ Shell", + "▁destroy ing", + "▁destro ying", + "▁Z el", + "▁Ze l", + "date r", + "da ter", + "dat er", + "d ater", + "če n", + "č en", + "▁f iling", + "▁fil ing", + "▁fi ling", + "▁integr ate", + "xi t", + "x it", + "▁R ET", + "▁RE T", + "▁ RET", + "le ne", + "len e", + "l ene", + "cal ls", + "call s", + "c alls", + "▁sl aughter", + "init ialized", + "initial ized", + "initialize d", + "un ches", + "unch es", + "unc hes", + "▁Tr ace", + "▁Tra ce", + "▁ Trace", + "eff icient", + "▁Wood s", + "▁long itud", + "G N", + "▁K ont", + "▁Kon t", + "▁Ko nt", + "▁chunk s", + "á ch", + "▁unem ployment", + "ac om", + "aco m", + "a com", + "▁sl owed", + "▁slow ed", + "▁out lined", + "▁outline d", + "xff ff", + "xf fff", + "x ffff", + "▁ik ke", + "▁work space", + "▁works pace", + "M c", + "▁k icking", + "▁kick ing", + "▁embed ding", + "ch nitt", + "chn itt", + "er ten", + "ert en", + "▁In terior", + "▁Inter ior", + "▁S ongs", + "▁Son gs", + "▁Song s", + "mm c", + "m mc", + "▁analy zed", + "▁analyze d", + "▁Cou pe", + "▁favor ites", + "▁favorite s", + "▁t t", + "▁ tt", + "▁то й", + "▁ той", + "R outing", + "▁Sil va", + "▁andere m", + "▁ander em", + "▁h onom", + "▁hon om", + "▁ho nom", + "▁исполь зова", + ".\" ]", + ". \"]", + "▁W u", + "le gt", + "leg t", + "▁s poon", + "▁sp oon", + "▁spo on", + "▁j ap", + "▁ja p", + "▁Ext ension", + "▁ Extension", + "er ne", + "ern e", + "▁v agy", + "▁va gy", + "▁vag y", + "▁се ла", + "▁ф унк", + "▁anal ytics", + "▁analyt ics", + "▁s ug", + "▁su g", + "▁A sync", + "▁As ync", + "▁ Async", + "▁pe aks", + "▁peak s", + "▁G ym", + "▁Gy m", + "▁law suit", + "▁laws uit", + "< >", + "ial is", + "i alis", + "et ric", + "etr ic", + "face d", + "fa ced", + "fac ed", + "f aced", + "▁dis rupt", + "▁f å", + "Input s", + "`) ;", + "` );", + "▁M end", + "▁Me nd", + "▁Men d", + "go n", + "g on", + "▁\" ,\"", + "▁\", \"", + "▁ \",\"", + "▁n erves", + "▁nerv es", + "▁nerve s", + "▁ner ves", + "▁doubt s", + "▁doub ts", + "sa p", + "s ap", + "▁s ow", + "▁so w", + ",\\ ,\\", + ",\\, \\", + ", \\,\\", + "▁B S", + "▁ BS", + "▁G lad", + "▁Gl ad", + "▁a ster", + "▁as ter", + "▁ast er", + "▁ aster", + "œuv re", + "▁Bang l", + "▁Ban gl", + "▁i Pad", + "use ppe", + "▁conduct ing", + "▁( {\\", + "▁({ \\", + "▁ ({\\", + "▁Har bor", + "ps z", + "p sz", + "▁FI FA", + "_* *", + "_ **", + "em or", + "e mor", + "▁ ▁", + "▁▁ ▁▁", + "▁▁▁ ▁", + "▁ ▁▁▁", + "▁▁ ▁▁▁▁▁▁", + "▁▁▁▁ ▁▁▁▁", + "▁▁▁▁▁ ▁▁▁", + "▁▁▁▁▁▁ ▁▁", + "▁▁▁ ▁▁▁▁▁", + "▁▁▁▁▁▁▁ ▁", + "▁ ▁▁▁▁▁▁▁", + "▁▁ ▁▁▁", + "▁▁▁▁ ▁", + "▁▁▁ ▁▁", + "▁ ▁▁▁▁", + "▁▁ ▁▁▁▁▁▁▁▁▁▁▁▁▁▁", + "▁▁▁▁ ▁▁▁▁▁▁▁▁▁▁▁▁", + "▁▁▁▁▁▁▁▁ ▁▁▁▁▁▁▁▁", + "▁▁▁▁▁ ▁▁▁▁▁▁▁▁▁▁▁", + "▁▁▁▁▁▁ ▁▁▁▁▁▁▁▁▁▁", + "▁▁▁▁▁▁▁▁▁▁▁▁ ▁▁▁▁", + "▁▁▁▁▁▁▁▁▁▁▁▁▁ ▁▁▁", + "▁▁▁▁▁▁▁▁▁▁ ▁▁▁▁▁▁", + "▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▁▁", + "▁▁▁ ▁▁▁▁▁▁▁▁▁▁▁▁▁", + "▁▁▁▁▁▁▁▁▁ ▁▁▁▁▁▁▁", + "▁▁▁▁▁▁▁ ▁▁▁▁▁▁▁▁▁", + "▁▁▁▁▁▁▁▁▁▁▁ ▁▁▁▁▁", + "▁▁ ▁▁▁▁", + "▁▁▁▁ ▁▁", + "▁▁▁▁▁ ▁", + "▁▁▁ ▁▁▁", + "▁ ▁▁▁▁▁", + "▁▁ ▁▁▁▁▁▁▁▁▁▁", + "▁▁▁▁ ▁▁▁▁▁▁▁▁", + "▁▁▁▁▁▁▁▁ ▁▁▁▁", + "▁▁▁▁▁ ▁▁▁▁▁▁▁", + "▁▁▁▁▁▁ ▁▁▁▁▁▁", + "▁▁▁▁▁▁▁▁▁▁ ▁▁", + "▁▁▁ ▁▁▁▁▁▁▁▁▁", + "▁▁▁▁▁▁▁▁▁ ▁▁▁", + "▁▁▁▁▁▁▁ ▁▁▁▁▁", + "▁▁▁▁▁▁▁▁▁▁▁ ▁", + "▁ ▁▁▁▁▁▁▁▁▁▁▁", + "▁▁ ▁▁▁▁▁▁▁▁▁▁▁", + "▁▁▁▁ ▁▁▁▁▁▁▁▁▁", + "▁▁▁▁▁▁▁▁ ▁▁▁▁▁", + "▁▁▁▁▁ ▁▁▁▁▁▁▁▁", + "▁▁▁▁▁▁ ▁▁▁▁▁▁▁", + "▁▁▁▁▁▁▁▁▁▁▁▁ ▁", + "▁▁▁▁▁▁▁▁▁▁ ▁▁▁", + "▁▁▁ ▁▁▁▁▁▁▁▁▁▁", + "▁▁▁▁▁▁▁▁▁ ▁▁▁▁", + "▁▁▁▁▁▁▁ ▁▁▁▁▁▁", + "▁▁▁▁▁▁▁▁▁▁▁ ▁▁", + "▁ ▁▁▁▁▁▁▁▁▁▁▁▁", + "▁▁ ▁▁▁▁▁▁▁▁", + "▁▁▁▁ ▁▁▁▁▁▁", + "▁▁▁▁▁▁▁▁ ▁▁", + "▁▁▁▁▁ ▁▁▁▁▁", + "▁▁▁▁▁▁ ▁▁▁▁", + "▁▁▁ ▁▁▁▁▁▁▁", + "▁▁▁▁▁▁▁▁▁ ▁", + "▁▁▁▁▁▁▁ ▁▁▁", + "▁ ▁▁▁▁▁▁▁▁▁", + "▁▁ ▁▁▁▁▁▁▁▁▁▁▁▁", + "▁▁▁▁ ▁▁▁▁▁▁▁▁▁▁", + "▁▁▁▁▁▁▁▁ ▁▁▁▁▁▁", + "▁▁▁▁▁ ▁▁▁▁▁▁▁▁▁", + "▁▁▁▁▁▁ ▁▁▁▁▁▁▁▁", + "▁▁▁▁▁▁▁▁▁▁▁▁ ▁▁", + "▁▁▁▁▁▁▁▁▁▁▁▁▁ ▁", + "▁▁▁▁▁▁▁▁▁▁ ▁▁▁▁", + "▁▁▁ ▁▁▁▁▁▁▁▁▁▁▁", + "▁▁▁▁▁▁▁▁▁ ▁▁▁▁▁", + "▁▁▁▁▁▁▁ ▁▁▁▁▁▁▁", + "▁▁▁▁▁▁▁▁▁▁▁ ▁▁▁", + "▁ ▁▁▁▁▁▁▁▁▁▁▁▁▁", + "▁▁ ▁", + "▁ ▁▁", + "▁▁ ▁▁▁▁▁▁▁", + "▁▁▁▁ ▁▁▁▁▁", + "▁▁▁▁▁▁▁▁ ▁", + "▁▁▁▁▁ ▁▁▁▁", + "▁▁▁▁▁▁ ▁▁▁", + "▁▁▁ ▁▁▁▁▁▁", + "▁▁▁▁▁▁▁ ▁▁", + "▁ ▁▁▁▁▁▁▁▁", + "▁▁ ▁▁▁▁▁", + "▁▁▁▁ ▁▁▁", + "▁▁▁▁▁ ▁▁", + "▁▁▁▁▁▁ ▁", + "▁▁▁ ▁▁▁▁", + "▁ ▁▁▁▁▁▁", + "▁▁ ▁▁▁▁▁▁▁▁▁", + "▁▁▁▁ ▁▁▁▁▁▁▁", + "▁▁▁▁▁▁▁▁ ▁▁▁", + "▁▁▁▁▁ ▁▁▁▁▁▁", + "▁▁▁▁▁▁ ▁▁▁▁▁", + "▁▁▁▁▁▁▁▁▁▁ ▁", + "▁▁▁ ▁▁▁▁▁▁▁▁", + "▁▁▁▁▁▁▁▁▁ ▁▁", + "▁▁▁▁▁▁▁ ▁▁▁▁", + "▁ ▁▁▁▁▁▁▁▁▁▁" + ] + } +} \ No newline at end of file diff --git a/tokenizer.model b/tokenizer.model new file mode 100644 index 0000000000000000000000000000000000000000..8b443ef19c2a19acc3ac64fb9c3db4a72921dff6 --- /dev/null +++ b/tokenizer.model @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055 +size 493443 diff --git a/tokenizer_config.json b/tokenizer_config.json new file mode 100644 index 0000000000000000000000000000000000000000..c57f1bf6ea28d2e3ca4540709beb3a80815c2aab --- /dev/null +++ b/tokenizer_config.json @@ -0,0 +1,46 @@ +{ + "added_tokens_decoder": { + "0": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "1": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "2": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + } + }, + "additional_special_tokens": [ + "", + "", + "" + ], + "bos_token": "", + "chat_template": "{% for message in messages %}\n{% if message['role'] == 'user' %}\n{{ '<|user|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'system' %}\n{{ '<|system|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'assistant' %}\n{{ '<|assistant|>\n' + message['content'] + eos_token }}\n{% endif %}\n{% if loop.last and add_generation_prompt %}\n{{ '<|assistant|>' }}\n{% endif %}\n{% endfor %}", + "clean_up_tokenization_spaces": false, + "eos_token": "", + "legacy": true, + "model_max_length": 1000000000000000019884624838656, + "pad_token": "", + "sp_model_kwargs": {}, + "spaces_between_special_tokens": false, + "tokenizer_class": "LlamaTokenizer", + "truncation_side": "left", + "unk_token": "", + "use_default_system_prompt": true +} diff --git a/train_results.json b/train_results.json new file mode 100644 index 0000000000000000000000000000000000000000..e8888ecb7c718b58d5cb9de7534763699ac02b6f --- /dev/null +++ b/train_results.json @@ -0,0 +1,8 @@ +{ + "epoch": 1.0, + "train_loss": 0.48803097129668166, + "train_runtime": 7971.1784, + "train_samples": 61966, + "train_samples_per_second": 7.774, + "train_steps_per_second": 0.243 +} \ No newline at end of file diff --git a/trainer_state.json b/trainer_state.json new file mode 100644 index 0000000000000000000000000000000000000000..d9bb067eb23cf5e71fd3ae0877cdc87be0eb6bb6 --- /dev/null +++ b/trainer_state.json @@ -0,0 +1,3048 @@ +{ + "best_metric": null, + "best_model_checkpoint": null, + "epoch": 1.0, + "eval_steps": 100, + "global_step": 1937, + "is_hyper_param_search": false, + "is_local_process_zero": true, + "is_world_process_zero": true, + "log_history": [ + { + "epoch": 0.0, + "learning_rate": 2.5773195876288657e-09, + "logits/chosen": -2.3950748443603516, + "logits/rejected": -2.440203905105591, + "logps/chosen": -134.41867065429688, + "logps/rejected": -130.99615478515625, + "loss": 0.6931, + "rewards/accuracies": 0.0, + "rewards/chosen": 0.0, + "rewards/margins": 0.0, + "rewards/rejected": 0.0, + "step": 1 + }, + { + "epoch": 0.01, + "learning_rate": 2.5773195876288656e-08, + "logits/chosen": -2.905548095703125, + "logits/rejected": -2.900480031967163, + "logps/chosen": -333.7513427734375, + "logps/rejected": -256.0840148925781, + "loss": 0.693, + "rewards/accuracies": 0.4444444477558136, + "rewards/chosen": 0.0011086573358625174, + "rewards/margins": 0.0019541841465979815, + "rewards/rejected": -0.0008455271599814296, + "step": 10 + }, + { + "epoch": 0.01, + "learning_rate": 5.154639175257731e-08, + "logits/chosen": -2.7942252159118652, + "logits/rejected": -2.5809876918792725, + "logps/chosen": -318.5552673339844, + "logps/rejected": -244.302734375, + "loss": 0.6874, + "rewards/accuracies": 0.6000000238418579, + "rewards/chosen": 0.007939901202917099, + "rewards/margins": 0.012212857604026794, + "rewards/rejected": -0.004272956866770983, + "step": 20 + }, + { + "epoch": 0.02, + "learning_rate": 7.731958762886598e-08, + "logits/chosen": -2.95878267288208, + "logits/rejected": -2.914288282394409, + "logps/chosen": -328.8043518066406, + "logps/rejected": -237.27151489257812, + "loss": 0.6807, + "rewards/accuracies": 0.5, + "rewards/chosen": 0.042353544384241104, + "rewards/margins": 0.013048592023551464, + "rewards/rejected": 0.029304955154657364, + "step": 30 + }, + { + "epoch": 0.02, + "learning_rate": 1.0309278350515462e-07, + "logits/chosen": -2.766305685043335, + "logits/rejected": -2.8168842792510986, + "logps/chosen": -298.6869812011719, + "logps/rejected": -283.5751037597656, + "loss": 0.6611, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": 0.053902607411146164, + "rewards/margins": 0.043827421963214874, + "rewards/rejected": 0.010075189173221588, + "step": 40 + }, + { + "epoch": 0.03, + "learning_rate": 1.2886597938144328e-07, + "logits/chosen": -2.940128803253174, + "logits/rejected": -2.9153294563293457, + "logps/chosen": -353.18408203125, + "logps/rejected": -278.1434326171875, + "loss": 0.6488, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": 0.10765328258275986, + "rewards/margins": 0.15659113228321075, + "rewards/rejected": -0.0489378497004509, + "step": 50 + }, + { + "epoch": 0.03, + "learning_rate": 1.5463917525773197e-07, + "logits/chosen": -2.9204277992248535, + "logits/rejected": -2.847609281539917, + "logps/chosen": -337.5159912109375, + "logps/rejected": -260.6918029785156, + "loss": 0.6253, + "rewards/accuracies": 0.8999999761581421, + "rewards/chosen": 0.17934152483940125, + "rewards/margins": 0.402275413274765, + "rewards/rejected": -0.22293388843536377, + "step": 60 + }, + { + "epoch": 0.04, + "learning_rate": 1.804123711340206e-07, + "logits/chosen": -2.9181199073791504, + "logits/rejected": -2.883514881134033, + "logps/chosen": -402.8895568847656, + "logps/rejected": -262.24609375, + "loss": 0.6168, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": 0.14267697930335999, + "rewards/margins": 0.45374640822410583, + "rewards/rejected": -0.3110693693161011, + "step": 70 + }, + { + "epoch": 0.04, + "learning_rate": 2.0618556701030925e-07, + "logits/chosen": -3.0056729316711426, + "logits/rejected": -2.903585433959961, + "logps/chosen": -391.81524658203125, + "logps/rejected": -263.700439453125, + "loss": 0.5578, + "rewards/accuracies": 0.699999988079071, + "rewards/chosen": -0.3499735891819, + "rewards/margins": 0.6209262609481812, + "rewards/rejected": -0.970899760723114, + "step": 80 + }, + { + "epoch": 0.05, + "learning_rate": 2.3195876288659794e-07, + "logits/chosen": -2.8997607231140137, + "logits/rejected": -2.787290573120117, + "logps/chosen": -333.6437072753906, + "logps/rejected": -248.12149047851562, + "loss": 0.5714, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.30834370851516724, + "rewards/margins": 0.7723485231399536, + "rewards/rejected": -1.0806924104690552, + "step": 90 + }, + { + "epoch": 0.05, + "learning_rate": 2.5773195876288655e-07, + "logits/chosen": -2.729923725128174, + "logits/rejected": -2.668921947479248, + "logps/chosen": -299.4058837890625, + "logps/rejected": -299.69464111328125, + "loss": 0.5602, + "rewards/accuracies": 0.699999988079071, + "rewards/chosen": -0.3236086368560791, + "rewards/margins": 0.48320120573043823, + "rewards/rejected": -0.8068099021911621, + "step": 100 + }, + { + "epoch": 0.05, + "eval_logits/chosen": -2.679702043533325, + "eval_logits/rejected": -2.655423879623413, + "eval_logps/chosen": -293.716064453125, + "eval_logps/rejected": -306.2606506347656, + "eval_loss": 0.5588741302490234, + "eval_rewards/accuracies": 0.71875, + "eval_rewards/chosen": -0.3358975350856781, + "eval_rewards/margins": 0.48091739416122437, + "eval_rewards/rejected": -0.8168148994445801, + "eval_runtime": 59.9141, + "eval_samples_per_second": 16.691, + "eval_steps_per_second": 0.267, + "step": 100 + }, + { + "epoch": 0.06, + "learning_rate": 2.835051546391752e-07, + "logits/chosen": -2.91627836227417, + "logits/rejected": -2.970510482788086, + "logps/chosen": -383.3172302246094, + "logps/rejected": -347.4027404785156, + "loss": 0.5764, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.4358394742012024, + "rewards/margins": 0.6925853490829468, + "rewards/rejected": -1.128424882888794, + "step": 110 + }, + { + "epoch": 0.06, + "learning_rate": 3.0927835051546394e-07, + "logits/chosen": -2.760704517364502, + "logits/rejected": -2.690364360809326, + "logps/chosen": -230.55453491210938, + "logps/rejected": -258.10711669921875, + "loss": 0.5352, + "rewards/accuracies": 0.550000011920929, + "rewards/chosen": -0.5467896461486816, + "rewards/margins": 0.3793262541294098, + "rewards/rejected": -0.926115870475769, + "step": 120 + }, + { + "epoch": 0.07, + "learning_rate": 3.3505154639175255e-07, + "logits/chosen": -2.9248669147491455, + "logits/rejected": -2.743758201599121, + "logps/chosen": -375.25665283203125, + "logps/rejected": -237.9368438720703, + "loss": 0.5478, + "rewards/accuracies": 0.6499999761581421, + "rewards/chosen": -0.2538636326789856, + "rewards/margins": 0.8244791030883789, + "rewards/rejected": -1.0783426761627197, + "step": 130 + }, + { + "epoch": 0.07, + "learning_rate": 3.608247422680412e-07, + "logits/chosen": -2.789468765258789, + "logits/rejected": -2.6087424755096436, + "logps/chosen": -341.793212890625, + "logps/rejected": -303.3952941894531, + "loss": 0.531, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.5787044763565063, + "rewards/margins": 0.7447447776794434, + "rewards/rejected": -1.3234491348266602, + "step": 140 + }, + { + "epoch": 0.08, + "learning_rate": 3.865979381443299e-07, + "logits/chosen": -3.055696964263916, + "logits/rejected": -2.8961217403411865, + "logps/chosen": -342.10162353515625, + "logps/rejected": -362.9927978515625, + "loss": 0.5117, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.22076380252838135, + "rewards/margins": 1.0446679592132568, + "rewards/rejected": -1.2654317617416382, + "step": 150 + }, + { + "epoch": 0.08, + "learning_rate": 4.123711340206185e-07, + "logits/chosen": -2.769010066986084, + "logits/rejected": -2.677729368209839, + "logps/chosen": -316.1387939453125, + "logps/rejected": -257.9635009765625, + "loss": 0.5264, + "rewards/accuracies": 0.6499999761581421, + "rewards/chosen": -0.5055423378944397, + "rewards/margins": 0.9351354837417603, + "rewards/rejected": -1.4406778812408447, + "step": 160 + }, + { + "epoch": 0.09, + "learning_rate": 4.381443298969072e-07, + "logits/chosen": -2.7085089683532715, + "logits/rejected": -2.600250005722046, + "logps/chosen": -276.8943176269531, + "logps/rejected": -258.60809326171875, + "loss": 0.478, + "rewards/accuracies": 0.8999999761581421, + "rewards/chosen": -0.2844000458717346, + "rewards/margins": 1.338111162185669, + "rewards/rejected": -1.6225111484527588, + "step": 170 + }, + { + "epoch": 0.09, + "learning_rate": 4.639175257731959e-07, + "logits/chosen": -2.8041911125183105, + "logits/rejected": -2.6761889457702637, + "logps/chosen": -272.099365234375, + "logps/rejected": -236.39675903320312, + "loss": 0.4904, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.0498843789100647, + "rewards/margins": 1.0812078714370728, + "rewards/rejected": -1.1310923099517822, + "step": 180 + }, + { + "epoch": 0.1, + "learning_rate": 4.896907216494845e-07, + "logits/chosen": -2.7893078327178955, + "logits/rejected": -2.787668228149414, + "logps/chosen": -315.68365478515625, + "logps/rejected": -278.1927490234375, + "loss": 0.5809, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.05461850017309189, + "rewards/margins": 0.6613677740097046, + "rewards/rejected": -0.7159863114356995, + "step": 190 + }, + { + "epoch": 0.1, + "learning_rate": 4.982788296041308e-07, + "logits/chosen": -2.7293038368225098, + "logits/rejected": -2.7431674003601074, + "logps/chosen": -319.84307861328125, + "logps/rejected": -285.5979309082031, + "loss": 0.4852, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.25821852684020996, + "rewards/margins": 0.9920754432678223, + "rewards/rejected": -1.2502939701080322, + "step": 200 + }, + { + "epoch": 0.1, + "eval_logits/chosen": -2.5956599712371826, + "eval_logits/rejected": -2.576249122619629, + "eval_logps/chosen": -297.6180725097656, + "eval_logps/rejected": -319.912353515625, + "eval_loss": 0.513595700263977, + "eval_rewards/accuracies": 0.8125, + "eval_rewards/chosen": -0.5309990048408508, + "eval_rewards/margins": 0.9684009552001953, + "eval_rewards/rejected": -1.4993999004364014, + "eval_runtime": 58.9562, + "eval_samples_per_second": 16.962, + "eval_steps_per_second": 0.271, + "step": 200 + }, + { + "epoch": 0.11, + "learning_rate": 4.954102122776821e-07, + "logits/chosen": -2.4963221549987793, + "logits/rejected": -2.2522215843200684, + "logps/chosen": -330.4200134277344, + "logps/rejected": -269.993896484375, + "loss": 0.5766, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.39517518877983093, + "rewards/margins": 1.0865150690078735, + "rewards/rejected": -1.4816901683807373, + "step": 210 + }, + { + "epoch": 0.11, + "learning_rate": 4.925415949512335e-07, + "logits/chosen": -2.709627628326416, + "logits/rejected": -2.6324925422668457, + "logps/chosen": -355.61602783203125, + "logps/rejected": -297.36346435546875, + "loss": 0.5109, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.4435829222202301, + "rewards/margins": 0.6400429606437683, + "rewards/rejected": -1.0836259126663208, + "step": 220 + }, + { + "epoch": 0.12, + "learning_rate": 4.896729776247848e-07, + "logits/chosen": -2.7627291679382324, + "logits/rejected": -2.728057384490967, + "logps/chosen": -326.62017822265625, + "logps/rejected": -297.73150634765625, + "loss": 0.4948, + "rewards/accuracies": 0.6499999761581421, + "rewards/chosen": -0.0938631147146225, + "rewards/margins": 0.8200471997261047, + "rewards/rejected": -0.9139102697372437, + "step": 230 + }, + { + "epoch": 0.12, + "learning_rate": 4.868043602983362e-07, + "logits/chosen": -2.758730173110962, + "logits/rejected": -2.7346835136413574, + "logps/chosen": -399.9264221191406, + "logps/rejected": -287.3780212402344, + "loss": 0.5489, + "rewards/accuracies": 0.550000011920929, + "rewards/chosen": -0.5322622656822205, + "rewards/margins": 0.36636096239089966, + "rewards/rejected": -0.8986232876777649, + "step": 240 + }, + { + "epoch": 0.13, + "learning_rate": 4.839357429718875e-07, + "logits/chosen": -2.922417163848877, + "logits/rejected": -2.7935752868652344, + "logps/chosen": -267.0278015136719, + "logps/rejected": -201.1547088623047, + "loss": 0.509, + "rewards/accuracies": 0.949999988079071, + "rewards/chosen": 0.024714648723602295, + "rewards/margins": 1.111724615097046, + "rewards/rejected": -1.087010145187378, + "step": 250 + }, + { + "epoch": 0.13, + "learning_rate": 4.810671256454389e-07, + "logits/chosen": -2.9726548194885254, + "logits/rejected": -2.8518033027648926, + "logps/chosen": -345.22869873046875, + "logps/rejected": -239.1555938720703, + "loss": 0.5219, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": 0.062345243990421295, + "rewards/margins": 1.2734425067901611, + "rewards/rejected": -1.211097240447998, + "step": 260 + }, + { + "epoch": 0.14, + "learning_rate": 4.781985083189902e-07, + "logits/chosen": -2.840888500213623, + "logits/rejected": -2.7617764472961426, + "logps/chosen": -343.1612243652344, + "logps/rejected": -278.6701354980469, + "loss": 0.4955, + "rewards/accuracies": 0.6000000238418579, + "rewards/chosen": -0.15016205608844757, + "rewards/margins": 0.7697200179100037, + "rewards/rejected": -0.9198821187019348, + "step": 270 + }, + { + "epoch": 0.14, + "learning_rate": 4.7532989099254154e-07, + "logits/chosen": -2.9551005363464355, + "logits/rejected": -2.8918609619140625, + "logps/chosen": -391.0706787109375, + "logps/rejected": -303.4861755371094, + "loss": 0.5101, + "rewards/accuracies": 0.699999988079071, + "rewards/chosen": -0.2836604118347168, + "rewards/margins": 0.7845603227615356, + "rewards/rejected": -1.068220853805542, + "step": 280 + }, + { + "epoch": 0.15, + "learning_rate": 4.724612736660929e-07, + "logits/chosen": -2.664578437805176, + "logits/rejected": -2.801819086074829, + "logps/chosen": -272.6911315917969, + "logps/rejected": -268.9104919433594, + "loss": 0.4815, + "rewards/accuracies": 0.6000000238418579, + "rewards/chosen": 0.15687711536884308, + "rewards/margins": 0.9703904986381531, + "rewards/rejected": -0.813513457775116, + "step": 290 + }, + { + "epoch": 0.15, + "learning_rate": 4.695926563396443e-07, + "logits/chosen": -2.7159955501556396, + "logits/rejected": -2.7721734046936035, + "logps/chosen": -444.1826171875, + "logps/rejected": -277.5564270019531, + "loss": 0.5212, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": 0.18774226307868958, + "rewards/margins": 1.6489717960357666, + "rewards/rejected": -1.4612294435501099, + "step": 300 + }, + { + "epoch": 0.15, + "eval_logits/chosen": -2.7125377655029297, + "eval_logits/rejected": -2.6865103244781494, + "eval_logps/chosen": -290.3699035644531, + "eval_logps/rejected": -313.44439697265625, + "eval_loss": 0.5167616009712219, + "eval_rewards/accuracies": 0.78125, + "eval_rewards/chosen": -0.1685914546251297, + "eval_rewards/margins": 1.0074106454849243, + "eval_rewards/rejected": -1.1760022640228271, + "eval_runtime": 57.2558, + "eval_samples_per_second": 17.465, + "eval_steps_per_second": 0.279, + "step": 300 + }, + { + "epoch": 0.16, + "learning_rate": 4.6672403901319564e-07, + "logits/chosen": -2.9332168102264404, + "logits/rejected": -2.83400297164917, + "logps/chosen": -351.634033203125, + "logps/rejected": -267.8584899902344, + "loss": 0.5018, + "rewards/accuracies": 0.8999999761581421, + "rewards/chosen": -0.10320504754781723, + "rewards/margins": 1.208617091178894, + "rewards/rejected": -1.3118221759796143, + "step": 310 + }, + { + "epoch": 0.17, + "learning_rate": 4.63855421686747e-07, + "logits/chosen": -2.86641001701355, + "logits/rejected": -2.9064583778381348, + "logps/chosen": -401.264404296875, + "logps/rejected": -330.1752624511719, + "loss": 0.4492, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.5078498125076294, + "rewards/margins": 1.2765679359436035, + "rewards/rejected": -1.784417748451233, + "step": 320 + }, + { + "epoch": 0.17, + "learning_rate": 4.609868043602983e-07, + "logits/chosen": -2.7677345275878906, + "logits/rejected": -2.808917760848999, + "logps/chosen": -373.0283203125, + "logps/rejected": -278.2051086425781, + "loss": 0.4576, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.5256485939025879, + "rewards/margins": 1.5822279453277588, + "rewards/rejected": -2.1078765392303467, + "step": 330 + }, + { + "epoch": 0.18, + "learning_rate": 4.581181870338497e-07, + "logits/chosen": -2.5862860679626465, + "logits/rejected": -2.6729650497436523, + "logps/chosen": -264.1101989746094, + "logps/rejected": -302.21832275390625, + "loss": 0.5585, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.3558864891529083, + "rewards/margins": 0.7631327509880066, + "rewards/rejected": -1.1190193891525269, + "step": 340 + }, + { + "epoch": 0.18, + "learning_rate": 4.55249569707401e-07, + "logits/chosen": -2.9297871589660645, + "logits/rejected": -2.8774681091308594, + "logps/chosen": -199.7515106201172, + "logps/rejected": -261.4060363769531, + "loss": 0.4773, + "rewards/accuracies": 0.6499999761581421, + "rewards/chosen": -0.46969500184059143, + "rewards/margins": 0.5962409973144531, + "rewards/rejected": -1.0659358501434326, + "step": 350 + }, + { + "epoch": 0.19, + "learning_rate": 4.5238095238095237e-07, + "logits/chosen": -2.9737091064453125, + "logits/rejected": -3.0415940284729004, + "logps/chosen": -417.215087890625, + "logps/rejected": -402.1128845214844, + "loss": 0.4545, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": 0.06597085297107697, + "rewards/margins": 1.6805864572525024, + "rewards/rejected": -1.6146154403686523, + "step": 360 + }, + { + "epoch": 0.19, + "learning_rate": 4.495123350545037e-07, + "logits/chosen": -2.9340670108795166, + "logits/rejected": -2.8952622413635254, + "logps/chosen": -200.85787963867188, + "logps/rejected": -190.8145751953125, + "loss": 0.462, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.39831048250198364, + "rewards/margins": 1.7207963466644287, + "rewards/rejected": -2.1191065311431885, + "step": 370 + }, + { + "epoch": 0.2, + "learning_rate": 4.46643717728055e-07, + "logits/chosen": -2.7802371978759766, + "logits/rejected": -2.692594051361084, + "logps/chosen": -342.89556884765625, + "logps/rejected": -360.488037109375, + "loss": 0.5404, + "rewards/accuracies": 0.6000000238418579, + "rewards/chosen": -0.2285180538892746, + "rewards/margins": 0.9076235890388489, + "rewards/rejected": -1.1361417770385742, + "step": 380 + }, + { + "epoch": 0.2, + "learning_rate": 4.437751004016064e-07, + "logits/chosen": -2.8958873748779297, + "logits/rejected": -2.8253509998321533, + "logps/chosen": -338.5411071777344, + "logps/rejected": -302.31536865234375, + "loss": 0.4479, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.5085374712944031, + "rewards/margins": 1.4444196224212646, + "rewards/rejected": -1.9529569149017334, + "step": 390 + }, + { + "epoch": 0.21, + "learning_rate": 4.4090648307515776e-07, + "logits/chosen": -3.072361469268799, + "logits/rejected": -3.0403876304626465, + "logps/chosen": -298.0294189453125, + "logps/rejected": -269.53759765625, + "loss": 0.5496, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.7352712154388428, + "rewards/margins": 0.4947175979614258, + "rewards/rejected": -1.229988694190979, + "step": 400 + }, + { + "epoch": 0.21, + "eval_logits/chosen": -2.8217618465423584, + "eval_logits/rejected": -2.7946977615356445, + "eval_logps/chosen": -290.2325744628906, + "eval_logps/rejected": -324.2634582519531, + "eval_loss": 0.4835220277309418, + "eval_rewards/accuracies": 0.828125, + "eval_rewards/chosen": -0.16172367334365845, + "eval_rewards/margins": 1.5552303791046143, + "eval_rewards/rejected": -1.716953992843628, + "eval_runtime": 58.8529, + "eval_samples_per_second": 16.992, + "eval_steps_per_second": 0.272, + "step": 400 + }, + { + "epoch": 0.21, + "learning_rate": 4.380378657487091e-07, + "logits/chosen": -2.878788948059082, + "logits/rejected": -2.7508370876312256, + "logps/chosen": -361.15850830078125, + "logps/rejected": -244.7436065673828, + "loss": 0.4209, + "rewards/accuracies": 0.6499999761581421, + "rewards/chosen": -0.3598092496395111, + "rewards/margins": 0.8073941469192505, + "rewards/rejected": -1.1672031879425049, + "step": 410 + }, + { + "epoch": 0.22, + "learning_rate": 4.3516924842226045e-07, + "logits/chosen": -2.940977096557617, + "logits/rejected": -2.811171531677246, + "logps/chosen": -342.15802001953125, + "logps/rejected": -299.4014587402344, + "loss": 0.4994, + "rewards/accuracies": 0.550000011920929, + "rewards/chosen": -0.5423839092254639, + "rewards/margins": 1.0953930616378784, + "rewards/rejected": -1.6377769708633423, + "step": 420 + }, + { + "epoch": 0.22, + "learning_rate": 4.323006310958118e-07, + "logits/chosen": -2.928069591522217, + "logits/rejected": -2.839228391647339, + "logps/chosen": -246.8507080078125, + "logps/rejected": -235.521728515625, + "loss": 0.4726, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.7062541246414185, + "rewards/margins": 0.9924257397651672, + "rewards/rejected": -1.6986801624298096, + "step": 430 + }, + { + "epoch": 0.23, + "learning_rate": 4.2943201376936315e-07, + "logits/chosen": -2.8603832721710205, + "logits/rejected": -2.7824721336364746, + "logps/chosen": -376.1601257324219, + "logps/rejected": -356.33905029296875, + "loss": 0.5524, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.40359026193618774, + "rewards/margins": 0.40071097016334534, + "rewards/rejected": -0.8043011426925659, + "step": 440 + }, + { + "epoch": 0.23, + "learning_rate": 4.265633964429145e-07, + "logits/chosen": -2.9410994052886963, + "logits/rejected": -2.9663491249084473, + "logps/chosen": -346.1313171386719, + "logps/rejected": -281.6514587402344, + "loss": 0.469, + "rewards/accuracies": 0.6000000238418579, + "rewards/chosen": -0.5330409407615662, + "rewards/margins": 0.553443968296051, + "rewards/rejected": -1.0864850282669067, + "step": 450 + }, + { + "epoch": 0.24, + "learning_rate": 4.2369477911646584e-07, + "logits/chosen": -2.9534189701080322, + "logits/rejected": -2.9417405128479004, + "logps/chosen": -362.89105224609375, + "logps/rejected": -356.94598388671875, + "loss": 0.546, + "rewards/accuracies": 0.8999999761581421, + "rewards/chosen": -0.10002334415912628, + "rewards/margins": 1.6824315786361694, + "rewards/rejected": -1.7824550867080688, + "step": 460 + }, + { + "epoch": 0.24, + "learning_rate": 4.208261617900172e-07, + "logits/chosen": -2.8067574501037598, + "logits/rejected": -2.815100908279419, + "logps/chosen": -359.10791015625, + "logps/rejected": -276.3143615722656, + "loss": 0.4473, + "rewards/accuracies": 0.6499999761581421, + "rewards/chosen": -0.5546383857727051, + "rewards/margins": 0.8130642175674438, + "rewards/rejected": -1.367702603340149, + "step": 470 + }, + { + "epoch": 0.25, + "learning_rate": 4.179575444635686e-07, + "logits/chosen": -3.0283010005950928, + "logits/rejected": -3.0092482566833496, + "logps/chosen": -320.7592468261719, + "logps/rejected": -327.0148010253906, + "loss": 0.4589, + "rewards/accuracies": 0.6000000238418579, + "rewards/chosen": -0.023906176909804344, + "rewards/margins": 0.516281008720398, + "rewards/rejected": -0.5401870608329773, + "step": 480 + }, + { + "epoch": 0.25, + "learning_rate": 4.150889271371199e-07, + "logits/chosen": -2.9509072303771973, + "logits/rejected": -2.9088292121887207, + "logps/chosen": -359.8896789550781, + "logps/rejected": -298.53167724609375, + "loss": 0.4272, + "rewards/accuracies": 0.6499999761581421, + "rewards/chosen": -0.9165481328964233, + "rewards/margins": 0.8448396921157837, + "rewards/rejected": -1.761387586593628, + "step": 490 + }, + { + "epoch": 0.26, + "learning_rate": 4.1222030981067123e-07, + "logits/chosen": -3.1632120609283447, + "logits/rejected": -2.959442138671875, + "logps/chosen": -348.33709716796875, + "logps/rejected": -335.841552734375, + "loss": 0.5209, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": 0.01871591992676258, + "rewards/margins": 1.6875536441802979, + "rewards/rejected": -1.668837547302246, + "step": 500 + }, + { + "epoch": 0.26, + "eval_logits/chosen": -2.8667073249816895, + "eval_logits/rejected": -2.8388283252716064, + "eval_logps/chosen": -296.5545654296875, + "eval_logps/rejected": -323.13250732421875, + "eval_loss": 0.5053695440292358, + "eval_rewards/accuracies": 0.734375, + "eval_rewards/chosen": -0.4778231978416443, + "eval_rewards/margins": 1.1825826168060303, + "eval_rewards/rejected": -1.6604057550430298, + "eval_runtime": 53.6768, + "eval_samples_per_second": 18.63, + "eval_steps_per_second": 0.298, + "step": 500 + }, + { + "epoch": 0.26, + "learning_rate": 4.093516924842226e-07, + "logits/chosen": -2.5999233722686768, + "logits/rejected": -2.6488568782806396, + "logps/chosen": -279.40753173828125, + "logps/rejected": -288.4382019042969, + "loss": 0.5262, + "rewards/accuracies": 0.6499999761581421, + "rewards/chosen": -0.03970176726579666, + "rewards/margins": 0.8601824641227722, + "rewards/rejected": -0.8998842239379883, + "step": 510 + }, + { + "epoch": 0.27, + "learning_rate": 4.064830751577739e-07, + "logits/chosen": -3.004120349884033, + "logits/rejected": -2.983384132385254, + "logps/chosen": -223.1768035888672, + "logps/rejected": -233.73312377929688, + "loss": 0.5426, + "rewards/accuracies": 0.699999988079071, + "rewards/chosen": -0.22979409992694855, + "rewards/margins": 1.1553027629852295, + "rewards/rejected": -1.3850969076156616, + "step": 520 + }, + { + "epoch": 0.27, + "learning_rate": 4.036144578313253e-07, + "logits/chosen": -3.057387351989746, + "logits/rejected": -2.9704792499542236, + "logps/chosen": -325.57025146484375, + "logps/rejected": -283.2383117675781, + "loss": 0.5058, + "rewards/accuracies": 0.949999988079071, + "rewards/chosen": 0.17162951827049255, + "rewards/margins": 1.4068710803985596, + "rewards/rejected": -1.2352415323257446, + "step": 530 + }, + { + "epoch": 0.28, + "learning_rate": 4.007458405048766e-07, + "logits/chosen": -2.940493583679199, + "logits/rejected": -2.9570419788360596, + "logps/chosen": -227.3856964111328, + "logps/rejected": -289.257080078125, + "loss": 0.5339, + "rewards/accuracies": 0.6499999761581421, + "rewards/chosen": -0.835346519947052, + "rewards/margins": 0.5295913219451904, + "rewards/rejected": -1.3649379014968872, + "step": 540 + }, + { + "epoch": 0.28, + "learning_rate": 3.9787722317842796e-07, + "logits/chosen": -3.0390350818634033, + "logits/rejected": -3.0303795337677, + "logps/chosen": -389.0104675292969, + "logps/rejected": -340.8678283691406, + "loss": 0.5671, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.15655937790870667, + "rewards/margins": 1.444120168685913, + "rewards/rejected": -1.6006797552108765, + "step": 550 + }, + { + "epoch": 0.29, + "learning_rate": 3.950086058519793e-07, + "logits/chosen": -3.060770034790039, + "logits/rejected": -3.0684587955474854, + "logps/chosen": -366.5193786621094, + "logps/rejected": -359.5283203125, + "loss": 0.501, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.8856328129768372, + "rewards/margins": 1.0001723766326904, + "rewards/rejected": -1.8858048915863037, + "step": 560 + }, + { + "epoch": 0.29, + "learning_rate": 3.9213998852553066e-07, + "logits/chosen": -3.022430896759033, + "logits/rejected": -2.831993341445923, + "logps/chosen": -379.4962463378906, + "logps/rejected": -278.2923889160156, + "loss": 0.4947, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.7191627621650696, + "rewards/margins": 1.4330142736434937, + "rewards/rejected": -2.152177095413208, + "step": 570 + }, + { + "epoch": 0.3, + "learning_rate": 3.8927137119908206e-07, + "logits/chosen": -2.922940731048584, + "logits/rejected": -2.9428744316101074, + "logps/chosen": -361.6253356933594, + "logps/rejected": -278.06280517578125, + "loss": 0.5375, + "rewards/accuracies": 0.6499999761581421, + "rewards/chosen": -0.573477029800415, + "rewards/margins": 0.7330626249313354, + "rewards/rejected": -1.30653977394104, + "step": 580 + }, + { + "epoch": 0.3, + "learning_rate": 3.864027538726334e-07, + "logits/chosen": -2.9140264987945557, + "logits/rejected": -2.889644145965576, + "logps/chosen": -289.79058837890625, + "logps/rejected": -292.6978759765625, + "loss": 0.4726, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.1533357799053192, + "rewards/margins": 1.4288156032562256, + "rewards/rejected": -1.582151174545288, + "step": 590 + }, + { + "epoch": 0.31, + "learning_rate": 3.835341365461847e-07, + "logits/chosen": -2.869101047515869, + "logits/rejected": -2.88112735748291, + "logps/chosen": -243.238037109375, + "logps/rejected": -313.36260986328125, + "loss": 0.4617, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.5780342221260071, + "rewards/margins": 1.605724573135376, + "rewards/rejected": -2.1837587356567383, + "step": 600 + }, + { + "epoch": 0.31, + "eval_logits/chosen": -2.8521082401275635, + "eval_logits/rejected": -2.823408603668213, + "eval_logps/chosen": -294.47406005859375, + "eval_logps/rejected": -320.2847900390625, + "eval_loss": 0.4909786283969879, + "eval_rewards/accuracies": 0.765625, + "eval_rewards/chosen": -0.373797744512558, + "eval_rewards/margins": 1.1442232131958008, + "eval_rewards/rejected": -1.5180209875106812, + "eval_runtime": 59.0534, + "eval_samples_per_second": 16.934, + "eval_steps_per_second": 0.271, + "step": 600 + }, + { + "epoch": 0.31, + "learning_rate": 3.8066551921973605e-07, + "logits/chosen": -2.9201841354370117, + "logits/rejected": -2.845341205596924, + "logps/chosen": -338.87127685546875, + "logps/rejected": -337.95928955078125, + "loss": 0.5093, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.7563498020172119, + "rewards/margins": 1.1776630878448486, + "rewards/rejected": -1.934012770652771, + "step": 610 + }, + { + "epoch": 0.32, + "learning_rate": 3.7779690189328745e-07, + "logits/chosen": -3.034162759780884, + "logits/rejected": -3.027251720428467, + "logps/chosen": -379.36590576171875, + "logps/rejected": -306.5363464355469, + "loss": 0.5269, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.6596435308456421, + "rewards/margins": 0.7433546185493469, + "rewards/rejected": -1.4029979705810547, + "step": 620 + }, + { + "epoch": 0.33, + "learning_rate": 3.749282845668388e-07, + "logits/chosen": -2.87540864944458, + "logits/rejected": -2.834453821182251, + "logps/chosen": -252.4665069580078, + "logps/rejected": -234.0683135986328, + "loss": 0.4806, + "rewards/accuracies": 0.75, + "rewards/chosen": -1.2540767192840576, + "rewards/margins": 1.1702220439910889, + "rewards/rejected": -2.4242987632751465, + "step": 630 + }, + { + "epoch": 0.33, + "learning_rate": 3.7205966724039014e-07, + "logits/chosen": -2.9034817218780518, + "logits/rejected": -2.8846733570098877, + "logps/chosen": -308.4912109375, + "logps/rejected": -294.0049743652344, + "loss": 0.5071, + "rewards/accuracies": 0.8999999761581421, + "rewards/chosen": -0.7412096261978149, + "rewards/margins": 1.7391561269760132, + "rewards/rejected": -2.480365753173828, + "step": 640 + }, + { + "epoch": 0.34, + "learning_rate": 3.6919104991394144e-07, + "logits/chosen": -2.931398868560791, + "logits/rejected": -2.982588291168213, + "logps/chosen": -284.8676452636719, + "logps/rejected": -328.3984069824219, + "loss": 0.5295, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.8696074485778809, + "rewards/margins": 1.0539840459823608, + "rewards/rejected": -1.9235913753509521, + "step": 650 + }, + { + "epoch": 0.34, + "learning_rate": 3.663224325874928e-07, + "logits/chosen": -3.033780097961426, + "logits/rejected": -2.9609603881835938, + "logps/chosen": -372.281005859375, + "logps/rejected": -324.37640380859375, + "loss": 0.4882, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.7286888957023621, + "rewards/margins": 1.0642763376235962, + "rewards/rejected": -1.792965292930603, + "step": 660 + }, + { + "epoch": 0.35, + "learning_rate": 3.634538152610442e-07, + "logits/chosen": -2.93717622756958, + "logits/rejected": -2.9276621341705322, + "logps/chosen": -240.49267578125, + "logps/rejected": -235.82901000976562, + "loss": 0.5725, + "rewards/accuracies": 0.6000000238418579, + "rewards/chosen": -0.5567636489868164, + "rewards/margins": 0.7625681161880493, + "rewards/rejected": -1.3193317651748657, + "step": 670 + }, + { + "epoch": 0.35, + "learning_rate": 3.6058519793459553e-07, + "logits/chosen": -2.834033727645874, + "logits/rejected": -2.778200387954712, + "logps/chosen": -311.96209716796875, + "logps/rejected": -279.34210205078125, + "loss": 0.4695, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.3849760890007019, + "rewards/margins": 1.1402137279510498, + "rewards/rejected": -1.5251898765563965, + "step": 680 + }, + { + "epoch": 0.36, + "learning_rate": 3.577165806081469e-07, + "logits/chosen": -3.0405967235565186, + "logits/rejected": -2.836169481277466, + "logps/chosen": -377.798828125, + "logps/rejected": -225.7850799560547, + "loss": 0.4235, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": 0.16026358306407928, + "rewards/margins": 1.4788005352020264, + "rewards/rejected": -1.3185369968414307, + "step": 690 + }, + { + "epoch": 0.36, + "learning_rate": 3.5484796328169817e-07, + "logits/chosen": -2.914154052734375, + "logits/rejected": -2.837174415588379, + "logps/chosen": -486.6206970214844, + "logps/rejected": -388.7563171386719, + "loss": 0.4452, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": 0.3467257618904114, + "rewards/margins": 1.8637710809707642, + "rewards/rejected": -1.517045021057129, + "step": 700 + }, + { + "epoch": 0.36, + "eval_logits/chosen": -2.7652909755706787, + "eval_logits/rejected": -2.7400858402252197, + "eval_logps/chosen": -296.1795959472656, + "eval_logps/rejected": -323.0770263671875, + "eval_loss": 0.48382142186164856, + "eval_rewards/accuracies": 0.703125, + "eval_rewards/chosen": -0.45907530188560486, + "eval_rewards/margins": 1.19855797290802, + "eval_rewards/rejected": -1.6576331853866577, + "eval_runtime": 56.4234, + "eval_samples_per_second": 17.723, + "eval_steps_per_second": 0.284, + "step": 700 + }, + { + "epoch": 0.37, + "learning_rate": 3.519793459552495e-07, + "logits/chosen": -2.81813645362854, + "logits/rejected": -2.891169309616089, + "logps/chosen": -304.12921142578125, + "logps/rejected": -300.01275634765625, + "loss": 0.4796, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -1.1196739673614502, + "rewards/margins": 0.9914043545722961, + "rewards/rejected": -2.1110780239105225, + "step": 710 + }, + { + "epoch": 0.37, + "learning_rate": 3.491107286288009e-07, + "logits/chosen": -2.8922622203826904, + "logits/rejected": -2.8977553844451904, + "logps/chosen": -317.526123046875, + "logps/rejected": -434.422119140625, + "loss": 0.4804, + "rewards/accuracies": 1.0, + "rewards/chosen": -0.36432236433029175, + "rewards/margins": 2.1755287647247314, + "rewards/rejected": -2.539850950241089, + "step": 720 + }, + { + "epoch": 0.38, + "learning_rate": 3.4624211130235227e-07, + "logits/chosen": -2.897742509841919, + "logits/rejected": -2.849132537841797, + "logps/chosen": -359.33770751953125, + "logps/rejected": -375.489990234375, + "loss": 0.4515, + "rewards/accuracies": 0.6499999761581421, + "rewards/chosen": -0.7771750688552856, + "rewards/margins": 0.7097036838531494, + "rewards/rejected": -1.4868788719177246, + "step": 730 + }, + { + "epoch": 0.38, + "learning_rate": 3.433734939759036e-07, + "logits/chosen": -2.8762059211730957, + "logits/rejected": -2.845869302749634, + "logps/chosen": -367.5733642578125, + "logps/rejected": -284.77886962890625, + "loss": 0.5371, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.7336939573287964, + "rewards/margins": 1.311545968055725, + "rewards/rejected": -2.0452399253845215, + "step": 740 + }, + { + "epoch": 0.39, + "learning_rate": 3.405048766494549e-07, + "logits/chosen": -2.8201470375061035, + "logits/rejected": -2.8354251384735107, + "logps/chosen": -340.94549560546875, + "logps/rejected": -293.8914794921875, + "loss": 0.4984, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.7219395637512207, + "rewards/margins": 1.4890177249908447, + "rewards/rejected": -2.2109572887420654, + "step": 750 + }, + { + "epoch": 0.39, + "learning_rate": 3.376362593230063e-07, + "logits/chosen": -2.7780914306640625, + "logits/rejected": -2.7091243267059326, + "logps/chosen": -291.79864501953125, + "logps/rejected": -293.62762451171875, + "loss": 0.5131, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.4624660611152649, + "rewards/margins": 1.375497579574585, + "rewards/rejected": -1.8379634618759155, + "step": 760 + }, + { + "epoch": 0.4, + "learning_rate": 3.3476764199655765e-07, + "logits/chosen": -2.9393422603607178, + "logits/rejected": -2.913353681564331, + "logps/chosen": -266.66033935546875, + "logps/rejected": -310.0787658691406, + "loss": 0.5009, + "rewards/accuracies": 0.699999988079071, + "rewards/chosen": -0.27197906374931335, + "rewards/margins": 1.5637094974517822, + "rewards/rejected": -1.835688591003418, + "step": 770 + }, + { + "epoch": 0.4, + "learning_rate": 3.31899024670109e-07, + "logits/chosen": -2.799229621887207, + "logits/rejected": -2.7572007179260254, + "logps/chosen": -322.958251953125, + "logps/rejected": -250.6745147705078, + "loss": 0.462, + "rewards/accuracies": 0.8999999761581421, + "rewards/chosen": -0.1813792735338211, + "rewards/margins": 1.8009974956512451, + "rewards/rejected": -1.9823768138885498, + "step": 780 + }, + { + "epoch": 0.41, + "learning_rate": 3.2903040734366035e-07, + "logits/chosen": -2.9396896362304688, + "logits/rejected": -2.8256068229675293, + "logps/chosen": -336.33111572265625, + "logps/rejected": -305.9268493652344, + "loss": 0.4724, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.5670480728149414, + "rewards/margins": 1.4373046159744263, + "rewards/rejected": -2.004352569580078, + "step": 790 + }, + { + "epoch": 0.41, + "learning_rate": 3.261617900172117e-07, + "logits/chosen": -2.8622994422912598, + "logits/rejected": -2.8278136253356934, + "logps/chosen": -284.0804443359375, + "logps/rejected": -313.81939697265625, + "loss": 0.4674, + "rewards/accuracies": 0.8999999761581421, + "rewards/chosen": 0.13986745476722717, + "rewards/margins": 1.9817402362823486, + "rewards/rejected": -1.8418725728988647, + "step": 800 + }, + { + "epoch": 0.41, + "eval_logits/chosen": -2.6945066452026367, + "eval_logits/rejected": -2.6739954948425293, + "eval_logps/chosen": -298.38177490234375, + "eval_logps/rejected": -327.2416076660156, + "eval_loss": 0.5077354907989502, + "eval_rewards/accuracies": 0.765625, + "eval_rewards/chosen": -0.5691820383071899, + "eval_rewards/margins": 1.2966790199279785, + "eval_rewards/rejected": -1.865861177444458, + "eval_runtime": 55.4449, + "eval_samples_per_second": 18.036, + "eval_steps_per_second": 0.289, + "step": 800 + }, + { + "epoch": 0.42, + "learning_rate": 3.2329317269076304e-07, + "logits/chosen": -2.8082878589630127, + "logits/rejected": -2.783658981323242, + "logps/chosen": -226.2019500732422, + "logps/rejected": -294.66326904296875, + "loss": 0.5152, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.869047999382019, + "rewards/margins": 1.269092321395874, + "rewards/rejected": -2.1381404399871826, + "step": 810 + }, + { + "epoch": 0.42, + "learning_rate": 3.204245553643144e-07, + "logits/chosen": -2.9127354621887207, + "logits/rejected": -2.894219160079956, + "logps/chosen": -230.79513549804688, + "logps/rejected": -305.60113525390625, + "loss": 0.5279, + "rewards/accuracies": 0.3499999940395355, + "rewards/chosen": -0.7637732028961182, + "rewards/margins": -0.17934365570545197, + "rewards/rejected": -0.5844296216964722, + "step": 820 + }, + { + "epoch": 0.43, + "learning_rate": 3.1755593803786574e-07, + "logits/chosen": -2.9059672355651855, + "logits/rejected": -2.9442248344421387, + "logps/chosen": -238.79248046875, + "logps/rejected": -285.3636474609375, + "loss": 0.5044, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.5089541673660278, + "rewards/margins": 1.2097771167755127, + "rewards/rejected": -1.718731164932251, + "step": 830 + }, + { + "epoch": 0.43, + "learning_rate": 3.146873207114171e-07, + "logits/chosen": -2.953200340270996, + "logits/rejected": -2.8728442192077637, + "logps/chosen": -345.51593017578125, + "logps/rejected": -307.09002685546875, + "loss": 0.5092, + "rewards/accuracies": 0.550000011920929, + "rewards/chosen": -1.007508397102356, + "rewards/margins": 0.5991470813751221, + "rewards/rejected": -1.6066553592681885, + "step": 840 + }, + { + "epoch": 0.44, + "learning_rate": 3.1181870338496843e-07, + "logits/chosen": -2.8444714546203613, + "logits/rejected": -2.791147470474243, + "logps/chosen": -296.8919677734375, + "logps/rejected": -274.53082275390625, + "loss": 0.4805, + "rewards/accuracies": 0.8999999761581421, + "rewards/chosen": -0.953098475933075, + "rewards/margins": 1.4887616634368896, + "rewards/rejected": -2.4418601989746094, + "step": 850 + }, + { + "epoch": 0.44, + "learning_rate": 3.089500860585198e-07, + "logits/chosen": -2.6704187393188477, + "logits/rejected": -2.6690378189086914, + "logps/chosen": -249.43881225585938, + "logps/rejected": -302.4588928222656, + "loss": 0.5111, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -1.3775177001953125, + "rewards/margins": 1.329897403717041, + "rewards/rejected": -2.7074151039123535, + "step": 860 + }, + { + "epoch": 0.45, + "learning_rate": 3.060814687320711e-07, + "logits/chosen": -2.839586019515991, + "logits/rejected": -2.834083318710327, + "logps/chosen": -304.1976318359375, + "logps/rejected": -256.19378662109375, + "loss": 0.4354, + "rewards/accuracies": 0.8999999761581421, + "rewards/chosen": -0.47585782408714294, + "rewards/margins": 2.021141529083252, + "rewards/rejected": -2.4969992637634277, + "step": 870 + }, + { + "epoch": 0.45, + "learning_rate": 3.0321285140562247e-07, + "logits/chosen": -2.7889621257781982, + "logits/rejected": -2.7733607292175293, + "logps/chosen": -280.20977783203125, + "logps/rejected": -347.7413635253906, + "loss": 0.4604, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.9127300977706909, + "rewards/margins": 0.9622424244880676, + "rewards/rejected": -1.8749723434448242, + "step": 880 + }, + { + "epoch": 0.46, + "learning_rate": 3.003442340791738e-07, + "logits/chosen": -2.904717206954956, + "logits/rejected": -2.835303783416748, + "logps/chosen": -336.64776611328125, + "logps/rejected": -316.78045654296875, + "loss": 0.5003, + "rewards/accuracies": 0.699999988079071, + "rewards/chosen": -0.2770691215991974, + "rewards/margins": 1.1059465408325195, + "rewards/rejected": -1.383015751838684, + "step": 890 + }, + { + "epoch": 0.46, + "learning_rate": 2.974756167527252e-07, + "logits/chosen": -3.022934913635254, + "logits/rejected": -2.956709146499634, + "logps/chosen": -286.189208984375, + "logps/rejected": -264.89776611328125, + "loss": 0.4656, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.7135367393493652, + "rewards/margins": 1.1268901824951172, + "rewards/rejected": -1.8404268026351929, + "step": 900 + }, + { + "epoch": 0.46, + "eval_logits/chosen": -2.8014752864837646, + "eval_logits/rejected": -2.781651258468628, + "eval_logps/chosen": -297.5553283691406, + "eval_logps/rejected": -323.15179443359375, + "eval_loss": 0.4927058219909668, + "eval_rewards/accuracies": 0.765625, + "eval_rewards/chosen": -0.5278611183166504, + "eval_rewards/margins": 1.1335095167160034, + "eval_rewards/rejected": -1.661370873451233, + "eval_runtime": 57.0998, + "eval_samples_per_second": 17.513, + "eval_steps_per_second": 0.28, + "step": 900 + }, + { + "epoch": 0.47, + "learning_rate": 2.946069994262765e-07, + "logits/chosen": -2.978717803955078, + "logits/rejected": -2.8918721675872803, + "logps/chosen": -336.64495849609375, + "logps/rejected": -247.2961883544922, + "loss": 0.4344, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.6280319094657898, + "rewards/margins": 1.4678236246109009, + "rewards/rejected": -2.095855712890625, + "step": 910 + }, + { + "epoch": 0.47, + "learning_rate": 2.9173838209982786e-07, + "logits/chosen": -2.786372423171997, + "logits/rejected": -2.8410544395446777, + "logps/chosen": -298.88360595703125, + "logps/rejected": -302.8692626953125, + "loss": 0.4686, + "rewards/accuracies": 0.6499999761581421, + "rewards/chosen": -0.6036109924316406, + "rewards/margins": 0.7661906480789185, + "rewards/rejected": -1.369801640510559, + "step": 920 + }, + { + "epoch": 0.48, + "learning_rate": 2.888697647733792e-07, + "logits/chosen": -2.915067195892334, + "logits/rejected": -2.732910633087158, + "logps/chosen": -356.8029479980469, + "logps/rejected": -226.57974243164062, + "loss": 0.4128, + "rewards/accuracies": 0.8999999761581421, + "rewards/chosen": -0.25983041524887085, + "rewards/margins": 2.1089906692504883, + "rewards/rejected": -2.368821144104004, + "step": 930 + }, + { + "epoch": 0.49, + "learning_rate": 2.8600114744693055e-07, + "logits/chosen": -2.872128486633301, + "logits/rejected": -2.8349528312683105, + "logps/chosen": -276.8409729003906, + "logps/rejected": -234.0821990966797, + "loss": 0.4633, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.33096224069595337, + "rewards/margins": 1.0401618480682373, + "rewards/rejected": -1.371124029159546, + "step": 940 + }, + { + "epoch": 0.49, + "learning_rate": 2.8313253012048195e-07, + "logits/chosen": -2.768357276916504, + "logits/rejected": -2.8650612831115723, + "logps/chosen": -286.5350341796875, + "logps/rejected": -295.7557678222656, + "loss": 0.4694, + "rewards/accuracies": 1.0, + "rewards/chosen": -0.42378902435302734, + "rewards/margins": 2.2293660640716553, + "rewards/rejected": -2.6531550884246826, + "step": 950 + }, + { + "epoch": 0.5, + "learning_rate": 2.802639127940333e-07, + "logits/chosen": -2.8344969749450684, + "logits/rejected": -2.8180437088012695, + "logps/chosen": -312.8253173828125, + "logps/rejected": -299.6221618652344, + "loss": 0.4998, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.2208956480026245, + "rewards/margins": 1.7145334482192993, + "rewards/rejected": -1.9354288578033447, + "step": 960 + }, + { + "epoch": 0.5, + "learning_rate": 2.773952954675846e-07, + "logits/chosen": -2.809140682220459, + "logits/rejected": -2.699197292327881, + "logps/chosen": -369.69854736328125, + "logps/rejected": -293.45477294921875, + "loss": 0.4849, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.28183019161224365, + "rewards/margins": 1.6505457162857056, + "rewards/rejected": -1.9323756694793701, + "step": 970 + }, + { + "epoch": 0.51, + "learning_rate": 2.7452667814113594e-07, + "logits/chosen": -2.8047561645507812, + "logits/rejected": -2.777113199234009, + "logps/chosen": -326.3648376464844, + "logps/rejected": -289.0523986816406, + "loss": 0.4543, + "rewards/accuracies": 0.949999988079071, + "rewards/chosen": -0.42743197083473206, + "rewards/margins": 1.8371082544326782, + "rewards/rejected": -2.264540195465088, + "step": 980 + }, + { + "epoch": 0.51, + "learning_rate": 2.716580608146873e-07, + "logits/chosen": -2.8819570541381836, + "logits/rejected": -2.8494954109191895, + "logps/chosen": -318.66204833984375, + "logps/rejected": -345.1630859375, + "loss": 0.4534, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.4583119750022888, + "rewards/margins": 1.3975725173950195, + "rewards/rejected": -1.8558847904205322, + "step": 990 + }, + { + "epoch": 0.52, + "learning_rate": 2.687894434882387e-07, + "logits/chosen": -2.8820109367370605, + "logits/rejected": -2.9153826236724854, + "logps/chosen": -251.1084747314453, + "logps/rejected": -264.5345153808594, + "loss": 0.4102, + "rewards/accuracies": 0.6499999761581421, + "rewards/chosen": -0.5280589461326599, + "rewards/margins": 1.0574977397918701, + "rewards/rejected": -1.5855568647384644, + "step": 1000 + }, + { + "epoch": 0.52, + "eval_logits/chosen": -2.745532512664795, + "eval_logits/rejected": -2.7159857749938965, + "eval_logps/chosen": -298.5311279296875, + "eval_logps/rejected": -331.2577819824219, + "eval_loss": 0.4771742522716522, + "eval_rewards/accuracies": 0.765625, + "eval_rewards/chosen": -0.5766510367393494, + "eval_rewards/margins": 1.4900195598602295, + "eval_rewards/rejected": -2.0666706562042236, + "eval_runtime": 60.3269, + "eval_samples_per_second": 16.576, + "eval_steps_per_second": 0.265, + "step": 1000 + }, + { + "epoch": 0.52, + "learning_rate": 2.6592082616179004e-07, + "logits/chosen": -2.753818988800049, + "logits/rejected": -2.6555442810058594, + "logps/chosen": -421.75408935546875, + "logps/rejected": -326.7854919433594, + "loss": 0.4969, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.7776181101799011, + "rewards/margins": 1.5239381790161133, + "rewards/rejected": -2.30155611038208, + "step": 1010 + }, + { + "epoch": 0.53, + "learning_rate": 2.6305220883534133e-07, + "logits/chosen": -2.795933961868286, + "logits/rejected": -2.875976085662842, + "logps/chosen": -359.01318359375, + "logps/rejected": -325.6358642578125, + "loss": 0.4814, + "rewards/accuracies": 0.699999988079071, + "rewards/chosen": -0.12685959041118622, + "rewards/margins": 1.6275579929351807, + "rewards/rejected": -1.7544174194335938, + "step": 1020 + }, + { + "epoch": 0.53, + "learning_rate": 2.601835915088927e-07, + "logits/chosen": -2.7979979515075684, + "logits/rejected": -2.7110390663146973, + "logps/chosen": -307.5633239746094, + "logps/rejected": -272.8973388671875, + "loss": 0.4354, + "rewards/accuracies": 0.8999999761581421, + "rewards/chosen": -0.4669399857521057, + "rewards/margins": 1.9002759456634521, + "rewards/rejected": -2.367216110229492, + "step": 1030 + }, + { + "epoch": 0.54, + "learning_rate": 2.573149741824441e-07, + "logits/chosen": -2.864424467086792, + "logits/rejected": -2.7874035835266113, + "logps/chosen": -301.7801513671875, + "logps/rejected": -366.8116149902344, + "loss": 0.4799, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.38910773396492004, + "rewards/margins": 2.2144553661346436, + "rewards/rejected": -2.6035633087158203, + "step": 1040 + }, + { + "epoch": 0.54, + "learning_rate": 2.544463568559954e-07, + "logits/chosen": -2.8506453037261963, + "logits/rejected": -2.7827541828155518, + "logps/chosen": -332.9029846191406, + "logps/rejected": -288.8146667480469, + "loss": 0.4728, + "rewards/accuracies": 0.75, + "rewards/chosen": -1.1103570461273193, + "rewards/margins": 1.3038640022277832, + "rewards/rejected": -2.4142210483551025, + "step": 1050 + }, + { + "epoch": 0.55, + "learning_rate": 2.5157773952954677e-07, + "logits/chosen": -2.746716022491455, + "logits/rejected": -2.738701105117798, + "logps/chosen": -362.2853698730469, + "logps/rejected": -263.8562316894531, + "loss": 0.5158, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.4159720540046692, + "rewards/margins": 1.3381980657577515, + "rewards/rejected": -1.7541700601577759, + "step": 1060 + }, + { + "epoch": 0.55, + "learning_rate": 2.4870912220309807e-07, + "logits/chosen": -2.7854630947113037, + "logits/rejected": -2.815742015838623, + "logps/chosen": -434.0101623535156, + "logps/rejected": -322.96990966796875, + "loss": 0.5047, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.38989701867103577, + "rewards/margins": 1.555858850479126, + "rewards/rejected": -1.9457557201385498, + "step": 1070 + }, + { + "epoch": 0.56, + "learning_rate": 2.4584050487664947e-07, + "logits/chosen": -2.9198765754699707, + "logits/rejected": -2.8616695404052734, + "logps/chosen": -290.16680908203125, + "logps/rejected": -285.9183349609375, + "loss": 0.4499, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.7213519811630249, + "rewards/margins": 1.0302088260650635, + "rewards/rejected": -1.751560926437378, + "step": 1080 + }, + { + "epoch": 0.56, + "learning_rate": 2.429718875502008e-07, + "logits/chosen": -2.9124810695648193, + "logits/rejected": -2.816594362258911, + "logps/chosen": -285.74139404296875, + "logps/rejected": -237.6147003173828, + "loss": 0.4673, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.599921464920044, + "rewards/margins": 1.2972980737686157, + "rewards/rejected": -1.8972194194793701, + "step": 1090 + }, + { + "epoch": 0.57, + "learning_rate": 2.4010327022375216e-07, + "logits/chosen": -2.8846845626831055, + "logits/rejected": -2.914504289627075, + "logps/chosen": -373.78887939453125, + "logps/rejected": -349.1485900878906, + "loss": 0.4663, + "rewards/accuracies": 0.8999999761581421, + "rewards/chosen": -0.4652225971221924, + "rewards/margins": 0.824032187461853, + "rewards/rejected": -1.289254903793335, + "step": 1100 + }, + { + "epoch": 0.57, + "eval_logits/chosen": -2.725741386413574, + "eval_logits/rejected": -2.6993935108184814, + "eval_logps/chosen": -303.0741271972656, + "eval_logps/rejected": -331.96044921875, + "eval_loss": 0.4740181565284729, + "eval_rewards/accuracies": 0.765625, + "eval_rewards/chosen": -0.803801417350769, + "eval_rewards/margins": 1.2980027198791504, + "eval_rewards/rejected": -2.10180401802063, + "eval_runtime": 57.8167, + "eval_samples_per_second": 17.296, + "eval_steps_per_second": 0.277, + "step": 1100 + }, + { + "epoch": 0.57, + "learning_rate": 2.3723465289730348e-07, + "logits/chosen": -2.9184155464172363, + "logits/rejected": -2.913644552230835, + "logps/chosen": -325.4616394042969, + "logps/rejected": -290.3975830078125, + "loss": 0.4519, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.7317468523979187, + "rewards/margins": 0.9042754173278809, + "rewards/rejected": -1.6360222101211548, + "step": 1110 + }, + { + "epoch": 0.58, + "learning_rate": 2.3436603557085483e-07, + "logits/chosen": -2.845045328140259, + "logits/rejected": -2.6523425579071045, + "logps/chosen": -370.0478820800781, + "logps/rejected": -286.46112060546875, + "loss": 0.5856, + "rewards/accuracies": 0.699999988079071, + "rewards/chosen": -0.8687052726745605, + "rewards/margins": 1.1778913736343384, + "rewards/rejected": -2.0465967655181885, + "step": 1120 + }, + { + "epoch": 0.58, + "learning_rate": 2.314974182444062e-07, + "logits/chosen": -2.8266937732696533, + "logits/rejected": -2.6822288036346436, + "logps/chosen": -350.9194641113281, + "logps/rejected": -341.3634033203125, + "loss": 0.4118, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -1.1032984256744385, + "rewards/margins": 1.7717195749282837, + "rewards/rejected": -2.8750178813934326, + "step": 1130 + }, + { + "epoch": 0.59, + "learning_rate": 2.2862880091795752e-07, + "logits/chosen": -2.71614146232605, + "logits/rejected": -2.7234530448913574, + "logps/chosen": -332.0517272949219, + "logps/rejected": -334.45977783203125, + "loss": 0.5011, + "rewards/accuracies": 0.75, + "rewards/chosen": -1.2248785495758057, + "rewards/margins": 1.2211261987686157, + "rewards/rejected": -2.4460043907165527, + "step": 1140 + }, + { + "epoch": 0.59, + "learning_rate": 2.257601835915089e-07, + "logits/chosen": -2.746645450592041, + "logits/rejected": -2.7202837467193604, + "logps/chosen": -285.22576904296875, + "logps/rejected": -271.9557800292969, + "loss": 0.4743, + "rewards/accuracies": 0.699999988079071, + "rewards/chosen": -0.9407691955566406, + "rewards/margins": 1.4142510890960693, + "rewards/rejected": -2.3550198078155518, + "step": 1150 + }, + { + "epoch": 0.6, + "learning_rate": 2.2289156626506022e-07, + "logits/chosen": -2.6213831901550293, + "logits/rejected": -2.5203697681427, + "logps/chosen": -377.27337646484375, + "logps/rejected": -397.376953125, + "loss": 0.4551, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.6566792726516724, + "rewards/margins": 1.3486577272415161, + "rewards/rejected": -2.0053369998931885, + "step": 1160 + }, + { + "epoch": 0.6, + "learning_rate": 2.200229489386116e-07, + "logits/chosen": -2.8519699573516846, + "logits/rejected": -2.9094433784484863, + "logps/chosen": -343.9521789550781, + "logps/rejected": -308.2449645996094, + "loss": 0.4565, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.4127805829048157, + "rewards/margins": 0.8223036527633667, + "rewards/rejected": -1.2350841760635376, + "step": 1170 + }, + { + "epoch": 0.61, + "learning_rate": 2.1715433161216294e-07, + "logits/chosen": -2.8739943504333496, + "logits/rejected": -2.8903346061706543, + "logps/chosen": -301.22967529296875, + "logps/rejected": -251.42855834960938, + "loss": 0.4426, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.35094451904296875, + "rewards/margins": 1.0077975988388062, + "rewards/rejected": -1.3587422370910645, + "step": 1180 + }, + { + "epoch": 0.61, + "learning_rate": 2.1428571428571426e-07, + "logits/chosen": -2.892425060272217, + "logits/rejected": -2.8500430583953857, + "logps/chosen": -328.62652587890625, + "logps/rejected": -307.386474609375, + "loss": 0.4722, + "rewards/accuracies": 0.699999988079071, + "rewards/chosen": -0.8684075474739075, + "rewards/margins": 1.0192383527755737, + "rewards/rejected": -1.887645959854126, + "step": 1190 + }, + { + "epoch": 0.62, + "learning_rate": 2.1141709695926563e-07, + "logits/chosen": -2.910890817642212, + "logits/rejected": -2.8874032497406006, + "logps/chosen": -265.87310791015625, + "logps/rejected": -210.3219451904297, + "loss": 0.4737, + "rewards/accuracies": 0.699999988079071, + "rewards/chosen": -0.5260100960731506, + "rewards/margins": 0.7490276098251343, + "rewards/rejected": -1.2750377655029297, + "step": 1200 + }, + { + "epoch": 0.62, + "eval_logits/chosen": -2.713484764099121, + "eval_logits/rejected": -2.6841983795166016, + "eval_logps/chosen": -294.5633850097656, + "eval_logps/rejected": -323.9544677734375, + "eval_loss": 0.47160398960113525, + "eval_rewards/accuracies": 0.796875, + "eval_rewards/chosen": -0.3782654404640198, + "eval_rewards/margins": 1.3232390880584717, + "eval_rewards/rejected": -1.7015043497085571, + "eval_runtime": 57.3455, + "eval_samples_per_second": 17.438, + "eval_steps_per_second": 0.279, + "step": 1200 + }, + { + "epoch": 0.62, + "learning_rate": 2.0854847963281698e-07, + "logits/chosen": -2.970987319946289, + "logits/rejected": -2.867969512939453, + "logps/chosen": -368.6965026855469, + "logps/rejected": -353.63604736328125, + "loss": 0.4535, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.4251924455165863, + "rewards/margins": 1.3024253845214844, + "rewards/rejected": -1.727617859840393, + "step": 1210 + }, + { + "epoch": 0.63, + "learning_rate": 2.0567986230636832e-07, + "logits/chosen": -2.964089870452881, + "logits/rejected": -2.933443784713745, + "logps/chosen": -293.6631164550781, + "logps/rejected": -330.43878173828125, + "loss": 0.4637, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.6487616300582886, + "rewards/margins": 1.531836986541748, + "rewards/rejected": -2.180598735809326, + "step": 1220 + }, + { + "epoch": 0.64, + "learning_rate": 2.0281124497991967e-07, + "logits/chosen": -2.9999537467956543, + "logits/rejected": -2.9158272743225098, + "logps/chosen": -342.1163330078125, + "logps/rejected": -396.15350341796875, + "loss": 0.4611, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.16167068481445312, + "rewards/margins": 1.715951919555664, + "rewards/rejected": -1.8776228427886963, + "step": 1230 + }, + { + "epoch": 0.64, + "learning_rate": 1.9994262765347102e-07, + "logits/chosen": -2.9242331981658936, + "logits/rejected": -2.841064929962158, + "logps/chosen": -337.76983642578125, + "logps/rejected": -282.3847351074219, + "loss": 0.4917, + "rewards/accuracies": 0.6499999761581421, + "rewards/chosen": -1.049020528793335, + "rewards/margins": 0.9485428929328918, + "rewards/rejected": -1.9975636005401611, + "step": 1240 + }, + { + "epoch": 0.65, + "learning_rate": 1.9707401032702237e-07, + "logits/chosen": -2.7807137966156006, + "logits/rejected": -2.775256872177124, + "logps/chosen": -282.36151123046875, + "logps/rejected": -339.3646545410156, + "loss": 0.4765, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.4363161623477936, + "rewards/margins": 1.3644092082977295, + "rewards/rejected": -1.8007253408432007, + "step": 1250 + }, + { + "epoch": 0.65, + "learning_rate": 1.942053930005737e-07, + "logits/chosen": -2.850567579269409, + "logits/rejected": -2.8491926193237305, + "logps/chosen": -355.41064453125, + "logps/rejected": -349.8357238769531, + "loss": 0.4739, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.5196908712387085, + "rewards/margins": 1.9353296756744385, + "rewards/rejected": -2.4550204277038574, + "step": 1260 + }, + { + "epoch": 0.66, + "learning_rate": 1.9133677567412506e-07, + "logits/chosen": -2.6965737342834473, + "logits/rejected": -2.7178375720977783, + "logps/chosen": -253.52798461914062, + "logps/rejected": -266.63330078125, + "loss": 0.4285, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.8542221784591675, + "rewards/margins": 1.580542802810669, + "rewards/rejected": -2.434765338897705, + "step": 1270 + }, + { + "epoch": 0.66, + "learning_rate": 1.884681583476764e-07, + "logits/chosen": -2.8671460151672363, + "logits/rejected": -2.8770196437835693, + "logps/chosen": -395.8647155761719, + "logps/rejected": -322.73565673828125, + "loss": 0.5017, + "rewards/accuracies": 0.699999988079071, + "rewards/chosen": -0.8471239805221558, + "rewards/margins": 0.8821918368339539, + "rewards/rejected": -1.729315996170044, + "step": 1280 + }, + { + "epoch": 0.67, + "learning_rate": 1.8559954102122778e-07, + "logits/chosen": -2.8296945095062256, + "logits/rejected": -2.8752951622009277, + "logps/chosen": -439.872314453125, + "logps/rejected": -365.8454284667969, + "loss": 0.4554, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.5656521320343018, + "rewards/margins": 1.5323896408081055, + "rewards/rejected": -2.0980417728424072, + "step": 1290 + }, + { + "epoch": 0.67, + "learning_rate": 1.827309236947791e-07, + "logits/chosen": -2.946869134902954, + "logits/rejected": -2.8591227531433105, + "logps/chosen": -316.06512451171875, + "logps/rejected": -311.39892578125, + "loss": 0.4259, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.757400631904602, + "rewards/margins": 1.6462675333023071, + "rewards/rejected": -2.40366792678833, + "step": 1300 + }, + { + "epoch": 0.67, + "eval_logits/chosen": -2.735586166381836, + "eval_logits/rejected": -2.7046470642089844, + "eval_logps/chosen": -299.4761047363281, + "eval_logps/rejected": -329.3312072753906, + "eval_loss": 0.486631840467453, + "eval_rewards/accuracies": 0.78125, + "eval_rewards/chosen": -0.6239006519317627, + "eval_rewards/margins": 1.346441626548767, + "eval_rewards/rejected": -1.9703422784805298, + "eval_runtime": 57.6013, + "eval_samples_per_second": 17.361, + "eval_steps_per_second": 0.278, + "step": 1300 + }, + { + "epoch": 0.68, + "learning_rate": 1.7986230636833047e-07, + "logits/chosen": -2.9283134937286377, + "logits/rejected": -2.797074556350708, + "logps/chosen": -277.39044189453125, + "logps/rejected": -298.46978759765625, + "loss": 0.4063, + "rewards/accuracies": 0.550000011920929, + "rewards/chosen": -0.8742395639419556, + "rewards/margins": 0.7818448543548584, + "rewards/rejected": -1.656084418296814, + "step": 1310 + }, + { + "epoch": 0.68, + "learning_rate": 1.769936890418818e-07, + "logits/chosen": -2.8751561641693115, + "logits/rejected": -2.8942153453826904, + "logps/chosen": -327.19158935546875, + "logps/rejected": -371.1598205566406, + "loss": 0.4662, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.4851578176021576, + "rewards/margins": 1.8855422735214233, + "rewards/rejected": -2.3707003593444824, + "step": 1320 + }, + { + "epoch": 0.69, + "learning_rate": 1.7412507171543314e-07, + "logits/chosen": -2.869723081588745, + "logits/rejected": -2.791024684906006, + "logps/chosen": -299.9261474609375, + "logps/rejected": -240.87722778320312, + "loss": 0.5043, + "rewards/accuracies": 0.6499999761581421, + "rewards/chosen": -1.252393364906311, + "rewards/margins": 0.8984333872795105, + "rewards/rejected": -2.1508266925811768, + "step": 1330 + }, + { + "epoch": 0.69, + "learning_rate": 1.7125645438898452e-07, + "logits/chosen": -2.8324294090270996, + "logits/rejected": -2.8103926181793213, + "logps/chosen": -360.5182800292969, + "logps/rejected": -295.58953857421875, + "loss": 0.4544, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.9667569994926453, + "rewards/margins": 0.8512316942214966, + "rewards/rejected": -1.8179887533187866, + "step": 1340 + }, + { + "epoch": 0.7, + "learning_rate": 1.6838783706253584e-07, + "logits/chosen": -2.848896026611328, + "logits/rejected": -2.7923531532287598, + "logps/chosen": -372.48175048828125, + "logps/rejected": -274.67169189453125, + "loss": 0.5062, + "rewards/accuracies": 0.699999988079071, + "rewards/chosen": -0.6907297372817993, + "rewards/margins": 1.647875189781189, + "rewards/rejected": -2.3386049270629883, + "step": 1350 + }, + { + "epoch": 0.7, + "learning_rate": 1.655192197360872e-07, + "logits/chosen": -3.0092597007751465, + "logits/rejected": -3.0115504264831543, + "logps/chosen": -414.8208923339844, + "logps/rejected": -411.04327392578125, + "loss": 0.5338, + "rewards/accuracies": 0.699999988079071, + "rewards/chosen": -0.2658500671386719, + "rewards/margins": 1.0859276056289673, + "rewards/rejected": -1.3517776727676392, + "step": 1360 + }, + { + "epoch": 0.71, + "learning_rate": 1.6265060240963853e-07, + "logits/chosen": -2.947781801223755, + "logits/rejected": -2.9552831649780273, + "logps/chosen": -383.8572082519531, + "logps/rejected": -299.138427734375, + "loss": 0.5202, + "rewards/accuracies": 0.699999988079071, + "rewards/chosen": -0.4735872149467468, + "rewards/margins": 1.4772822856903076, + "rewards/rejected": -1.9508693218231201, + "step": 1370 + }, + { + "epoch": 0.71, + "learning_rate": 1.597819850831899e-07, + "logits/chosen": -2.812499523162842, + "logits/rejected": -2.7452359199523926, + "logps/chosen": -339.11981201171875, + "logps/rejected": -291.80035400390625, + "loss": 0.513, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.7339299917221069, + "rewards/margins": 1.1406757831573486, + "rewards/rejected": -1.8746061325073242, + "step": 1380 + }, + { + "epoch": 0.72, + "learning_rate": 1.5691336775674125e-07, + "logits/chosen": -2.7743217945098877, + "logits/rejected": -2.7072505950927734, + "logps/chosen": -318.3100891113281, + "logps/rejected": -287.02581787109375, + "loss": 0.5028, + "rewards/accuracies": 0.699999988079071, + "rewards/chosen": -0.30296844244003296, + "rewards/margins": 1.0583657026290894, + "rewards/rejected": -1.3613340854644775, + "step": 1390 + }, + { + "epoch": 0.72, + "learning_rate": 1.5404475043029257e-07, + "logits/chosen": -2.930776834487915, + "logits/rejected": -2.942199468612671, + "logps/chosen": -239.4253387451172, + "logps/rejected": -327.1017150878906, + "loss": 0.4935, + "rewards/accuracies": 0.699999988079071, + "rewards/chosen": -0.503673255443573, + "rewards/margins": 0.9641939401626587, + "rewards/rejected": -1.4678672552108765, + "step": 1400 + }, + { + "epoch": 0.72, + "eval_logits/chosen": -2.7443745136260986, + "eval_logits/rejected": -2.71527099609375, + "eval_logps/chosen": -298.24908447265625, + "eval_logps/rejected": -325.1242980957031, + "eval_loss": 0.4747045338153839, + "eval_rewards/accuracies": 0.78125, + "eval_rewards/chosen": -0.5625503063201904, + "eval_rewards/margins": 1.1974470615386963, + "eval_rewards/rejected": -1.7599973678588867, + "eval_runtime": 59.3107, + "eval_samples_per_second": 16.86, + "eval_steps_per_second": 0.27, + "step": 1400 + }, + { + "epoch": 0.73, + "learning_rate": 1.5117613310384395e-07, + "logits/chosen": -2.92527437210083, + "logits/rejected": -3.0139377117156982, + "logps/chosen": -237.60580444335938, + "logps/rejected": -268.05389404296875, + "loss": 0.4175, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.6132401823997498, + "rewards/margins": 1.2935967445373535, + "rewards/rejected": -1.9068371057510376, + "step": 1410 + }, + { + "epoch": 0.73, + "learning_rate": 1.483075157773953e-07, + "logits/chosen": -2.7750067710876465, + "logits/rejected": -2.552576780319214, + "logps/chosen": -268.2431945800781, + "logps/rejected": -310.37359619140625, + "loss": 0.531, + "rewards/accuracies": 0.699999988079071, + "rewards/chosen": -0.5963152647018433, + "rewards/margins": 0.873083770275116, + "rewards/rejected": -1.469399094581604, + "step": 1420 + }, + { + "epoch": 0.74, + "learning_rate": 1.4543889845094664e-07, + "logits/chosen": -2.900043249130249, + "logits/rejected": -2.9251770973205566, + "logps/chosen": -294.197998046875, + "logps/rejected": -420.86981201171875, + "loss": 0.5, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.7220600247383118, + "rewards/margins": 1.1596812009811401, + "rewards/rejected": -1.8817412853240967, + "step": 1430 + }, + { + "epoch": 0.74, + "learning_rate": 1.42570281124498e-07, + "logits/chosen": -2.8284263610839844, + "logits/rejected": -2.861384630203247, + "logps/chosen": -330.3293762207031, + "logps/rejected": -352.83013916015625, + "loss": 0.5019, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.5237518548965454, + "rewards/margins": 1.484616756439209, + "rewards/rejected": -2.008368730545044, + "step": 1440 + }, + { + "epoch": 0.75, + "learning_rate": 1.3970166379804933e-07, + "logits/chosen": -2.9035301208496094, + "logits/rejected": -2.877016544342041, + "logps/chosen": -330.01885986328125, + "logps/rejected": -309.932373046875, + "loss": 0.4207, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.6725677847862244, + "rewards/margins": 1.2416926622390747, + "rewards/rejected": -1.9142605066299438, + "step": 1450 + }, + { + "epoch": 0.75, + "learning_rate": 1.3683304647160068e-07, + "logits/chosen": -2.8522450923919678, + "logits/rejected": -2.804635763168335, + "logps/chosen": -315.32598876953125, + "logps/rejected": -319.85528564453125, + "loss": 0.484, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.6710207462310791, + "rewards/margins": 2.302441120147705, + "rewards/rejected": -2.973461627960205, + "step": 1460 + }, + { + "epoch": 0.76, + "learning_rate": 1.3396442914515203e-07, + "logits/chosen": -2.795161008834839, + "logits/rejected": -2.835954189300537, + "logps/chosen": -301.64306640625, + "logps/rejected": -345.019287109375, + "loss": 0.5058, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.31656789779663086, + "rewards/margins": 1.5656077861785889, + "rewards/rejected": -1.8821756839752197, + "step": 1470 + }, + { + "epoch": 0.76, + "learning_rate": 1.3109581181870338e-07, + "logits/chosen": -2.8382556438446045, + "logits/rejected": -2.7719860076904297, + "logps/chosen": -416.19622802734375, + "logps/rejected": -278.07989501953125, + "loss": 0.3959, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.7636359930038452, + "rewards/margins": 2.003495216369629, + "rewards/rejected": -2.7671313285827637, + "step": 1480 + }, + { + "epoch": 0.77, + "learning_rate": 1.2822719449225472e-07, + "logits/chosen": -2.7587904930114746, + "logits/rejected": -2.747542381286621, + "logps/chosen": -255.3273468017578, + "logps/rejected": -239.77218627929688, + "loss": 0.4899, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.0010371461976319551, + "rewards/margins": 2.0150508880615234, + "rewards/rejected": -2.0160880088806152, + "step": 1490 + }, + { + "epoch": 0.77, + "learning_rate": 1.253585771658061e-07, + "logits/chosen": -2.7033538818359375, + "logits/rejected": -2.7172279357910156, + "logps/chosen": -317.5029296875, + "logps/rejected": -332.539794921875, + "loss": 0.4211, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.8115976452827454, + "rewards/margins": 1.1709970235824585, + "rewards/rejected": -1.9825948476791382, + "step": 1500 + }, + { + "epoch": 0.77, + "eval_logits/chosen": -2.7236168384552, + "eval_logits/rejected": -2.6944026947021484, + "eval_logps/chosen": -299.1958923339844, + "eval_logps/rejected": -329.9108581542969, + "eval_loss": 0.4644867181777954, + "eval_rewards/accuracies": 0.765625, + "eval_rewards/chosen": -0.6098894476890564, + "eval_rewards/margins": 1.389434814453125, + "eval_rewards/rejected": -1.9993242025375366, + "eval_runtime": 57.1288, + "eval_samples_per_second": 17.504, + "eval_steps_per_second": 0.28, + "step": 1500 + }, + { + "epoch": 0.78, + "learning_rate": 1.2248995983935742e-07, + "logits/chosen": -2.7994678020477295, + "logits/rejected": -2.593963384628296, + "logps/chosen": -300.65960693359375, + "logps/rejected": -338.9661560058594, + "loss": 0.4655, + "rewards/accuracies": 0.8999999761581421, + "rewards/chosen": -0.44014763832092285, + "rewards/margins": 2.050450325012207, + "rewards/rejected": -2.49059796333313, + "step": 1510 + }, + { + "epoch": 0.78, + "learning_rate": 1.1962134251290876e-07, + "logits/chosen": -2.8634955883026123, + "logits/rejected": -2.9231014251708984, + "logps/chosen": -350.2730407714844, + "logps/rejected": -394.0359191894531, + "loss": 0.4687, + "rewards/accuracies": 0.6000000238418579, + "rewards/chosen": -0.7544436454772949, + "rewards/margins": 0.6226231455802917, + "rewards/rejected": -1.377066731452942, + "step": 1520 + }, + { + "epoch": 0.79, + "learning_rate": 1.1675272518646012e-07, + "logits/chosen": -2.714735984802246, + "logits/rejected": -2.6250107288360596, + "logps/chosen": -271.64691162109375, + "logps/rejected": -272.4801330566406, + "loss": 0.3846, + "rewards/accuracies": 0.8999999761581421, + "rewards/chosen": 0.06765889376401901, + "rewards/margins": 1.8679250478744507, + "rewards/rejected": -1.800265908241272, + "step": 1530 + }, + { + "epoch": 0.8, + "learning_rate": 1.1388410786001147e-07, + "logits/chosen": -2.952043294906616, + "logits/rejected": -2.9354848861694336, + "logps/chosen": -237.10208129882812, + "logps/rejected": -248.58023071289062, + "loss": 0.4791, + "rewards/accuracies": 0.5, + "rewards/chosen": -0.8253666162490845, + "rewards/margins": 0.6920875906944275, + "rewards/rejected": -1.5174543857574463, + "step": 1540 + }, + { + "epoch": 0.8, + "learning_rate": 1.1101549053356282e-07, + "logits/chosen": -2.839200735092163, + "logits/rejected": -2.827392101287842, + "logps/chosen": -347.25390625, + "logps/rejected": -284.7652893066406, + "loss": 0.4351, + "rewards/accuracies": 0.8999999761581421, + "rewards/chosen": -0.3293851315975189, + "rewards/margins": 2.0348496437072754, + "rewards/rejected": -2.3642349243164062, + "step": 1550 + }, + { + "epoch": 0.81, + "learning_rate": 1.0814687320711418e-07, + "logits/chosen": -2.853760242462158, + "logits/rejected": -2.728318214416504, + "logps/chosen": -224.41683959960938, + "logps/rejected": -228.2812042236328, + "loss": 0.4528, + "rewards/accuracies": 0.8999999761581421, + "rewards/chosen": -0.3215061128139496, + "rewards/margins": 1.6524702310562134, + "rewards/rejected": -1.9739763736724854, + "step": 1560 + }, + { + "epoch": 0.81, + "learning_rate": 1.0527825588066551e-07, + "logits/chosen": -2.870434284210205, + "logits/rejected": -2.793577194213867, + "logps/chosen": -282.13787841796875, + "logps/rejected": -271.94915771484375, + "loss": 0.4144, + "rewards/accuracies": 0.949999988079071, + "rewards/chosen": -0.5694266557693481, + "rewards/margins": 1.3067474365234375, + "rewards/rejected": -1.8761742115020752, + "step": 1570 + }, + { + "epoch": 0.82, + "learning_rate": 1.0240963855421686e-07, + "logits/chosen": -2.73341703414917, + "logits/rejected": -2.7930026054382324, + "logps/chosen": -214.6037139892578, + "logps/rejected": -285.880859375, + "loss": 0.4287, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.6492952108383179, + "rewards/margins": 1.3862565755844116, + "rewards/rejected": -2.0355517864227295, + "step": 1580 + }, + { + "epoch": 0.82, + "learning_rate": 9.95410212277682e-08, + "logits/chosen": -2.885023593902588, + "logits/rejected": -2.929999351501465, + "logps/chosen": -294.29364013671875, + "logps/rejected": -273.53729248046875, + "loss": 0.4548, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.3889610171318054, + "rewards/margins": 1.3147281408309937, + "rewards/rejected": -1.7036889791488647, + "step": 1590 + }, + { + "epoch": 0.83, + "learning_rate": 9.667240390131957e-08, + "logits/chosen": -2.8551316261291504, + "logits/rejected": -2.913240432739258, + "logps/chosen": -301.7610168457031, + "logps/rejected": -309.9018249511719, + "loss": 0.4931, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.7070005536079407, + "rewards/margins": 1.0328757762908936, + "rewards/rejected": -1.7398761510849, + "step": 1600 + }, + { + "epoch": 0.83, + "eval_logits/chosen": -2.7304749488830566, + "eval_logits/rejected": -2.7005884647369385, + "eval_logps/chosen": -300.5933532714844, + "eval_logps/rejected": -332.0889892578125, + "eval_loss": 0.4683745205402374, + "eval_rewards/accuracies": 0.765625, + "eval_rewards/chosen": -0.6797637343406677, + "eval_rewards/margins": 1.4284672737121582, + "eval_rewards/rejected": -2.1082310676574707, + "eval_runtime": 58.1263, + "eval_samples_per_second": 17.204, + "eval_steps_per_second": 0.275, + "step": 1600 + }, + { + "epoch": 0.83, + "learning_rate": 9.380378657487091e-08, + "logits/chosen": -2.786533832550049, + "logits/rejected": -2.8000881671905518, + "logps/chosen": -252.6224822998047, + "logps/rejected": -307.4824523925781, + "loss": 0.4282, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.869725227355957, + "rewards/margins": 1.490140438079834, + "rewards/rejected": -2.359865665435791, + "step": 1610 + }, + { + "epoch": 0.84, + "learning_rate": 9.093516924842226e-08, + "logits/chosen": -2.9314494132995605, + "logits/rejected": -2.888124942779541, + "logps/chosen": -322.18988037109375, + "logps/rejected": -340.00958251953125, + "loss": 0.4095, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.7641150951385498, + "rewards/margins": 1.8909038305282593, + "rewards/rejected": -2.6550190448760986, + "step": 1620 + }, + { + "epoch": 0.84, + "learning_rate": 8.806655192197361e-08, + "logits/chosen": -2.91792368888855, + "logits/rejected": -2.8502402305603027, + "logps/chosen": -327.46807861328125, + "logps/rejected": -310.23199462890625, + "loss": 0.4625, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.5610017776489258, + "rewards/margins": 1.5278574228286743, + "rewards/rejected": -2.0888590812683105, + "step": 1630 + }, + { + "epoch": 0.85, + "learning_rate": 8.519793459552494e-08, + "logits/chosen": -2.8847482204437256, + "logits/rejected": -2.8378400802612305, + "logps/chosen": -315.1554870605469, + "logps/rejected": -282.12652587890625, + "loss": 0.418, + "rewards/accuracies": 0.8999999761581421, + "rewards/chosen": -0.6100664138793945, + "rewards/margins": 1.6232362985610962, + "rewards/rejected": -2.233302593231201, + "step": 1640 + }, + { + "epoch": 0.85, + "learning_rate": 8.23293172690763e-08, + "logits/chosen": -2.943779230117798, + "logits/rejected": -2.894235372543335, + "logps/chosen": -380.72894287109375, + "logps/rejected": -279.9917907714844, + "loss": 0.4489, + "rewards/accuracies": 0.8999999761581421, + "rewards/chosen": 0.3063332736492157, + "rewards/margins": 2.2862744331359863, + "rewards/rejected": -1.9799413681030273, + "step": 1650 + }, + { + "epoch": 0.86, + "learning_rate": 7.946069994262765e-08, + "logits/chosen": -2.738778829574585, + "logits/rejected": -2.805850028991699, + "logps/chosen": -221.29507446289062, + "logps/rejected": -314.97076416015625, + "loss": 0.4268, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.5129801034927368, + "rewards/margins": 1.8285270929336548, + "rewards/rejected": -2.3415069580078125, + "step": 1660 + }, + { + "epoch": 0.86, + "learning_rate": 7.6592082616179e-08, + "logits/chosen": -2.73237681388855, + "logits/rejected": -2.7697629928588867, + "logps/chosen": -295.9866638183594, + "logps/rejected": -282.50189208984375, + "loss": 0.5111, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.5220045447349548, + "rewards/margins": 1.3860810995101929, + "rewards/rejected": -1.908085823059082, + "step": 1670 + }, + { + "epoch": 0.87, + "learning_rate": 7.372346528973034e-08, + "logits/chosen": -2.9239017963409424, + "logits/rejected": -2.8694822788238525, + "logps/chosen": -266.6614074707031, + "logps/rejected": -303.3201599121094, + "loss": 0.5028, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.5870487689971924, + "rewards/margins": 0.9326974749565125, + "rewards/rejected": -1.5197464227676392, + "step": 1680 + }, + { + "epoch": 0.87, + "learning_rate": 7.08548479632817e-08, + "logits/chosen": -2.8852458000183105, + "logits/rejected": -2.9049694538116455, + "logps/chosen": -256.247802734375, + "logps/rejected": -329.19305419921875, + "loss": 0.4537, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.6385833024978638, + "rewards/margins": 1.4285590648651123, + "rewards/rejected": -2.0671424865722656, + "step": 1690 + }, + { + "epoch": 0.88, + "learning_rate": 6.798623063683305e-08, + "logits/chosen": -2.8753390312194824, + "logits/rejected": -2.7723491191864014, + "logps/chosen": -285.18365478515625, + "logps/rejected": -267.1637878417969, + "loss": 0.5029, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.7313726544380188, + "rewards/margins": 1.1651675701141357, + "rewards/rejected": -1.8965400457382202, + "step": 1700 + }, + { + "epoch": 0.88, + "eval_logits/chosen": -2.740252733230591, + "eval_logits/rejected": -2.710822343826294, + "eval_logps/chosen": -297.1233215332031, + "eval_logps/rejected": -327.8266906738281, + "eval_loss": 0.4595467746257782, + "eval_rewards/accuracies": 0.78125, + "eval_rewards/chosen": -0.5062620639801025, + "eval_rewards/margins": 1.3888535499572754, + "eval_rewards/rejected": -1.895115852355957, + "eval_runtime": 55.2023, + "eval_samples_per_second": 18.115, + "eval_steps_per_second": 0.29, + "step": 1700 + }, + { + "epoch": 0.88, + "learning_rate": 6.511761331038438e-08, + "logits/chosen": -2.895554542541504, + "logits/rejected": -2.7735402584075928, + "logps/chosen": -375.2083435058594, + "logps/rejected": -386.50689697265625, + "loss": 0.4635, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.7999193072319031, + "rewards/margins": 1.7455612421035767, + "rewards/rejected": -2.545480728149414, + "step": 1710 + }, + { + "epoch": 0.89, + "learning_rate": 6.224899598393573e-08, + "logits/chosen": -2.897815704345703, + "logits/rejected": -2.833980083465576, + "logps/chosen": -349.2824401855469, + "logps/rejected": -320.26080322265625, + "loss": 0.5059, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.8347634077072144, + "rewards/margins": 1.1928595304489136, + "rewards/rejected": -2.027622938156128, + "step": 1720 + }, + { + "epoch": 0.89, + "learning_rate": 5.9380378657487085e-08, + "logits/chosen": -2.747729539871216, + "logits/rejected": -2.749624490737915, + "logps/chosen": -389.26104736328125, + "logps/rejected": -379.7428283691406, + "loss": 0.4561, + "rewards/accuracies": 0.550000011920929, + "rewards/chosen": -0.46041926741600037, + "rewards/margins": 1.0110886096954346, + "rewards/rejected": -1.4715079069137573, + "step": 1730 + }, + { + "epoch": 0.9, + "learning_rate": 5.651176133103844e-08, + "logits/chosen": -2.9265472888946533, + "logits/rejected": -2.7790768146514893, + "logps/chosen": -344.9275817871094, + "logps/rejected": -267.4143371582031, + "loss": 0.4472, + "rewards/accuracies": 0.949999988079071, + "rewards/chosen": -0.3255007266998291, + "rewards/margins": 2.0611414909362793, + "rewards/rejected": -2.3866424560546875, + "step": 1740 + }, + { + "epoch": 0.9, + "learning_rate": 5.3643144004589786e-08, + "logits/chosen": -2.8731534481048584, + "logits/rejected": -2.7964863777160645, + "logps/chosen": -293.29681396484375, + "logps/rejected": -305.7369079589844, + "loss": 0.4481, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.3717958629131317, + "rewards/margins": 1.912921667098999, + "rewards/rejected": -2.2847177982330322, + "step": 1750 + }, + { + "epoch": 0.91, + "learning_rate": 5.077452667814113e-08, + "logits/chosen": -2.771488666534424, + "logits/rejected": -2.865757465362549, + "logps/chosen": -309.177978515625, + "logps/rejected": -285.9333801269531, + "loss": 0.4892, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.7683748602867126, + "rewards/margins": 1.0694787502288818, + "rewards/rejected": -1.8378536701202393, + "step": 1760 + }, + { + "epoch": 0.91, + "learning_rate": 4.790590935169248e-08, + "logits/chosen": -2.8640894889831543, + "logits/rejected": -2.8574938774108887, + "logps/chosen": -270.6001281738281, + "logps/rejected": -282.6536865234375, + "loss": 0.4509, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.5418082475662231, + "rewards/margins": 1.5019443035125732, + "rewards/rejected": -2.043752670288086, + "step": 1770 + }, + { + "epoch": 0.92, + "learning_rate": 4.5037292025243834e-08, + "logits/chosen": -2.916822910308838, + "logits/rejected": -2.945218086242676, + "logps/chosen": -330.62493896484375, + "logps/rejected": -340.20599365234375, + "loss": 0.4988, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.980034351348877, + "rewards/margins": 1.0700609683990479, + "rewards/rejected": -2.0500950813293457, + "step": 1780 + }, + { + "epoch": 0.92, + "learning_rate": 4.2168674698795174e-08, + "logits/chosen": -2.9247031211853027, + "logits/rejected": -2.9124653339385986, + "logps/chosen": -289.2491149902344, + "logps/rejected": -264.84283447265625, + "loss": 0.4577, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.3735916018486023, + "rewards/margins": 0.9222102165222168, + "rewards/rejected": -1.2958018779754639, + "step": 1790 + }, + { + "epoch": 0.93, + "learning_rate": 3.930005737234653e-08, + "logits/chosen": -2.9057695865631104, + "logits/rejected": -2.819000005722046, + "logps/chosen": -337.3312683105469, + "logps/rejected": -276.67010498046875, + "loss": 0.4965, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.23481640219688416, + "rewards/margins": 1.076974868774414, + "rewards/rejected": -1.3117913007736206, + "step": 1800 + }, + { + "epoch": 0.93, + "eval_logits/chosen": -2.752323627471924, + "eval_logits/rejected": -2.7225875854492188, + "eval_logps/chosen": -298.1203308105469, + "eval_logps/rejected": -328.0830993652344, + "eval_loss": 0.46130359172821045, + "eval_rewards/accuracies": 0.78125, + "eval_rewards/chosen": -0.5561107397079468, + "eval_rewards/margins": 1.3518264293670654, + "eval_rewards/rejected": -1.9079371690750122, + "eval_runtime": 57.0782, + "eval_samples_per_second": 17.52, + "eval_steps_per_second": 0.28, + "step": 1800 + }, + { + "epoch": 0.93, + "learning_rate": 3.6431440045897875e-08, + "logits/chosen": -2.8899993896484375, + "logits/rejected": -2.886475086212158, + "logps/chosen": -212.6807861328125, + "logps/rejected": -243.3516082763672, + "loss": 0.4966, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.7232971787452698, + "rewards/margins": 1.2528116703033447, + "rewards/rejected": -1.9761087894439697, + "step": 1810 + }, + { + "epoch": 0.94, + "learning_rate": 3.356282271944923e-08, + "logits/chosen": -2.936781406402588, + "logits/rejected": -2.913865566253662, + "logps/chosen": -259.8666076660156, + "logps/rejected": -268.0596008300781, + "loss": 0.4455, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.7119141221046448, + "rewards/margins": 1.462996244430542, + "rewards/rejected": -2.174910306930542, + "step": 1820 + }, + { + "epoch": 0.94, + "learning_rate": 3.0694205393000576e-08, + "logits/chosen": -2.8601667881011963, + "logits/rejected": -2.8055875301361084, + "logps/chosen": -324.9699401855469, + "logps/rejected": -261.33355712890625, + "loss": 0.4484, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.6698704361915588, + "rewards/margins": 1.4717018604278564, + "rewards/rejected": -2.1415722370147705, + "step": 1830 + }, + { + "epoch": 0.95, + "learning_rate": 2.782558806655192e-08, + "logits/chosen": -2.9527735710144043, + "logits/rejected": -2.9381892681121826, + "logps/chosen": -338.42889404296875, + "logps/rejected": -293.7242126464844, + "loss": 0.4437, + "rewards/accuracies": 0.6000000238418579, + "rewards/chosen": -0.5508059859275818, + "rewards/margins": 0.5796544551849365, + "rewards/rejected": -1.130460500717163, + "step": 1840 + }, + { + "epoch": 0.96, + "learning_rate": 2.4956970740103267e-08, + "logits/chosen": -2.8355095386505127, + "logits/rejected": -2.847282648086548, + "logps/chosen": -303.50482177734375, + "logps/rejected": -281.2980041503906, + "loss": 0.4709, + "rewards/accuracies": 0.949999988079071, + "rewards/chosen": -0.2886872887611389, + "rewards/margins": 2.1929566860198975, + "rewards/rejected": -2.4816439151763916, + "step": 1850 + }, + { + "epoch": 0.96, + "learning_rate": 2.2088353413654617e-08, + "logits/chosen": -2.8765408992767334, + "logits/rejected": -2.8786520957946777, + "logps/chosen": -321.27203369140625, + "logps/rejected": -308.62628173828125, + "loss": 0.4371, + "rewards/accuracies": 0.800000011920929, + "rewards/chosen": -0.6084128618240356, + "rewards/margins": 1.1289150714874268, + "rewards/rejected": -1.7373279333114624, + "step": 1860 + }, + { + "epoch": 0.97, + "learning_rate": 1.9219736087205964e-08, + "logits/chosen": -2.8658809661865234, + "logits/rejected": -2.86934232711792, + "logps/chosen": -271.0024719238281, + "logps/rejected": -249.09765625, + "loss": 0.4678, + "rewards/accuracies": 0.6499999761581421, + "rewards/chosen": -0.7377957105636597, + "rewards/margins": 0.9156107902526855, + "rewards/rejected": -1.6534065008163452, + "step": 1870 + }, + { + "epoch": 0.97, + "learning_rate": 1.6351118760757314e-08, + "logits/chosen": -2.7913639545440674, + "logits/rejected": -2.7344272136688232, + "logps/chosen": -299.87823486328125, + "logps/rejected": -340.60467529296875, + "loss": 0.4679, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.9796409606933594, + "rewards/margins": 1.4006431102752686, + "rewards/rejected": -2.380284070968628, + "step": 1880 + }, + { + "epoch": 0.98, + "learning_rate": 1.3482501434308661e-08, + "logits/chosen": -2.9909162521362305, + "logits/rejected": -2.8788750171661377, + "logps/chosen": -316.1296081542969, + "logps/rejected": -329.0151062011719, + "loss": 0.4584, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.6042786836624146, + "rewards/margins": 1.349906325340271, + "rewards/rejected": -1.954184889793396, + "step": 1890 + }, + { + "epoch": 0.98, + "learning_rate": 1.061388410786001e-08, + "logits/chosen": -2.840143918991089, + "logits/rejected": -2.8358423709869385, + "logps/chosen": -347.96185302734375, + "logps/rejected": -336.83831787109375, + "loss": 0.4337, + "rewards/accuracies": 0.75, + "rewards/chosen": -0.25721627473831177, + "rewards/margins": 1.7318767309188843, + "rewards/rejected": -1.9890931844711304, + "step": 1900 + }, + { + "epoch": 0.98, + "eval_logits/chosen": -2.746859073638916, + "eval_logits/rejected": -2.717475175857544, + "eval_logps/chosen": -297.12957763671875, + "eval_logps/rejected": -327.35992431640625, + "eval_loss": 0.4607921838760376, + "eval_rewards/accuracies": 0.765625, + "eval_rewards/chosen": -0.5065746307373047, + "eval_rewards/margins": 1.365204095840454, + "eval_rewards/rejected": -1.871778964996338, + "eval_runtime": 59.1698, + "eval_samples_per_second": 16.901, + "eval_steps_per_second": 0.27, + "step": 1900 + }, + { + "epoch": 0.99, + "learning_rate": 7.745266781411359e-09, + "logits/chosen": -2.944617986679077, + "logits/rejected": -2.859666347503662, + "logps/chosen": -312.51654052734375, + "logps/rejected": -263.56494140625, + "loss": 0.4129, + "rewards/accuracies": 0.550000011920929, + "rewards/chosen": -0.29929789900779724, + "rewards/margins": 0.7224828004837036, + "rewards/rejected": -1.0217806100845337, + "step": 1910 + }, + { + "epoch": 0.99, + "learning_rate": 4.8766494549627085e-09, + "logits/chosen": -2.8194212913513184, + "logits/rejected": -2.776564598083496, + "logps/chosen": -363.00396728515625, + "logps/rejected": -299.03338623046875, + "loss": 0.4647, + "rewards/accuracies": 0.550000011920929, + "rewards/chosen": -0.38584208488464355, + "rewards/margins": 0.8067569732666016, + "rewards/rejected": -1.1925990581512451, + "step": 1920 + }, + { + "epoch": 1.0, + "learning_rate": 2.008032128514056e-09, + "logits/chosen": -2.8939270973205566, + "logits/rejected": -2.778672933578491, + "logps/chosen": -323.1568298339844, + "logps/rejected": -284.89361572265625, + "loss": 0.4376, + "rewards/accuracies": 0.8500000238418579, + "rewards/chosen": -0.581506609916687, + "rewards/margins": 1.8644784688949585, + "rewards/rejected": -2.4459850788116455, + "step": 1930 + }, + { + "epoch": 1.0, + "step": 1937, + "total_flos": 0.0, + "train_loss": 0.48803097129668166, + "train_runtime": 7971.1784, + "train_samples_per_second": 7.774, + "train_steps_per_second": 0.243 + } + ], + "logging_steps": 10, + "max_steps": 1937, + "num_train_epochs": 1, + "save_steps": 500, + "total_flos": 0.0, + "trial_name": null, + "trial_params": null +} diff --git a/training_args.bin b/training_args.bin new file mode 100644 index 0000000000000000000000000000000000000000..53eade4f1b283e9165b1b04fddfc50ceba1d6010 --- /dev/null +++ b/training_args.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:64e623b49514a1cd5515d049be8d0aff5d04ae5562289d80ac5c9059e9bef51e +size 5435 diff --git a/zephyr-7b-alpha/.gitattributes b/zephyr-7b-alpha/.gitattributes new file mode 100644 index 0000000000000000000000000000000000000000..a6344aac8c09253b3b630fb776ae94478aa0275b --- /dev/null +++ b/zephyr-7b-alpha/.gitattributes @@ -0,0 +1,35 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text