
# Baize v2 13b GPTQ

GPTQ 4bit quantized version of [Baize V2 13b](https://huggingface.co/project-baize/baize-v2-13b) by [Project Baize](https://huggingface.co/project-baize).
<br>Please check the original repo for more information about this model!

This model was quantized using [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa) with the settings:
  ```
  python llama.py ./input/baize-v2-13b c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors ./output/baize-v2-13b-4bit-128g.safetensors
  ```

<hr>

## Available Sizes:

  * [Baize V2 7b GPTQ](https://huggingface.co/GamaTech/baize-v2-7b-GPTQ)
  * [Baize V2 13b GPTQ](https://huggingface.co/GamaTech/baize-v2-13b-GPTQ)

<hr>

## Usage with [Oobabooga's](https://github.com/oobabooga) [Text Generation WebUI](https://github.com/oobabooga/text-generation-webui):

1. In the **Model** tab, enter "`GamaTech/baize-v2-13b-GPTQ`" and click **Download**.
2. Once the download is complete, click the **Refresh** button next to the **Model drop-down** in the top left.
3. Uncheck the **Autoload the model** checkbox. This allows you to select the correct settings before attempting to load the model.
4. Select the model from the **Model drop-down** list.
5. On the right, set the following settings:
   * Wbits = 4
   * Groupsize = 128
   * Model_Type = LLaMA
7. Click **Save settings for this model**.
8. Click **Load Model**.

<hr>
<center>license: cc-by-nc-4.0</center>

{"license": "cc-by-nc-4.0", "inference": false}
transformers llama text-generation license:cc-by-nc-4.0 autotrain_compatible text-generation-inference region:us
TAGS
#transformers #llama #text-generation #license-cc-by-nc-4.0 #autotrain_compatible #text-generation-inference #region-us 


# Baize v2 13b GPTQ

GPTQ 4bit quantized version of Baize V2 13b by Project Baize.
<br>Please check the original repo for more information about this model!

This model was quantized using GPTQ-for-LLaMa with the settings:
  

<hr>

## Available Sizes:

  * Baize V2 7b GPTQ
  * Baize V2 13b GPTQ

<hr>

## Usage with Oobabooga's Text Generation WebUI:

1. In the Model tab, enter "'GamaTech/baize-v2-13b-GPTQ'" and click Download.
2. Once the download is complete, click the Refresh button next to the Model drop-down in the top left.
3. Uncheck the Autoload the model checkbox. This allows you to select the correct settings before attempting to load the model.
4. Select the model from the Model drop-down list.
5. On the right, set the following settings:
   * Wbits = 4
   * Groupsize = 128
   * Model_Type = LLaMA
7. Click Save settings for this model.
8. Click Load Model.

<hr>
<center>license: cc-by-nc-4.0</center>

# Baize v2 13b GPTQ

GPTQ 4bit quantized version of Baize V2 13b by Project Baize.
<br>Please check the original repo for more information about this model!

This model was quantized using GPTQ-for-LLaMa with the settings:
  

<hr> ## Available Sizes:

  * Baize V2 7b GPTQ
  * Baize V2 13b GPTQ

<hr> ## Usage with Oobabooga's Text Generation WebUI:

1. In the Model tab, enter "'GamaTech/baize-v2-13b-GPTQ'" and click Download.
2. Once the download is complete, click the Refresh button next to the Model drop-down in the top left.
3. Uncheck the Autoload the model checkbox. This allows you to select the correct settings before attempting to load the model.
4. Select the model from the Model drop-down list.
5. On the right, set the following settings:
   * Wbits = 4
   * Groupsize = 128
   * Model_Type = LLaMA
7. Click Save settings for this model.
8. Click Load Model.

<hr>
<center>license: cc-by-nc-4.0</center>
TAGS
#transformers #llama #text-generation #license-cc-by-nc-4.0 #autotrain_compatible #text-generation-inference #region-us 
 # Baize v2 13b GPTQ

GPTQ 4bit quantized version of Baize V2 13b by Project Baize.
<br>Please check the original repo for more information about this model!

This model was quantized using GPTQ-for-LLaMa with the settings:
  

<hr> ## Available Sizes:

  * Baize V2 7b GPTQ
  * Baize V2 13b GPTQ

<hr> ## Usage with Oobabooga's Text Generation WebUI:

1. In the Model tab, enter "'GamaTech/baize-v2-13b-GPTQ'" and click Download.
2. Once the download is complete, click the Refresh button next to the Model drop-down in the top left.
3. Uncheck the Autoload the model checkbox. This allows you to select the correct settings before attempting to load the model.
4. Select the model from the Model drop-down list.
5. On the right, set the following settings:
   * Wbits = 4
   * Groupsize = 128
   * Model_Type = LLaMA
7. Click Save settings for this model.
8. Click Load Model.

<hr>
<center>license: cc-by-nc-4.0</center>