README.md

---
language:
- en
base_model: defog/llama-3-sqlcoder-8b
pipeline_tag: text-generation
tags:
- SQL
- Code
- Meta
---
# SandLogic Technology - Quantized llama-3-sqlcoder-8b Models

## Model Description

We have quantized the llama-3-sqlcoder-8b model into two variants:

1. Q5_KM
2. Q4_KM

These quantized models offer improved efficiency while maintaining performance.

Discover our full range of quantized language models by visiting our [SandLogic Lexicon GitHub](https://github.com/sandlogic/SandLogic-Lexicons). To learn more about our company and services, check out our website at [SandLogic](https://www.sandlogic.com).
## Original Model Information

- **Name**: [llama-3-sqlcoder-8b](https://huggingface.co/defog/llama-3-sqlcoder-8b)
- **Developer**: Defog, Inc.
- **Model Type**: Text-to-SQL generation
- **Base Model**: Meta-Llama-3-8B-Instruct
- **Parameters**: 8 billion
- **License**: CC-by-SA-4.0

## Model Capabilities

The llama-3-sqlcoder-8b model is designed for generating SQL queries to answer questions, with support for Postgres, Redshift, and Snowflake databases. It has performance on-par with the most capable generalist frontier models.

## Inference Parameters

- **Temperature**: 0 (no sampling)
- **Prompt Format**:
```<|begin_of_text|><|start_header_id|>user<|end_header_id|>
Generate a SQL query to answer this question: {user_question}
{instructions}
DDL statements:
{create_table_statements}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
The following SQL query best answers the question {user_question}:
```
## Evaluation

The model was evaluated on SQL-Eval, a PostgreSQL-based evaluation framework developed by Defog for testing and alignment of model capabilities.

## Intended Use Cases

1. **SQL Generation**: Automatically generate SQL queries based on natural language questions or instructions.
2. **Database Interaction**: Assist users in interacting with Postgres, Redshift, or Snowflake databases through text-based interfaces.
3. **Data Analysis Support**: Provide SQL-based solutions to data analysis problems described in natural language.
4. **Programming Education**: Help students learn SQL concepts and syntax by providing example queries and explanations.

## Model Variants

We offer two quantized versions of the llama-3-sqlcoder-8b model:

1. **Q5_KM**: 5-bit quantization using the KM method
2. **Q4_KM**: 4-bit quantization using the KM method

These quantized models aim to reduce model size and improve inference speed while maintaining performance as close to the original model as possible.


## Usage

```bash
pip install llama-cpp-python 
```
Please refer to the llama-cpp-python [documentation](https://llama-cpp-python.readthedocs.io/en/latest/) to install with GPU support.

### Basic Text Completion
Here's an example demonstrating how to use the high-level API for basic text completion:

```bash
from llama_cpp import Llama

llm = Llama(
    model_path="./model/llama-3-sqlcoder-8b.Q5_K_M.gguf",
    verbose=False,
    # n_gpu_layers=-1, # Uncomment to use GPU acceleration
    # n_ctx=2048, # Uncomment to increase the context window
)

output = llm.create_chat_completion(
    messages = [
        {"role": "system", "content": "You're an AI SQL coding  assistant who help in solving coding questions"},
        {
            "role": "user",
            "content": "write an simple sql table query and code to search employee name"
        }
    ]
)

print(output["choices"][0]['message']['content'])
```

## Download
You can download `Llama` models in `gguf` format directly from Hugging Face using the `from_pretrained` method. This feature requires the `huggingface-hub` package.

To install it, run: `pip install huggingface-hub`

```bash
from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="SandLogicTechnologies/Llama-3-Sqlcoder-8B-GGUF",
    filename="*llama-3-sqlcoder-8b.Q5_K_M.gguf",
    verbose=False
)
```
By default, from_pretrained will download the model to the Hugging Face cache directory. You can manage installed model files using the huggingface-cli tool.

## License

License: [CC-by-SA-4.0] Finetuned from model: [Meta-Llama-3-8B-Instruct]



## Acknowledgements

We thank Defog, Inc. for developing the original llama-3-sqlcoder-8b model and the creators of Llama3 for their foundational work.
Special thanks to Georgi Gerganov and the entire llama.cpp development team for their outstanding contributions.



## Contact

For any inquiries or support, please contact us at [email protected] or visit our [support page](https://www.sandlogic.com).