mlabonne/llama-2-7b-miniguanaco

🦙🧠 Miniguanaco-7b

📝 Article | 💻 Colab | 📄 Script

This is a Llama-2-7b-chat-hf model fine-tuned using QLoRA (4-bit precision) on the mlabonne/guanaco-llama2-1k dataset, which is a subset of the timdettmers/openassistant-guanaco.

🔧 Training

It was trained on a Google Colab notebook with a T4 GPU and high RAM. It is mainly designed for educational purposes, not for inference.

💻 Usage

# pip install transformers accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "mlabonne/llama-2-7b-miniguanaco"
prompt = "What is a large language model?"

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto",
)

sequences = pipeline(
    f'<s>[INST] {prompt} [/INST]',
    do_sample=True,
    top_k=10,
    num_return_sequences=1,
    eos_token_id=tokenizer.eos_token_id,
    max_length=200,
)
for seq in sequences:
    print(f"Result: {seq['generated_text']}")

Output:

A large language model is trained on massive amounts of text data to understand and generate human language. The model learns by predicting the next word in a sequence based on the context of the previous words. This process allows the language model to learn patterns, rules, and relationships within the language that allow it to generate text that looks and sounds authentic and coherent. These large language models are used for many applications, such as language translation, sentiment analysis, and language generation. These models can also be used to generate text summaries of complex documents, such as legal or scientific papers, or to generate text summaries of social media posts. These models are often used in natural language processing (NLP) and machine learning applications. The large language models are trained using a large number of parameters, often in the billions or even in the tens of billions.