Hugging Face's logo

iashchak
/
ruGPT-3.5-13B-ggml

Transformers
Russian
English
gpt2
ruGPT
GGML
NLP
Text Generation
Inference Endpoints
Model card Files Community
3
ruGPT-3.5-13B-ggml / README.md
iashchak's picture
iashchak
Update README.md
3a29a4e
preview code
|
raw
history blame contribute delete
8.61 kB
metadata
language:
  - ru
  - en
tags:
  - ruGPT
  - GGML
  - NLP
  - Text Generation
license: mit

ruGPT-3.5-13B Converted to GGML Format / ruGPT-3.5-13B Конвертированная в формат GGML

Model Description / Описание модели

English

This repository contains a GGML-formatted version of the ruGPT-3.5-13B model originally hosted on Hugging Face. The model has 13 billion parameters and was initially trained on a 300GB dataset from various domains. It was further fine-tuned on 100GB of code and legal documents. The model understands both Russian and English.

Dataset Details

  • Training Data: 300GB from various domains
  • Fine-tuning Data: 100GB of code and legal documents
  • Technical Specs: Trained using Deepspeed and Megatron libraries on 300B tokens dataset for 3 epochs, around 45 days on 512 V100 GPUs. Fine-tuned for 1 epoch with a sequence length of 2048, around 20 days on 200 A100 GPUs.
  • Perplexity: Around 8.8 for Russian language

Usage

For QuantizationType.Q4_0 and ContainerType.GGJT 
from llm_rs import AutoModel
model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-q4_0-ggjt.bin")
print(model.generate("The meaning of life is ").text)
For QuantizationType.Q4_0 and ContainerType.GGML 
from llm_rs import AutoModel
model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-q4_0.bin")
print(model.generate("The meaning of life is ").text)
For QuantizationType.Q4_1 and ContainerType.GGJT 
from llm_rs import AutoModel
model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-q4_1-ggjt.bin")
print(model.generate("The meaning of life is ").text)
For QuantizationType.Q4_1 and ContainerType.GGML 
from llm_rs import AutoModel
model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-q4_1.bin")
print(model.generate("The meaning of life is ").text)
For QuantizationType.Q5_0 and ContainerType.GGJT 
from llm_rs import AutoModel
model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-q5_0-ggjt.bin")
print(model.generate("The meaning of life is ").text)
For QuantizationType.Q5_0 and ContainerType.GGML 
from llm_rs import AutoModel
model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-q5_0.bin")
print(model.generate("The meaning of life is ").text)
For QuantizationType.Q5_1 and ContainerType.GGJT 
from llm_rs import AutoModel
model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-q5_1-ggjt.bin")
print(model.generate("The meaning of life is ").text)
For QuantizationType.Q5_1 and ContainerType.GGML 
from llm_rs import AutoModel
model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-q5_1.bin")
print(model.generate("The meaning of life is ").text)
For QuantizationType.Q8_0 and ContainerType.GGJT 
from llm_rs import AutoModel
model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-q8_0-ggjt.bin")
print(model.generate("The meaning of life is ").text)
For QuantizationType.Q8_0 and ContainerType.GGML 
from llm_rs import AutoModel
model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-q8_0.bin")
print(model.generate("The meaning of life is ").text)
f16 Version 
# f16 Version
from llm_rs import AutoModel

model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-f16.bin")
print(model.generate("Смысл жизни в ").text)

Compatibility

While this model is intended to be compatible with any GGML-compatible UI, it has not been extensively tested in such environments. Use at your own risk.

Русский

Этот репозиторий содержит версию модели ruGPT-3.5-13B в формате GGML. Модель имеет 13 миллиардов параметров и изначально обучалась на 300ГБ данных из различных доменов. Далее она была дообучена на 100ГБ кода и юридических документов. Модель понимает как русский, так и английский языки.

Детали набора данных

  • Тренировочные данные: 300ГБ из различных доменов
  • Данные для дообучения: 100ГБ кода и юридических документов
  • Технические характеристики: Обучена с использованием библиотек Deepspeed и Megatron на наборе данных из 300 миллиардов токенов за 3 эпохи, примерно 45 дней на 512 GPU V100. Дообучена 1 эпоху с длиной последовательности 2048, примерно 20 дней на 200 GPU A100.
  • Перплексия: Около 8,8 для русского языка

Использование

Для QuantizationType.Q4_0 и ContainerType.GGJT 
from llm_rs import AutoModel
model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-q4_0-ggjt.bin")
print(model.generate("Смысл жизни в ").text)
Для QuantizationType.Q4_0 и ContainerType.GGML 
from llm_rs import AutoModel
model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-q4_0.bin")
print(model.generate("Смысл жизни в ").text)
Для QuantizationType.Q4_1 и ContainerType.GGJT 
from llm_rs import AutoModel
model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-q4_1-ggjt.bin")
print(model.generate("Смысл жизни в ").text)
Для QuantizationType.Q4_1 и ContainerType.GGML 
from llm_rs import AutoModel
model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-q4_1.bin")
print(model.generate("Смысл жизни в ").text)
Для QuantizationType.Q5_0 и ContainerType.GGJT 
from llm_rs import AutoModel
model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-q5_0-ggjt.bin")
print(model.generate("Смысл жизни в ").text)
Для QuantizationType.Q5_0 и ContainerType.GGML 
from llm_rs import AutoModel
model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-q5_0.bin")
print(model.generate("Смысл жизни в ").text)
Для QuantizationType.Q5_1 и ContainerType.GGJT 
from llm_rs import AutoModel
model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-q5_1-ggjt.bin")
print(model.generate("Смысл жизни в ").text)
Для QuantizationType.Q5_1 и ContainerType.GGML 
from llm_rs import AutoModel
model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-q5_1.bin")
print(model.generate("Смысл жизни в ").text)
Для QuantizationType.Q8_0 и ContainerType.GGJT 
from llm_rs import AutoModel
model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-q8_0-ggjt.bin")
print(model.generate("Смысл жизни в ").text)
Для QuantizationType.Q8_0 и ContainerType.GGML 
from llm_rs import AutoModel
model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-q8_0.bin")
print(model.generate("Смысл жизни в ").text)
Версия f16 
from llm_rs import AutoModel
model = AutoModel.from_pretrained("iashchak/ruGPT-3.5-13B-ggml", model_file="ruGPT-3.5-13B-f16.bin")
print(model.generate("Смысл жизни в ").text)

Совместимость

Хотя эта модель предназначена для совместимости с любым GGML-совместимым интерфейсом, она не была тщательно протестирована в таких средах. Используйте на свой страх и риск.