Hugging Face's logo

NghiemAbe
/
QAvietnamese

Text Generation
PEFT
Vietnamese
Model card Files Community
1
Edit model card

Information

  • Model Pretrain: VietAI/gpt-neo-1.3B-vietnamese-news

Training procedure

batch_size: int = 128

micro_batch_size: int = 2

num_epochs: int = 1

learning_rate: float = 3e-4

cutoff_len: int = 256

val_set_size: int = 0

Lora hyperparams

lora_r: int = 6

lora_alpha: int = 12

lora_dropout: float = 0.05

lora_target_modules: str = "q_proj k_proj v_proj"

Framework versions

  • transformer 4.32.1
  • pytorch
  • PEFT 0.5.0

How to use 

BASE_MODEL = "VietAI/gpt-neo-1.3B-vietnamese-news"
PEFT_WEIGHTS = "NghiemAbe/QAvietnamese"

model = AutoModelForCausalLM.from_pretrained(BASE_MODEL, torch_dtype=torch.bfloat16)
model = PeftModel.from_pretrained(model, PEFT_WEIGHTS, torch_dtype=torch.bfloat16)
Downloads last month
8
Inference Examples
Text Generation
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.