Example:

from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig

tokenizer = AutoTokenizer.from_pretrained("Qwen-1_8B-m4-LDJnr-combined", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("Qwen-1_8B-m4-LDJnr-combined", device_map="auto", trust_remote_code=True).eval()
response, _ = model.chat(tokenizer, "What kind of a noise annoys a noisy oyster?", history=None)
print(response)

Downloads last month: 11

Safetensors

Model size

1.84B params

Tensor type

FP16

Inference Examples

Text Generation

Inference API (serverless) does not yet support model repos that contain custom code.

Collection including aloobun/Qwen-1_8B-m4-LDJnr-combined

qwenstar

Collection

3 items • Updated Feb 16