AstraMindAI
/

AstraQuasar-4B

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

mlinmg commited on Feb 15

Commit

34534bc

•

1 Parent(s): e00b54b

Update README.md

Files changed (1) hide show

README.md +2 -1

README.md CHANGED Viewed

@@ -48,8 +48,9 @@ AstraQuasar-4B can be easily instantiated using the Hugging Face Transformers li
 ```python
 from transformers import AutoTokenizer, AutoModelForCausalLM
-model = AutoModelForCausalLM.from_pretrained("AstraMindAI/AstraQuasar-4B", trust_remote_code=True)
 tokenizer = AutoTokenizer.from_pretrained("AstraMindAI/AstraQuasar-4B")
 # you can optionally disable the duplicate trick

 ```python
 from transformers import AutoTokenizer, AutoModelForCausalLM
+import torch
+model = AutoModelForCausalLM.from_pretrained("AstraMindAI/AstraQuasar-4B", torch_dtype=torch.float16, trust_remote_code=True)
 tokenizer = AutoTokenizer.from_pretrained("AstraMindAI/AstraQuasar-4B")
 # you can optionally disable the duplicate trick