Hugging Face's logo

Xenova
/
Nemotron-4-340B-Instruct-Tokenizer

Transformers
Transformers.js
tokenizers
Inference Endpoints
Model card Files Community
1

Is this only for loading the nemotron tokenizer?

#1
by ZQ-Dev - opened
Discussion
ZQ-Dev

This repo seems to make the tokenizer load-able with HF transformers, but what about the model itself. Is this tokenizer fix all you need to load the model with AutoModelForCausalLM? If not, what are the next steps to work towards that?

Would be great to one day load nemotron-4 with TGI.

Xenova
 Owner

Hi there! Yes, this repo is only for the tokenizer; the model is being ported over here which you can follow for updates.

AyeshaIrshad124

how can i use the nemotron for the data generation ??

Sign up or log in to comment