Transformers
Safetensors
openlm
Inference Endpoints

not found tokenizer_config.json and tokenizer.json

#2
by sinlew - opened

Why can't the tokenizer_config.json and tokenizer.json... be found?
tokenizer = AutoTokenizer.from_pretrained("apple/DCLM-Baseline-7B-8k")
model = AutoModelForCausalLM.from_pretrained("apple/DCLM-Baseline-7B-8k")
DCLM-Baseline-7B-8k ?

Apple org

I have uploaded the tokenizer files. Note that this is just a GPT-NeoX tokenizer

apf1 changed discussion status to closed

Sign up or log in to comment