Vocabulary Size?

#10

by MH1P - opened Mar 22

MH1P

Mar 22

Hi!

Thank you for the model.
I am just wondering: the vocab size in config.json says 32032, but only 2 tokens are added. What are the extra 30 tokens?

Thanks!

NousResearch org Apr 2

Axolotl doesnt actually add padded tokens for and so it didn't. I just merged a PR that does

MH1P

Apr 11

Thanks!

MH1P changed discussion status to closed Apr 11

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment