No pad token

#2
by BartekSadlej - opened

Hi, what should I add as a padding token? I can see that the tokenizer doesn't have pad token.

EleutherAI org

You can use the bos token. Any token should be fine really so long as you mask them so that they don't interfere with the backpropagation signal.

Sign up or log in to comment