What is the max token length for text input?

#8
by kk3dmax - opened

What is the max token length for text input?

IIRC you get a warning past 1250 tokens

Maybe @ylacombe knows from working with it?

Hey,
It is a soft maximum which corresponds to the minimum between 1024 tokens and a linear transformation of the input length:
For text, it's min(1024, inp_length + 200).

Sign up or log in to comment