Tokenizer
#41
by
startlightquyet
- opened
How to train tokenizer custum dataset language not en and chinese . Continue 13000 vocab english and chinese model chatglm6b concat tokenizer custum after traning
How to train tokenizer custum dataset language not en and chinese . Continue 13000 vocab english and chinese model chatglm6b concat tokenizer custum after traning