chatglm3-6b / tokenization_chatglm.py

Commit History

Add a fast tokenizer implementation and converter
92670ad

chielo commited on

Add encode_special_tokens option
e46a148

duzx16 commited on

Implement new interface
1676f07

duzx16 commited on

Add system prompt
591fa87

duzx16 commited on

Implement new chat method
85ba2d2

duzx16 commited on