zRzRzRzRzRzRzR's picture katuni4ka's picture
update tokenizer for compatibility with new transformers (#64)
741f31c verified