eduagarcia commited on
Commit
ff9ccb0
1 Parent(s): 1253c3b

Add tiktoken for stabilityai/stablelm-2-1_6b and Qwen/Qwen-* models

Browse files
Files changed (2) hide show
  1. .gitignore +1 -0
  2. requirements.txt +1 -0
.gitignore CHANGED
@@ -6,6 +6,7 @@ __pycache__/
6
  .vscode/
7
  run_dot_env.sh
8
  hub/
 
9
 
10
  eval-queue/
11
  eval-results/
 
6
  .vscode/
7
  run_dot_env.sh
8
  hub/
9
+ modules/
10
 
11
  eval-queue/
12
  eval-results/
requirements.txt CHANGED
@@ -15,4 +15,5 @@ sentencepiece
15
  tqdm==4.65.0
16
  transformers==4.36.0
17
  tokenizers>=0.15.0
 
18
  gradio-space-ci @ git+https://huggingface.co/spaces/Wauplin/[email protected] # CI !!!
 
15
  tqdm==4.65.0
16
  transformers==4.36.0
17
  tokenizers>=0.15.0
18
+ tiktoken>=0.5.2
19
  gradio-space-ci @ git+https://huggingface.co/spaces/Wauplin/[email protected] # CI !!!