casperhansen
/

mpt-7b-8k-chat-awq

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

casperhansen commited on Jul 28, 2023

Commit

75f4cb5

•

1 Parent(s): 19fe3d1

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -19,7 +19,7 @@ RTX 4090 + Intel i9 13900K (2 different VMs):
 - CUDA 12.0, Driver 525.125.06: 134 tokens/s (7.46 ms/token)
 - CUDA 12.0, Driver 525.125.06: 117 tokens/s (8.52 ms/token)
-RTX 4090 + AMD EPYC 7-Series (2 different VMs):
 - CUDA 12.2, Driver 535.54.03: 53 tokens/s (18.6 ms/token)
 - CUDA 12.2, Driver 535.54.03: 56 tokens/s (17.71 ms/token)
 - CUDA 12.0, Driver 525.125.06: 55 tokens/ (18.15 ms/token)

 - CUDA 12.0, Driver 525.125.06: 134 tokens/s (7.46 ms/token)
 - CUDA 12.0, Driver 525.125.06: 117 tokens/s (8.52 ms/token)
+RTX 4090 + AMD EPYC 7-Series (3 different VMs):
 - CUDA 12.2, Driver 535.54.03: 53 tokens/s (18.6 ms/token)
 - CUDA 12.2, Driver 535.54.03: 56 tokens/s (17.71 ms/token)
 - CUDA 12.0, Driver 525.125.06: 55 tokens/ (18.15 ms/token)