Update README.md
Browse files
README.md
CHANGED
@@ -17,7 +17,7 @@ Converted and quantized checkpoint of [nvidia/Nemotron-4-340B-Base](https://hugg
|
|
17 |
|
18 |
You can deploy this model with `vllm>=0.5.4` ([PR#6611](https://github.com/vllm-project/vllm/pull/6611)):
|
19 |
```
|
20 |
-
vllm serve mgoin/Nemotron-4-340B-
|
21 |
```
|
22 |
|
23 |
### Evaluations
|
|
|
17 |
|
18 |
You can deploy this model with `vllm>=0.5.4` ([PR#6611](https://github.com/vllm-project/vllm/pull/6611)):
|
19 |
```
|
20 |
+
vllm serve mgoin/Nemotron-4-340B-Base-hf-FP8 --tensor-parallel-size 8
|
21 |
```
|
22 |
|
23 |
### Evaluations
|