FriendliAI
/

Llama-2-70b-chat-hf-fp8

Text Generation

text-generation-inference

8-bit precision

Model card Files Files and versions Community

seungahdev commited on Apr 19

Commit

4ce01ee

•

1 Parent(s): 3a0aaf2

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -94,7 +94,7 @@ docker run \
   -e FRIENDLI_CONTAINER_SECRET="YOUR CONTAINER SECRET" \
   registry.friendli.ai/trial \
     --web-server-port 8000 \
-    --hf-model-name meta-llama/Llama-2-70b-chat-hf-fp8 \
     --num-devices 2  # Use tensor parallelism degree 2
 ```

   -e FRIENDLI_CONTAINER_SECRET="YOUR CONTAINER SECRET" \
   registry.friendli.ai/trial \
     --web-server-port 8000 \
+    --hf-model-name FriendliAI/Llama-2-70b-chat-hf-fp8 \
     --num-devices 2  # Use tensor parallelism degree 2
 ```