yunmorning commited on
Commit
ccbfb55
1 Parent(s): b8a3236

Update docker run command

Browse files
Files changed (1) hide show
  1. README.md +8 -18
README.md CHANGED
@@ -49,7 +49,6 @@ This model is compatible with **[Friendli Container](https://friendli.ai/product
49
  - Before you begin, make sure you have signed up for [Friendli Suite](https://suite.friendli.ai/). **You can use Friendli Containers free of charge for four weeks.**
50
  - Prepare a Personal Access Token following [this guide](#preparing-personal-access-token).
51
  - Prepare a Friendli Container Secret following [this guide](#preparing-container-secret).
52
- - Install Hugging Face CLI with `pip install -U "huggingface_hub[cli]"`
53
 
54
  ### Preparing Personal Access Token
55
 
@@ -88,24 +87,15 @@ You should pass the container secret as an environment variable to run the conta
88
  Once you've prepared the image of Friendli Container, you can launch it to create a serving endpoint.
89
 
90
  ```sh
91
- export MODEL_DIR=$PWD/FriendliAI--Llama-2-13b-chat-hf-fp8
92
- export FRIENDLI_CONTAINER_SECRET="YOUR CONTAINER SECRET"
93
- export FRIENDLI_CONTAINER_IMAGE="registry.friendli.ai/trial"
94
- export GPU_ENUMERATION='"device=0"'
95
-
96
- huggingface-cli download FriendliAI/Llama-2-13b-chat-hf-fp8 \
97
- --local-dir $MODEL_DIR \
98
- --local-dir-use-symlinks False
99
-
100
  docker run \
101
- --gpus $GPU_ENUMERATION --network=host --ipc=host \
102
- -v $MODEL_DIR:/model \
103
- -e FRIENDLI_CONTAINER_SECRET=$FRIENDLI_CONTAINER_SECRET \
104
- $FRIENDLI_CONTAINER_IMAGE /bin/bash -c \
105
- "/root/launcher \
106
- --web-server-port 6000 \
107
- --ckpt-path /model \
108
- --ckpt-type hf_safetensors"
109
  ```
110
 
111
  ---
 
49
  - Before you begin, make sure you have signed up for [Friendli Suite](https://suite.friendli.ai/). **You can use Friendli Containers free of charge for four weeks.**
50
  - Prepare a Personal Access Token following [this guide](#preparing-personal-access-token).
51
  - Prepare a Friendli Container Secret following [this guide](#preparing-container-secret).
 
52
 
53
  ### Preparing Personal Access Token
54
 
 
87
  Once you've prepared the image of Friendli Container, you can launch it to create a serving endpoint.
88
 
89
  ```sh
 
 
 
 
 
 
 
 
 
90
  docker run \
91
+ --gpus '"device=0"' \
92
+ -p 8000:8000 \
93
+ -v ~/.cache/huggingface:/root/.cache/huggingface \
94
+ -e FRIENDLI_CONTAINER_SECRET="YOUR CONTAINER SECRET" \
95
+ -e HF_TOKEN="YOUR HUGGING FACE TOKEN" \
96
+ registry.friendli.ai/trial \
97
+ --web-server-port 8000 \
98
+ --hf-model-name meta-llama/Llama-2-13b-chat-hf-fp8
99
  ```
100
 
101
  ---