Update instructions on usage with infinity

#25
Files changed (1) hide show
  1. README.md +7 -0
README.md CHANGED
@@ -2909,6 +2909,13 @@ async def main():
2909
  asyncio.run(main())
2910
  ```
2911
 
 
 
 
 
 
 
 
2912
  ### Usage for Reranker
2913
 
2914
  Different from embedding model, reranker uses question and document as input and directly output similarity instead of embedding.
 
2909
  asyncio.run(main())
2910
  ```
2911
 
2912
+ for a docker based deployment with infinity:
2913
+ ```bash
2914
+ docker run --gpus all -v $PWD/data:/app/.cache -e HF_TOKEN=$HF_TOKEN -p "7997":"7997" \
2915
+ michaelf34/infinity:0.0.68 \
2916
+ v2 --model-id BAAI/bge-large-en-v1.5 --revision "main" --dtype float16 --batch-size 32 --engine torch --port 7997
2917
+ ```
2918
+
2919
  ### Usage for Reranker
2920
 
2921
  Different from embedding model, reranker uses question and document as input and directly output similarity instead of embedding.