Upload folder using huggingface_hub

README.md

@@ -367,6 +367,18 @@ with torch.no_grad():
     print(scores)
 ```
 
+## Load model in local
+
+1. make sure `gemma_config.py` and `gemma_model.py` from [BAAI/bge-reranker-v2.5-gemma2-lightweight](https://huggingface.co/BAAI/bge-reranker-v2.5-gemma2-lightweight/tree/main) in your local path.
+2. modify the following part of config.json:
+```
+"auto_map": {
+    "AutoConfig": "gemma_config.CostWiseGemmaConfig",
+    "AutoModel": "gemma_model.CostWiseGemmaModel",
+    "AutoModelForCausalLM": "gemma_model.CostWiseGemmaForCausalLM"
+  },
+```
+
 ## Evaluation
 
 The configuration of saving 60% Flops is: `compress_ratios=2`, `compress_layer=[8]`, `cutoff_layers=[25]`.