ColPali
Safetensors
English
vidore_no_match
manu commited on
Commit
fd4e1bb
1 Parent(s): 5612e75

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -3
README.md CHANGED
@@ -1,7 +1,7 @@
1
  ---
2
- base_model: vidore/colpaligemma-3b-mix-448-base
3
  license: mit
4
  library_name: colpali
 
5
  language:
6
  - en
7
  tags:
@@ -14,6 +14,8 @@ It is a [PaliGemma-3B](https://huggingface.co/google/paligemma-3b-mix-448) exten
14
  It was introduced in the paper [ColPali: Efficient Document Retrieval with Vision Language Models](https://arxiv.org/abs/2407.01449) and first released in [this repository](https://github.com/ManuelFay/colpali)
15
 
16
  This version has right padding to fix unwanted tokens in the query encoding + hard negative mining.
 
 
17
 
18
  ## Model Description
19
 
@@ -59,8 +61,8 @@ def main() -> None:
59
  """Example script to run inference with ColPali"""
60
 
61
  # Load model
62
- model_name = "vidore/colpali"
63
- model = ColPali.from_pretrained("google/paligemma-3b-mix-448", torch_dtype=torch.bfloat16, device_map="cuda").eval()
64
  model.load_adapter(model_name)
65
  processor = AutoProcessor.from_pretrained(model_name)
66
 
 
1
  ---
 
2
  license: mit
3
  library_name: colpali
4
+ base_model: vidore/colpaligemma-3b-mix-448-base
5
  language:
6
  - en
7
  tags:
 
14
  It was introduced in the paper [ColPali: Efficient Document Retrieval with Vision Language Models](https://arxiv.org/abs/2407.01449) and first released in [this repository](https://github.com/ManuelFay/colpali)
15
 
16
  This version has right padding to fix unwanted tokens in the query encoding + hard negative mining.
17
+ It also stems from the fixed `vidore/colpaligemma-3b-mix-448-base` to guarantee deterministic projection layer initialization.
18
+
19
 
20
  ## Model Description
21
 
 
61
  """Example script to run inference with ColPali"""
62
 
63
  # Load model
64
+ model_name = "manu/colpali-hard-v1.1"
65
+ model = ColPali.from_pretrained("vidore/colpaligemma-3b-mix-448-base", torch_dtype=torch.bfloat16, device_map="cuda").eval()
66
  model.load_adapter(model_name)
67
  processor = AutoProcessor.from_pretrained(model_name)
68