Update README.md
Browse files
README.md
CHANGED
@@ -1,7 +1,7 @@
|
|
1 |
---
|
2 |
-
base_model: vidore/colpaligemma-3b-mix-448-base
|
3 |
license: mit
|
4 |
library_name: colpali
|
|
|
5 |
language:
|
6 |
- en
|
7 |
tags:
|
@@ -14,6 +14,8 @@ It is a [PaliGemma-3B](https://huggingface.co/google/paligemma-3b-mix-448) exten
|
|
14 |
It was introduced in the paper [ColPali: Efficient Document Retrieval with Vision Language Models](https://arxiv.org/abs/2407.01449) and first released in [this repository](https://github.com/ManuelFay/colpali)
|
15 |
|
16 |
This version has right padding to fix unwanted tokens in the query encoding + hard negative mining.
|
|
|
|
|
17 |
|
18 |
## Model Description
|
19 |
|
@@ -59,8 +61,8 @@ def main() -> None:
|
|
59 |
"""Example script to run inference with ColPali"""
|
60 |
|
61 |
# Load model
|
62 |
-
model_name = "
|
63 |
-
model = ColPali.from_pretrained("
|
64 |
model.load_adapter(model_name)
|
65 |
processor = AutoProcessor.from_pretrained(model_name)
|
66 |
|
|
|
1 |
---
|
|
|
2 |
license: mit
|
3 |
library_name: colpali
|
4 |
+
base_model: vidore/colpaligemma-3b-mix-448-base
|
5 |
language:
|
6 |
- en
|
7 |
tags:
|
|
|
14 |
It was introduced in the paper [ColPali: Efficient Document Retrieval with Vision Language Models](https://arxiv.org/abs/2407.01449) and first released in [this repository](https://github.com/ManuelFay/colpali)
|
15 |
|
16 |
This version has right padding to fix unwanted tokens in the query encoding + hard negative mining.
|
17 |
+
It also stems from the fixed `vidore/colpaligemma-3b-mix-448-base` to guarantee deterministic projection layer initialization.
|
18 |
+
|
19 |
|
20 |
## Model Description
|
21 |
|
|
|
61 |
"""Example script to run inference with ColPali"""
|
62 |
|
63 |
# Load model
|
64 |
+
model_name = "manu/colpali-hard-v1.1"
|
65 |
+
model = ColPali.from_pretrained("vidore/colpaligemma-3b-mix-448-base", torch_dtype=torch.bfloat16, device_map="cuda").eval()
|
66 |
model.load_adapter(model_name)
|
67 |
processor = AutoProcessor.from_pretrained(model_name)
|
68 |
|