README.md

metadata

license: gemma
library_name: colpali
base_model: vidore/colpaligemma-3b-pt-448-base
language:
  - en
tags:
  - colpali
datasets:
  - vidore/colpali_train_set

ColPali: Visual Retriever based on PaliGemma-3B with ColBERT strategy (merged, state_dict only)

WARNING: This repository should only be used to convert the original ColPali weights to the Hf version of Colpali.

State dict was exported from vidore/colpali-v1.2-merged. The model was loaded using the following code:

# Get the device
device = "mps"

# Define dtype
dtype_precision = torch.bfloat16

# Load the original model and state_dict
model = (
    cast(
        ColPali,
        ColPali.from_pretrained(
            "vidore/colpali-v1.2-merged",
            torch_dtype=dtype_precision,
            device_map=device,
        ),
    )
    .to(dtype_precision)
    .eval()
)