ColBERT Models
Collection
A collection of ColBERT models finetuned for several languages
•
5 items
•
Updated
Training
Details
The model is initialized from the dbmdz/bert-base-german-uncased checkpoint and fine-tuned on 10M triples via pairwise softmax cross-entropy loss over the computed scores of the positive and negative passages associated to a query. It was trained on a single Tesla A100 GPU with 40GBs of memory during 200k steps with 10% of warmup steps using a batch size of 96 and the AdamW optimizer with a constant learning rate of 3e-06. Total training time was around 12 hours.
Data
The model is fine-tuned on the German version of the mMARCO dataset, a multi-lingual machine-translated version of the MS MARCO dataset. The triples are sampled from the ~39.8M triples of triples.train.small.tsv
Evaluation
The model is evaluated on the smaller development set of mMARCO-de, which consists of 6,980 queries for a corpus of 8.8M candidate passages. We report the mean reciprocal rank (MRR) and recall at various cut-offs (R@k).
| model | Vocab. | #Param. | Size | MRR@10 | R@50 | R@1000 |
|---|---|---|---|---|---|---|
| ColBERTv1.0-german-mmarcoDE | german | 110M | 440MB | 26.62 | 63.66 | 68.32 |
- Downloads last month
- 3