What is the difference between unified fine-tuning and dense fine-tuning?
#75
by
HonorOfKing
- opened
Are the parameters fine-tuned in "unified fine-tuning" only two additional linear layers(sparse_linear and colbert_linear) compared to "dense fine-tuning"?
unified fine-tuning will fine-tune all parameters for three retrieval modes (encoder, and two additional linear layers), while dense fine-tuning only fine-tune the encoder for dense embedding.