We introduceInstructOR, a new method for computing text embeddings given task instructions: every text input
is embedded together with instructions explaining the use case (e.g., task and domain descriptions) .
Unlike encoders from prior work that are more specialized,InstructORis a single embedder that can generate text embeddings tailored to different downstream tasks and domains,withoutany further training.
We first annotate instructions for 330 diverse tasks and trainInstructORon this multitask mixture with a contrastive loss.
We evaluateInstructORon 70 embedding evaluation tasks (66 of which areunseenduring training) , ranging from classification and information retrieval to semantic textual similarity and text generation evaluation.InstructOR, while having an order of magnitude fewer parameters than the previous best model, achieves state-of-the-art performance, with an average improvement of 3.4% compared to the previous best results on the 70 diverse datasets.
Our analysis suggests thatInstructORis robust to changes in instructions, and that instruction finetuning mitigates the challenge of training a single model on diverse datasets.
Our model, code, and data are available athttps://instructor-embedding.github.io.