uploaded readme

README.md

@@ -0,0 +1,176 @@
+Quantization made by Richard Erkhov.
+
+[Github](https://github.com/RichardErkhov)
+
+[Discord](https://discord.gg/pvy7H8DZMG)
+
+[Request more models](https://github.com/RichardErkhov/quant_request)
+
+
+FollowIR-7B - bnb 4bits
+- Model creator: https://huggingface.co/jhu-clsp/
+- Original model: https://huggingface.co/jhu-clsp/FollowIR-7B/
+
+
+
+
+Original model description:
+---
+license: apache-2.0
+language:
+- en
+tags:
+- retrieval
+- instructions
+- reranking
+- mteb
+datasets:
+- jhu-clsp/FollowIR-train
+model-index:
+- name: FollowIR-7B
+  results:
+  - task:
+      type: InstructionRetrieval
+    dataset:
+      type: jhu-clsp/core17-instructions
+      name: MTEB Core17InstructionRetrieval
+      config: default
+      split: test
+      revision: e39ff896cf3efbbdeeb950e6bd7c79f266995b07
+    metrics:
+    - type: p-MRR
+      value: 16.47851858684521
+  - task:
+      type: InstructionRetrieval
+    dataset:
+      type: jhu-clsp/news21-instructions
+      name: MTEB News21InstructionRetrieval
+      config: default
+      split: test
+      revision: e0144086b45fe31ac125e9ac1a83b6a409bb6ca6
+    metrics:
+    - type: p-MRR
+      value: 6.2615989256510005
+  - task:
+      type: InstructionRetrieval
+    dataset:
+      type: jhu-clsp/robust04-instructions
+      name: MTEB Robust04InstructionRetrieval
+      config: default
+      split: test
+      revision: a5a1c4fe2bc528ac12e83f8cdf82178da85d2f1d
+    metrics:
+    - type: p-MRR
+      value: 13.717553757582253
+---
+
+# Model Summary
+
+FollowIR-7B is an instruction-tuned language model to be used for reranking in retrieval. It is [Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) fine-tuned on retrieval data with instructions from the FollowIR dataset. These instructions were taken from TREC tracks and are human written. FollowIR-7B outperforms all other retrieval models at following instructions. See the paper for more details.
+
+- **Repository:** [orionw/FollowIR](https://github.com/orionw/FollowIR)
+- **Paper:** https://arxiv.org/abs/2403.15246 
+- **Instruction-Training Dataset:** [jhu-clsp/followir-train](https://huggingface.co/datasets/jhu-clsp/FollowIR-train)
+
+
+# Use
+
+Below is an example to compute the similarity score of a query-document pair
+```python
+from transformers import (
+    AutoTokenizer,
+    AutoModelForCausalLM,
+)
+import torch
+
+# model loading and setup
+model_name = "jhu-clsp/FollowIR-7B"
+model = AutoModelForCausalLM.from_pretrained(
+    model_name
+).cuda()
+tokenizer = AutoTokenizer.from_pretrained(
+    model_name, padding_side="left"
+)
+tokenizer.pad_token = tokenizer.eos_token
+tokenizer.padding_side = "left"
+token_false_id = tokenizer.get_vocab()["false"]
+token_true_id = tokenizer.get_vocab()["true"]
+template = """<s> [INST] You are an expert Google searcher, whose job is to determine if the following document is relevant to the query (true/false). Answer using only one word, one of those two choices.
+
+Query: {query}
+Document: {text}
+Relevant (only output one word, either "true" or "false"): [/INST] """
+
+
+## Lets define some example queries with instructions in the query and the passage
+query1 = "What movies were written by James Cameron? A relevant document would describe a movie that was written by James Cameron only and not with anyone else"
+query2 = "What movies were directed by James Cameron? A relevant document would describe any movie that was directed by James Cameron"
+passages = ["Avatar: The Way of Water is a 2022 American epic science fiction film co-produced and directed by James Cameron, who co-wrote the screenplay with Rick Jaffa and Amanda Silver from a story the trio wrote with Josh Friedman and Shane Salerno. Distributed by 20th Century Studios, it is the sequel to Avatar (2009) and the second installment in the Avatar film series."] * 2
+
+prompts = [
+    template.format(query=query, text=text) for (query, text) in zip([query1, query2], passages)
+]
+tokens = tokenizer(
+    prompts,
+    padding=True,
+    truncation=True,
+    return_tensors="pt",
+    pad_to_multiple_of=None,
+)
+
+# move to cuda if desired
+for key in tokens:
+    tokens[key] = tokens[key].cuda()
+
+# calculate the scores by comparing true and false tokens
+batch_scores = model(**tokens).logits[:, -1, :]
+true_vector = batch_scores[:, token_true_id]
+false_vector = batch_scores[:, token_false_id]
+batch_scores = torch.stack([false_vector, true_vector], dim=1)
+batch_scores = torch.nn.functional.log_softmax(batch_scores, dim=1)
+scores = batch_scores[:, 1].exp().tolist()
+print(scores) # [0.0020704232156276703, 0.9999990463256836] first document is not relevant, as expected
+```
+
+# Training
+
+We used [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) to fine-tune Mistral to create FollowIR-7B, after transforming it to fit their format (input of "query" + "instruction" inside the template, output is the label, and instruction as the beginning of the template) with the following training script:
+```bash
+#!/bin/bash
+accelerate launch src/train_bash.py \
+    --stage sft \
+    --do_train \
+    --model_name_or_path "mistralai/Mistral-7B-Instruct-v0.2" \
+    --dataset followIR-train \
+    --template mistral \
+    --output_dir OUTPUT \
+    --finetuning_type lora \
+    --lora_target q_proj,v_proj,o_proj,k_proj \
+    --overwrite_cache \
+    --per_device_train_batch_size 32 \
+    --gradient_accumulation_steps 1 \
+    --lr_scheduler_type cosine \
+    --logging_steps 2 \
+    --save_steps 29 \
+    --learning_rate 3e-5 \
+    --num_train_epochs 8.0 \
+    --plot_loss \
+    --max_length 2048 \
+    --lora_rank 8 \
+    --lora_alpha 16 \
+    --bf16 
+```
+
+# Citation
+
+```bibtex
+@misc{weller2024followir,
+      title={FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions}, 
+      author={Orion Weller and Benjamin Chang and Sean MacAvaney and Kyle Lo and Arman Cohan and Benjamin Van Durme and Dawn Lawrie and Luca Soldaini},
+      year={2024},
+      eprint={2403.15246},
+      archivePrefix={arXiv},
+      primaryClass={cs.IR}
+}
+```
+