thxCode
commited on
Commit
•
38d1577
0
Parent(s):
feat: first commit
Browse filesSigned-off-by: thxCode <[email protected]>
- .gitattributes +36 -0
- README.md +225 -0
- jina-reranker-v1-tiny-en-FP16.gguf +3 -0
- jina-reranker-v1-tiny-en-Q2_K.gguf +3 -0
- jina-reranker-v1-tiny-en-Q3_K_M.gguf +3 -0
- jina-reranker-v1-tiny-en-Q4_0.gguf +3 -0
- jina-reranker-v1-tiny-en-Q4_K_M.gguf +3 -0
- jina-reranker-v1-tiny-en-Q5_0.gguf +3 -0
- jina-reranker-v1-tiny-en-Q5_K_M.gguf +3 -0
- jina-reranker-v1-tiny-en-Q6_K.gguf +3 -0
- jina-reranker-v1-tiny-en-Q8_0.gguf +3 -0
.gitattributes
ADDED
@@ -0,0 +1,36 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
*.7z filter=lfs diff=lfs merge=lfs -text
|
2 |
+
*.arrow filter=lfs diff=lfs merge=lfs -text
|
3 |
+
*.bin filter=lfs diff=lfs merge=lfs -text
|
4 |
+
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
5 |
+
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
6 |
+
*.ftz filter=lfs diff=lfs merge=lfs -text
|
7 |
+
*.gz filter=lfs diff=lfs merge=lfs -text
|
8 |
+
*.h5 filter=lfs diff=lfs merge=lfs -text
|
9 |
+
*.joblib filter=lfs diff=lfs merge=lfs -text
|
10 |
+
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
11 |
+
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
12 |
+
*.model filter=lfs diff=lfs merge=lfs -text
|
13 |
+
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
14 |
+
*.npy filter=lfs diff=lfs merge=lfs -text
|
15 |
+
*.npz filter=lfs diff=lfs merge=lfs -text
|
16 |
+
*.onnx filter=lfs diff=lfs merge=lfs -text
|
17 |
+
*.ot filter=lfs diff=lfs merge=lfs -text
|
18 |
+
*.parquet filter=lfs diff=lfs merge=lfs -text
|
19 |
+
*.pb filter=lfs diff=lfs merge=lfs -text
|
20 |
+
*.pickle filter=lfs diff=lfs merge=lfs -text
|
21 |
+
*.pkl filter=lfs diff=lfs merge=lfs -text
|
22 |
+
*.pt filter=lfs diff=lfs merge=lfs -text
|
23 |
+
*.pth filter=lfs diff=lfs merge=lfs -text
|
24 |
+
*.rar filter=lfs diff=lfs merge=lfs -text
|
25 |
+
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
26 |
+
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
27 |
+
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
28 |
+
*.tar filter=lfs diff=lfs merge=lfs -text
|
29 |
+
*.tflite filter=lfs diff=lfs merge=lfs -text
|
30 |
+
*.tgz filter=lfs diff=lfs merge=lfs -text
|
31 |
+
*.wasm filter=lfs diff=lfs merge=lfs -text
|
32 |
+
*.xz filter=lfs diff=lfs merge=lfs -text
|
33 |
+
*.zip filter=lfs diff=lfs merge=lfs -text
|
34 |
+
*.zst filter=lfs diff=lfs merge=lfs -text
|
35 |
+
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
36 |
+
*.gguf filter=lfs diff=lfs merge=lfs -text
|
README.md
ADDED
@@ -0,0 +1,225 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
library_name: transformers
|
3 |
+
license: apache-2.0
|
4 |
+
language:
|
5 |
+
- en
|
6 |
+
tags:
|
7 |
+
- reranker
|
8 |
+
- cross-encoder
|
9 |
+
- transformers.js
|
10 |
+
pipeline_tag: text-classification
|
11 |
+
---
|
12 |
+
|
13 |
+
# jina-reranker-v1-tiny-en-GGUF
|
14 |
+
|
15 |
+
**Model creator**: [Jina AI](https://huggingface.co/jinaai)
|
16 |
+
**Original model**: [jina-reranker-v1-tiny-en](https://huggingface.co/jinaai/jina-reranker-v1-tiny-en)
|
17 |
+
**GGUF quantization**: based on llama.cpp release [f4d2b](https://github.com/ggerganov/llama.cpp/commit/f4d2b8846a6b34419ff9e9491aee6cd95e444bfc)
|
18 |
+
|
19 |
+
---
|
20 |
+
|
21 |
+
<br><br>
|
22 |
+
|
23 |
+
<p align="center">
|
24 |
+
<img src="https://aeiljuispo.cloudimg.io/v7/https://cdn-uploads.huggingface.co/production/uploads/603763514de52ff951d89793/AFoybzd5lpBQXEBrQHuTt.png?w=200&h=200&f=face" alt="Finetuner logo: Finetuner helps you to create experiments in order to improve embeddings on search tasks. It accompanies you to deliver the last mile of performance-tuning for neural search applications." width="150px">
|
25 |
+
</p>
|
26 |
+
|
27 |
+
<p align="center">
|
28 |
+
<b>Trained by <a href="https://jina.ai/"><b>Jina AI</b></a>.</b>
|
29 |
+
</p>
|
30 |
+
|
31 |
+
# jina-reranker-v1-tiny-en
|
32 |
+
|
33 |
+
This model is designed for **blazing-fast** reranking while maintaining **competitive performance**. What's more, it leverages the power of our [JinaBERT](https://arxiv.org/abs/2310.19923) model as its foundation. `JinaBERT` itself is a unique variant of the BERT architecture that supports the symmetric bidirectional variant of [ALiBi](https://arxiv.org/abs/2108.12409). This allows `jina-reranker-v1-tiny-en` to process significantly longer sequences of text compared to other reranking models, up to an impressive **8,192** tokens.
|
34 |
+
|
35 |
+
To achieve the remarkable speed, the `jina-reranker-v1-tiny-en` employ a technique called knowledge distillation. Here, a complex, but slower, model (like our original [jina-reranker-v1-base-en](https://jina.ai/reranker/)) acts as a teacher, condensing its knowledge into a smaller, faster student model. This student retains most of the teacher's knowledge, allowing it to deliver similar accuracy in a fraction of the time.
|
36 |
+
|
37 |
+
Here's a breakdown of the reranker models we provide:
|
38 |
+
|
39 |
+
| Model Name | Layers | Hidden Size | Parameters (Millions) |
|
40 |
+
| ------------------------------------------------------------------------------------ | ------ | ----------- | --------------------- |
|
41 |
+
| [jina-reranker-v1-base-en](https://jina.ai/reranker/) | 12 | 768 | 137.0 |
|
42 |
+
| [jina-reranker-v1-turbo-en](https://huggingface.co/jinaai/jina-reranker-v1-turbo-en) | 6 | 384 | 37.8 |
|
43 |
+
| [jina-reranker-v1-tiny-en](https://huggingface.co/jinaai/jina-reranker-v1-tiny-en) | 4 | 384 | 33.0 |
|
44 |
+
|
45 |
+
> Currently, the `jina-reranker-v1-base-en` model is not available on Hugging Face. You can access it via the [Jina AI Reranker API](https://jina.ai/reranker/).
|
46 |
+
|
47 |
+
As you can see, the `jina-reranker-v1-turbo-en` offers a balanced approach with **6 layers** and **37.8 million** parameters. This translates to fast search and reranking while preserving a high degree of accuracy. The `jina-reranker-v1-tiny-en` prioritizes speed even further, achieving the fastest inference speeds with its **4-layer**, **33.0 million** parameter architecture. This makes it ideal for scenarios where absolute top accuracy is less crucial.
|
48 |
+
|
49 |
+
# Usage
|
50 |
+
|
51 |
+
1. The easiest way to starting using `jina-reranker-v1-tiny-en` is to use Jina AI's [Reranker API](https://jina.ai/reranker/).
|
52 |
+
|
53 |
+
```bash
|
54 |
+
curl https://api.jina.ai/v1/rerank \
|
55 |
+
-H "Content-Type: application/json" \
|
56 |
+
-H "Authorization: Bearer YOUR_API_KEY" \
|
57 |
+
-d '{
|
58 |
+
"model": "jina-reranker-v1-tiny-en",
|
59 |
+
"query": "Organic skincare products for sensitive skin",
|
60 |
+
"documents": [
|
61 |
+
"Eco-friendly kitchenware for modern homes",
|
62 |
+
"Biodegradable cleaning supplies for eco-conscious consumers",
|
63 |
+
"Organic cotton baby clothes for sensitive skin",
|
64 |
+
"Natural organic skincare range for sensitive skin",
|
65 |
+
"Tech gadgets for smart homes: 2024 edition",
|
66 |
+
"Sustainable gardening tools and compost solutions",
|
67 |
+
"Sensitive skin-friendly facial cleansers and toners",
|
68 |
+
"Organic food wraps and storage solutions",
|
69 |
+
"All-natural pet food for dogs with allergies",
|
70 |
+
"Yoga mats made from recycled materials"
|
71 |
+
],
|
72 |
+
"top_n": 3
|
73 |
+
}'
|
74 |
+
```
|
75 |
+
|
76 |
+
2. Alternatively, you can use the latest version of the `sentence-transformers>=0.27.0` library. You can install it via pip:
|
77 |
+
|
78 |
+
```bash
|
79 |
+
pip install -U sentence-transformers
|
80 |
+
```
|
81 |
+
|
82 |
+
Then, you can use the following code to interact with the model:
|
83 |
+
|
84 |
+
```python
|
85 |
+
from sentence_transformers import CrossEncoder
|
86 |
+
|
87 |
+
# Load the model, here we use our tiny sized model
|
88 |
+
model = CrossEncoder("jinaai/jina-reranker-v1-tiny-en", trust_remote_code=True)
|
89 |
+
|
90 |
+
# Example query and documents
|
91 |
+
query = "Organic skincare products for sensitive skin"
|
92 |
+
documents = [
|
93 |
+
"Eco-friendly kitchenware for modern homes",
|
94 |
+
"Biodegradable cleaning supplies for eco-conscious consumers",
|
95 |
+
"Organic cotton baby clothes for sensitive skin",
|
96 |
+
"Natural organic skincare range for sensitive skin",
|
97 |
+
"Tech gadgets for smart homes: 2024 edition",
|
98 |
+
"Sustainable gardening tools and compost solutions",
|
99 |
+
"Sensitive skin-friendly facial cleansers and toners",
|
100 |
+
"Organic food wraps and storage solutions",
|
101 |
+
"All-natural pet food for dogs with allergies",
|
102 |
+
"Yoga mats made from recycled materials"
|
103 |
+
]
|
104 |
+
|
105 |
+
results = model.rank(query, documents, return_documents=True, top_k=3)
|
106 |
+
```
|
107 |
+
|
108 |
+
3. You can also use the `transformers` library to interact with the model programmatically.
|
109 |
+
|
110 |
+
```python
|
111 |
+
!pip install transformers
|
112 |
+
from transformers import AutoModelForSequenceClassification
|
113 |
+
|
114 |
+
model = AutoModelForSequenceClassification.from_pretrained(
|
115 |
+
'jinaai/jina-reranker-v1-tiny-en', num_labels=1, trust_remote_code=True
|
116 |
+
)
|
117 |
+
|
118 |
+
# Example query and documents
|
119 |
+
query = "Organic skincare products for sensitive skin"
|
120 |
+
documents = [
|
121 |
+
"Eco-friendly kitchenware for modern homes",
|
122 |
+
"Biodegradable cleaning supplies for eco-conscious consumers",
|
123 |
+
"Organic cotton baby clothes for sensitive skin",
|
124 |
+
"Natural organic skincare range for sensitive skin",
|
125 |
+
"Tech gadgets for smart homes: 2024 edition",
|
126 |
+
"Sustainable gardening tools and compost solutions",
|
127 |
+
"Sensitive skin-friendly facial cleansers and toners",
|
128 |
+
"Organic food wraps and storage solutions",
|
129 |
+
"All-natural pet food for dogs with allergies",
|
130 |
+
"Yoga mats made from recycled materials"
|
131 |
+
]
|
132 |
+
|
133 |
+
# construct sentence pairs
|
134 |
+
sentence_pairs = [[query, doc] for doc in documents]
|
135 |
+
|
136 |
+
scores = model.compute_score(sentence_pairs)
|
137 |
+
```
|
138 |
+
|
139 |
+
4. You can also use the `transformers.js` library to run the model directly in JavaScript (in-browser, Node.js, Deno, etc.)!
|
140 |
+
|
141 |
+
If you haven't already, you can install the [Transformers.js](https://huggingface.co/docs/transformers.js) JavaScript library from [NPM](https://www.npmjs.com/package/@xenova/transformers) using:
|
142 |
+
```bash
|
143 |
+
npm i @xenova/transformers
|
144 |
+
```
|
145 |
+
|
146 |
+
Then, you can use the following code to interact with the model:
|
147 |
+
```js
|
148 |
+
import { AutoTokenizer, AutoModelForSequenceClassification } from '@xenova/transformers';
|
149 |
+
|
150 |
+
const model_id = 'jinaai/jina-reranker-v1-tiny-en';
|
151 |
+
const model = await AutoModelForSequenceClassification.from_pretrained(model_id, { quantized: false });
|
152 |
+
const tokenizer = await AutoTokenizer.from_pretrained(model_id);
|
153 |
+
|
154 |
+
/**
|
155 |
+
* Performs ranking with the CrossEncoder on the given query and documents. Returns a sorted list with the document indices and scores.
|
156 |
+
* @param {string} query A single query
|
157 |
+
* @param {string[]} documents A list of documents
|
158 |
+
* @param {Object} options Options for ranking
|
159 |
+
* @param {number} [options.top_k=undefined] Return the top-k documents. If undefined, all documents are returned.
|
160 |
+
* @param {number} [options.return_documents=false] If true, also returns the documents. If false, only returns the indices and scores.
|
161 |
+
*/
|
162 |
+
async function rank(query, documents, {
|
163 |
+
top_k = undefined,
|
164 |
+
return_documents = false,
|
165 |
+
} = {}) {
|
166 |
+
const inputs = tokenizer(
|
167 |
+
new Array(documents.length).fill(query),
|
168 |
+
{ text_pair: documents, padding: true, truncation: true }
|
169 |
+
)
|
170 |
+
const { logits } = await model(inputs);
|
171 |
+
return logits.sigmoid().tolist()
|
172 |
+
.map(([score], i) => ({
|
173 |
+
corpus_id: i,
|
174 |
+
score,
|
175 |
+
...(return_documents ? { text: documents[i] } : {})
|
176 |
+
})).sort((a, b) => b.score - a.score).slice(0, top_k);
|
177 |
+
}
|
178 |
+
|
179 |
+
// Example usage:
|
180 |
+
const query = "Organic skincare products for sensitive skin"
|
181 |
+
const documents = [
|
182 |
+
"Eco-friendly kitchenware for modern homes",
|
183 |
+
"Biodegradable cleaning supplies for eco-conscious consumers",
|
184 |
+
"Organic cotton baby clothes for sensitive skin",
|
185 |
+
"Natural organic skincare range for sensitive skin",
|
186 |
+
"Tech gadgets for smart homes: 2024 edition",
|
187 |
+
"Sustainable gardening tools and compost solutions",
|
188 |
+
"Sensitive skin-friendly facial cleansers and toners",
|
189 |
+
"Organic food wraps and storage solutions",
|
190 |
+
"All-natural pet food for dogs with allergies",
|
191 |
+
"Yoga mats made from recycled materials",
|
192 |
+
]
|
193 |
+
|
194 |
+
const results = await rank(query, documents, { return_documents: true, top_k: 3 });
|
195 |
+
console.log(results);
|
196 |
+
```
|
197 |
+
|
198 |
+
|
199 |
+
That's it! You can now use the `jina-reranker-v1-tiny-en` model in your projects.
|
200 |
+
|
201 |
+
# Evaluation
|
202 |
+
|
203 |
+
We evaluated Jina Reranker on 3 key benchmarks to ensure top-tier performance and search relevance.
|
204 |
+
|
205 |
+
| Model Name | NDCG@10 (17 BEIR datasets) | NDCG@10 (5 LoCo datasets) | Hit Rate (LlamaIndex RAG) |
|
206 |
+
| ------------------------------------------ | -------------------------- | ------------------------- | ------------------------- |
|
207 |
+
| `jina-reranker-v1-base-en` | **52.45** | **87.31** | **85.53** |
|
208 |
+
| `jina-reranker-v1-turbo-en` | **49.60** | **69.21** | **85.13** |
|
209 |
+
| `jina-reranker-v1-tiny-en` (you are here) | **48.54** | **70.29** | **85.00** |
|
210 |
+
| `mxbai-rerank-base-v1` | 49.19 | - | 82.50 |
|
211 |
+
| `mxbai-rerank-xsmall-v1` | 48.80 | - | 83.69 |
|
212 |
+
| `ms-marco-MiniLM-L-6-v2` | 48.64 | - | 82.63 |
|
213 |
+
| `ms-marco-MiniLM-L-4-v2` | 47.81 | - | 83.82 |
|
214 |
+
| `bge-reranker-base` | 47.89 | - | 83.03 |
|
215 |
+
|
216 |
+
**Note:**
|
217 |
+
|
218 |
+
- `NDCG@10` is a measure of ranking quality, with higher scores indicating better search results. `Hit Rate` measures the percentage of relevant documents that appear in the top 10 search results.
|
219 |
+
- The results of LoCo datasets on other models are not available since they **do not support** long documents more than 512 tokens.
|
220 |
+
|
221 |
+
For more details, please refer to our [benchmarking sheets](https://docs.google.com/spreadsheets/d/1V8pZjENdBBqrKMzZzOWc2aL60wtnR0yrEBY3urfO5P4/edit?usp=sharing).
|
222 |
+
|
223 |
+
# Contact
|
224 |
+
|
225 |
+
Join our [Discord community](https://discord.jina.ai/) and chat with other community members about ideas.
|
jina-reranker-v1-tiny-en-FP16.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:ad9f450c1053a431e2e3746d1f9f7768fb9183cf0796e046983a3449dca093c2
|
3 |
+
size 67504480
|
jina-reranker-v1-tiny-en-Q2_K.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:dbd88c851aaf373569d38e25d34203f8e7ab17a899f767f1f035245cb00b1188
|
3 |
+
size 31645024
|
jina-reranker-v1-tiny-en-Q3_K_M.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:c9de4caa94e08899432aac407e4ee9124d61be5653a94dd57626a60b8c704692
|
3 |
+
size 32124256
|
jina-reranker-v1-tiny-en-Q4_0.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:ed226ea4efdb7cc8409961e22969231c90aa1194344486ed9e54b97a97e86b3a
|
3 |
+
size 31958368
|
jina-reranker-v1-tiny-en-Q4_K_M.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:4a31ab756f72581e1b457c10228abb3fa9778b15726191db5353339ed8f96aba
|
3 |
+
size 33257824
|
jina-reranker-v1-tiny-en-Q5_0.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:167e2325c46fe7b1edcf6038823fab5dc2afefe17a783867462257ce369e9811
|
3 |
+
size 33138016
|
jina-reranker-v1-tiny-en-Q5_K_M.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:b18581e13d637277a588544b5dc8cbf7860c697b881443cdb618fee357ab4b74
|
3 |
+
size 33829216
|
jina-reranker-v1-tiny-en-Q6_K.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:64989a5f36dbde883704060d8885771f3f6878d7980c8b5b8ebda99a26bc4958
|
3 |
+
size 36105568
|
jina-reranker-v1-tiny-en-Q8_0.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:0defc1f8a1f4dd22183124a2a25a97765603e5a9e42258046c9b2c8a26d1f553
|
3 |
+
size 36676960
|