Update README.md

README.md

@@ -1,47 +1,107 @@
 ---
 tags:
 - generated_from_keras_callback
-model-index:
-- name: multiqa-Mmini-L6-H384
-  results: []
+- sentence-transformers
+- feature-extraction
+- sentence-similarity
 ---
 
-<!-- This model card has been generated automatically according to the information Keras had access to. You should
-probably proofread and complete it, then remove this comment. -->
 
-# multiqa-Mmini-L6-H384
+# MultiQA-mMini-L6-H384
 
-This model is a fine-tuned version of [nreimers/mMiniLMv2-L6-H384-distilled-from-XLMR-Large](https://huggingface.co/nreimers/mMiniLMv2-L6-H384-distilled-from-XLMR-Large) on an unknown dataset.
-It achieves the following results on the evaluation set:
+This model is a fine-tuned version of [nreimers/mMiniLMv2-L6-H384-distilled-from-XLMR-Large](https://huggingface.co/nreimers/mMiniLMv2-L6-H384-distilled-from-XLMR-Large) on the [carlesoctav/en-id-parallel-sentences](https://huggingface.co/datasets/carlesoctav/en-id-parallel-sentences) dataset using the following procedure: [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://paperswithcode.com/paper/making-monolingual-sentence-embeddings). It achieves 92% accuracy on the validation split of the dataset for the English-Indonesian language pair in the bitext mining task.
 
+## Model Description
 
-## Model description
+Since we followed the approach outlined in [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://paperswithcode.com/paper/making-monolingual-sentence-embeddings), we used [sentence-transformers/multi-qa-MiniLM-L6-dot-v1](https://huggingface.co/sentence-transformers/multi-qa-MiniLM-L6-dot-v1) as the teacher model and [nreimers/mMiniLMv2-L6-H384-distilled-from-XLMR-Large](https://huggingface.co/nreimers/mMiniLMv2-L6-H384-distilled-from-XLMR-Large) as the student model (multilingual).
 
-More information needed
+Example of usage:
+```python
 
-## Intended uses & limitations
+from transformers import AutoTokenizer, AutoModel
+import torch
 
-More information needed
+#CLS Pooling - Take output from first token
+def cls_pooling(model_output):
+    return model_output.last_hidden_state[:,0]
 
-## Training and evaluation data
+#Encode text
+def encode(texts):
+    # Tokenize sentences
+    encoded_input = tokenizer(texts, padding=True, truncation=True, return_tensors='pt')
 
-More information needed
+    # Compute token embeddings
+    with torch.no_grad():
+        model_output = model(**encoded_input, return_dict=True)
 
-## Training procedure
+    # Perform pooling
+    embeddings = cls_pooling(model_output)
 
-### Training hyperparameters
+    return embeddings
 
-The following hyperparameters were used during training:
-- optimizer: None
-- training_precision: float32
 
-### Training results
+# Sentences we want sentence embeddings for
+query = "How many people live in London?"
+docs = ["Around 9 Million people live in London", "London is known for its financial district", "sekitar 9 juta orang tinggal di london", " London terkenal sebagai distrik finansial"]
 
+# Load model from HuggingFace Hub
+tokenizer = AutoTokenizer.from_pretrained("carlesoctav/multi-qa-en-id-mMiniLMv2-L6-H384")
 
+model = AutoModel.from_pretrained("carlesoctav/multi-qa-en-id-mMiniLMv2-L6-H384", from_tf = True)
 
-### Framework versions
+#Encode query and docs
+query_emb = encode(query)
+doc_emb = encode(docs)
+
+#Compute dot score between query and all document embeddings
+scores = torch.mm(query_emb, doc_emb.transpose(0, 1))[0].cpu().tolist()
+
+#Combine docs & scores
+doc_score_pairs = list(zip(docs, scores))
+
+#Sort by decreasing score
+doc_score_pairs = sorted(doc_score_pairs, key=lambda x: x[1], reverse=True)
+
+#Output passages & scores
+for doc, score in doc_score_pairs:
+    print(score, doc)
+```
+
+
+Take a look at the demo on Google Colab [here](https://colab.research.google.com/drive/1EZb0qACRIug9BVRX7LziKPchpYBUru9e#scrollTo=tZAjbx-_AOsg).
+
+## Intended Uses & Limitations
+
+Our model is intended to be used for semantic search. It encodes queries/questions and text paragraphs into dense vectors, allowing it to find relevant documents based on the given passages.
+
+The model is designed to create sentence embeddings specifically for semantic search and information retrieval tasks. As the student model, it inherits this capability from the fine-tuned teacher model. It supports both English and Indonesian languages, making it suitable for cross-lingual information retrieval tasks.
+
+Please note that there is a limit of 256 word pieces, and any text longer than that will be truncated. Additionally, the model was trained using input text up to 80 word pieces, so it may not perform optimally on longer text.
+
+In the following some technical details how this model must be used:
+
+| Setting | Value |
+| --- | :---: |
+| Dimensions | 384 |
+| Produces normalized embeddings | No |
+| Pooling-Method | CLS pooling |
+| Suitable score functions | dot-product (e.g. `util.dot_score`) |
+
+----
+
+## Training and Evaluation Data
+
+We utilized the [carlesoctav/en-id-parallel-sentences](https://huggingface.co/datasets/carlesoctav/en-id-parallel-sentences) dataset for training and evaluation purposes. The data was dynamically split into 95% for training and 5% for validation.
+
+## Training Procedure
+
+The complete training script can be found in the current repository under the name `train.py`.
+
+### Framework Versions
+
+The following framework versions were used:
 
 - Transformers 4.29.1
 - TensorFlow 2.12.0
 - Datasets 2.12.0
-- Tokenizers 0.13.3
+- Tokenizers 0.13.3