Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,178 @@
+
+---
+
+license: apache-2.0
+language:
+- uz
+- en
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: transformers
+tags:
+- text-generation-inference
+- summarization
+- translation
+- question-answering
+datasets:
+- tahrirchi/uz-crawl
+- allenai/c4
+- MLDataScientist/Wikipedia-uzbek-2024-05-01
+- yahma/alpaca-cleaned
+- behbudiy/alpaca-cleaned-uz
+- behbudiy/translation-instruction
+metrics:
+- bleu
+- comet
+- accuracy
+pipeline_tag: text-generation
+
+---
+
+[![QuantFactory Banner](https://lh7-rt.googleusercontent.com/docsz/AD_4nXeiuCm7c8lEwEJuRey9kiVZsRn2W-b4pWlu3-X534V3YmVuVc2ZL-NXg2RkzSOOS2JXGHutDuyyNAUtdJI65jGTo8jT9Y99tMi4H4MqL44Uc5QKG77B0d6-JfIkZHFaUA71-RtjyYZWVIhqsNZcx8-OMaA?key=xt3VSDoCbmTY7o-cwwOFwQ)](https://hf.co/QuantFactory)
+
+
+# QuantFactory/Mistral-7B-Instruct-Uz-GGUF
+This is quantized version of [behbudiy/Mistral-7B-Instruct-Uz](https://huggingface.co/behbudiy/Mistral-7B-Instruct-Uz) created using llama.cpp
+
+# Original Model Card
+
+
+### Model Description
+
+The Mistral-7B-Instruct-Uz model has been continually pre-trained and instruction-tuned using a mix of publicly available and syntheticly constructed Uzbek and English data to preserve its original knowledge while enhancing its capabilities. This model is designed to support various natural language processing tasks in Uzbek, such as machine translation, summarization, and dialogue systems, ensuring robust performance across these applications.
+For details regarding the performance metrics compared to the base model, see [this post.](https://www.linkedin.com/feed/update/urn:li:activity:7241389815559008256/)
+
+- **Developed by:**
+  - [Eldor Fozilov](https://www.linkedin.com/in/eldor-fozilov/)
+  - [Azimjon Urinov](https://azimjonn.github.io/)
+  - [Khurshid Juraev](https://kjuraev.com/)
+
+## Installation
+
+It is recommended to use `behbudiy/Mistral-7B-Instruct-Uz` with [mistral-inference](https://github.com/mistralai/mistral-inference). For HF transformers code snippets, please keep scrolling.
+
+```
+pip install mistral_inference
+```
+
+## Download
+
+```py
+from huggingface_hub import snapshot_download
+from pathlib import Path
+
+mistral_models_path = Path.home().joinpath('mistral_models', '7B-Instruct-Uz')
+mistral_models_path.mkdir(parents=True, exist_ok=True)
+
+snapshot_download(repo_id="behbudiy/Mistral-7B-Instruct-Uz", allow_patterns=["params.json", "consolidated.safetensors", "tokenizer.model.v3"], local_dir=mistral_models_path)
+```
+
+### Chat
+
+After installing `mistral_inference`, a `mistral-chat` CLI command should be available in your environment. You can chat with the model using
+
+```
+mistral-chat $HOME/mistral_models/7B-Instruct-Uz --instruct --max_tokens 256
+```
+
+### Instructiong Following
+
+```py
+from mistral_inference.transformer import Transformer
+from mistral_inference.generate import generate
+
+from mistral_common.tokens.tokenizers.mistral import MistralTokenizer
+from mistral_common.protocol.instruct.messages import UserMessage
+from mistral_common.protocol.instruct.request import ChatCompletionRequest
+
+
+tokenizer = MistralTokenizer.from_file(f"{mistral_models_path}/tokenizer.model.v3")
+model = Transformer.from_folder(mistral_models_path)
+
+completion_request = ChatCompletionRequest(messages=[UserMessage(content="O'zbekiston haqida ma'lumot ber.")])
+
+tokens = tokenizer.encode_chat_completion(completion_request).tokens
+
+out_tokens, _ = generate([tokens], model, max_tokens=64, temperature=0.0, eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id)
+result = tokenizer.instruct_tokenizer.tokenizer.decode(out_tokens[0])
+
+print(result)
+```
+
+## Generate with `transformers`
+
+If you want to use Hugging Face `transformers` to generate text, you can do something like this.
+
+```py
+from transformers import pipeline
+
+messages = [
+    {"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
+    {"role": "user", "content": "Who are you?"},
+]
+chatbot = pipeline("text-generation", model="behbudiy/Mistral-7B-Instruct-Uz", device='cuda')
+chatbot(messages)
+```
+
+## Information on Evaluation Method
+
+To evaluate on the translation task, we used FLORES+ Uz-En / En-Uz datasets, where we merged the dev and test sets to create a bigger evaluation data for each Uz-En and En-Uz subsets.
+We used the following prompt to do one-shot Uz-En evaluation both for the base model and Uzbek-optimized model (for En-Uz eval, we changed the positions of the words "English" and "Uzbek").
+
+```python
+  prompt = f'''You are a professional Uzbek-English translator. Your task is to accurately translate the given Uzbek text into English.
+
+  Instructions:
+  1. Translate the text from Uzbek to English.
+  2. Maintain the original meaning and tone.
+  3. Use appropriate English grammar and vocabulary.
+  4. If you encounter an ambiguous or unfamiliar word, provide the most likely translation based on context.
+  5. Output only the English translation, without any additional comments.
+
+  Example:
+  Uzbek: "Bugun ob-havo juda yaxshi, quyosh charaqlab turibdi."
+  English: "The weather is very nice today, the sun is shining brightly."
+
+  Now, please translate the following Uzbek text into English:
+  "{sentence}"
+    '''
+```
+
+To assess the model's ability in Uzbek sentiment analysis, we used the **risqaliyevds/uzbek-sentiment-analysis** dataset, for which we created binary labels (0: Negative, 1: Positive) using GPT-4o API (refer to **behbudiy/uzbek-sentiment-analysis** dataset).
+We used the following prompt for the evaluation:
+
+```python
+prompt = f'''Given the following text, determine the sentiment as either 'Positive' or 'Negative.' Respond with only the word 'Positive' or 'Negative' without any additional text or explanation.
+
+Text: {text}"
+'''
+```
+For Uzbek News Classification, we used **risqaliyevds/uzbek-zero-shot-classification** dataset and asked the model to predict the category of the news using the following prompt:
+
+```python
+prompt = f'''Classify the given Uzbek news article into one of the following categories. Provide only the category number as the answer.
+
+Categories:
+0 - Politics (Siyosat)
+1 - Economy (Iqtisodiyot)
+2 - Technology (Texnologiya)
+3 - Sports (Sport)
+4 - Culture (Madaniyat)
+5 - Health (Salomatlik)
+6 - Family and Society (Oila va Jamiyat)
+7 - Education (Ta'lim)
+8 - Ecology (Ekologiya)
+9 - Foreign News (Xorijiy Yangiliklar)
+
+Now classify this article:
+"{text}"
+
+Answer (number only):"
+'''
+```
+
+## MMLU
+We used [this script](https://github.com/FranxYao/chain-of-thought-hub/blob/461e2d551f3f12d54caee75fa1e915fdbc3e9d12/MMLU/run_mmlu_open_source.py).
+
+## More
+For more details and examples, refer to the base model below:
+https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3