uploaded readme

README.md

@@ -0,0 +1,328 @@
+Quantization made by Richard Erkhov.
+
+[Github](https://github.com/RichardErkhov)
+
+[Discord](https://discord.gg/pvy7H8DZMG)
+
+[Request more models](https://github.com/RichardErkhov/quant_request)
+
+
+sabia-7b - GGUF
+- Model creator: https://huggingface.co/maritaca-ai/
+- Original model: https://huggingface.co/maritaca-ai/sabia-7b/
+
+
+| Name | Quant method | Size |
+| ---- | ---- | ---- |
+| [sabia-7b.Q2_K.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.Q2_K.gguf) | Q2_K | 2.36GB |
+| [sabia-7b.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.IQ3_XS.gguf) | IQ3_XS | 2.6GB |
+| [sabia-7b.IQ3_S.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.IQ3_S.gguf) | IQ3_S | 2.75GB |
+| [sabia-7b.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.Q3_K_S.gguf) | Q3_K_S | 2.75GB |
+| [sabia-7b.IQ3_M.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.IQ3_M.gguf) | IQ3_M | 2.9GB |
+| [sabia-7b.Q3_K.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.Q3_K.gguf) | Q3_K | 3.07GB |
+| [sabia-7b.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.Q3_K_M.gguf) | Q3_K_M | 3.07GB |
+| [sabia-7b.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.Q3_K_L.gguf) | Q3_K_L | 3.35GB |
+| [sabia-7b.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.IQ4_XS.gguf) | IQ4_XS | 3.4GB |
+| [sabia-7b.Q4_0.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.Q4_0.gguf) | Q4_0 | 3.56GB |
+| [sabia-7b.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.IQ4_NL.gguf) | IQ4_NL | 3.58GB |
+| [sabia-7b.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.Q4_K_S.gguf) | Q4_K_S | 3.59GB |
+| [sabia-7b.Q4_K.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.Q4_K.gguf) | Q4_K | 3.8GB |
+| [sabia-7b.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.Q4_K_M.gguf) | Q4_K_M | 3.8GB |
+| [sabia-7b.Q4_1.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.Q4_1.gguf) | Q4_1 | 3.95GB |
+| [sabia-7b.Q5_0.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.Q5_0.gguf) | Q5_0 | 4.33GB |
+| [sabia-7b.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.Q5_K_S.gguf) | Q5_K_S | 4.33GB |
+| [sabia-7b.Q5_K.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.Q5_K.gguf) | Q5_K | 4.45GB |
+| [sabia-7b.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.Q5_K_M.gguf) | Q5_K_M | 4.45GB |
+| [sabia-7b.Q5_1.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.Q5_1.gguf) | Q5_1 | 4.72GB |
+| [sabia-7b.Q6_K.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.Q6_K.gguf) | Q6_K | 5.15GB |
+| [sabia-7b.Q8_0.gguf](https://huggingface.co/RichardErkhov/maritaca-ai_-_sabia-7b-gguf/blob/main/sabia-7b.Q8_0.gguf) | Q8_0 | 4.88GB |
+
+
+
+
+Original model description:
+---
+language:
+- pt
+model-index:
+- name: sabia-7b
+  results:
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: ENEM Challenge (No Images)
+      type: eduagarcia/enem_challenge
+      split: train
+      args:
+        num_few_shot: 3
+    metrics:
+    - type: acc
+      value: 55.07
+      name: accuracy
+    source:
+      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=maritaca-ai/sabia-7b
+      name: Open Portuguese LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: BLUEX (No Images)
+      type: eduagarcia-temp/BLUEX_without_images
+      split: train
+      args:
+        num_few_shot: 3
+    metrics:
+    - type: acc
+      value: 47.71
+      name: accuracy
+    source:
+      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=maritaca-ai/sabia-7b
+      name: Open Portuguese LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: OAB Exams
+      type: eduagarcia/oab_exams
+      split: train
+      args:
+        num_few_shot: 3
+    metrics:
+    - type: acc
+      value: 41.41
+      name: accuracy
+    source:
+      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=maritaca-ai/sabia-7b
+      name: Open Portuguese LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: Assin2 RTE
+      type: assin2
+      split: test
+      args:
+        num_few_shot: 15
+    metrics:
+    - type: f1_macro
+      value: 46.68
+      name: f1-macro
+    source:
+      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=maritaca-ai/sabia-7b
+      name: Open Portuguese LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: Assin2 STS
+      type: eduagarcia/portuguese_benchmark
+      split: test
+      args:
+        num_few_shot: 15
+    metrics:
+    - type: pearson
+      value: 1.89
+      name: pearson
+    source:
+      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=maritaca-ai/sabia-7b
+      name: Open Portuguese LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: FaQuAD NLI
+      type: ruanchaves/faquad-nli
+      split: test
+      args:
+        num_few_shot: 15
+    metrics:
+    - type: f1_macro
+      value: 58.34
+      name: f1-macro
+    source:
+      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=maritaca-ai/sabia-7b
+      name: Open Portuguese LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: HateBR Binary
+      type: ruanchaves/hatebr
+      split: test
+      args:
+        num_few_shot: 25
+    metrics:
+    - type: f1_macro
+      value: 61.93
+      name: f1-macro
+    source:
+      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=maritaca-ai/sabia-7b
+      name: Open Portuguese LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: PT Hate Speech Binary
+      type: hate_speech_portuguese
+      split: test
+      args:
+        num_few_shot: 25
+    metrics:
+    - type: f1_macro
+      value: 64.13
+      name: f1-macro
+    source:
+      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=maritaca-ai/sabia-7b
+      name: Open Portuguese LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: tweetSentBR
+      type: eduagarcia-temp/tweetsentbr
+      split: test
+      args:
+        num_few_shot: 25
+    metrics:
+    - type: f1_macro
+      value: 46.64
+      name: f1-macro
+    source:
+      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=maritaca-ai/sabia-7b
+      name: Open Portuguese LLM Leaderboard
+---
+
+Sabiá-7B is Portuguese language model developed by [Maritaca AI](https://www.maritaca.ai/).
+
+**Input:** The model accepts only text input.
+
+**Output:** The Model generates text only.
+
+**Model Architecture:** Sabiá-7B is an auto-regressive language model that uses the same architecture of LLaMA-1-7B.
+
+**Tokenizer:** It uses the same tokenizer as LLaMA-1-7B.
+
+**Maximum sequence length:** 2048 tokens.
+
+**Pretraining data:** The model was pretrained on 7 billion tokens from the Portuguese subset of ClueWeb22, starting with the weights of LLaMA-1-7B and further trained for an additional 10 billion tokens, approximately 1.4 epochs of the training dataset.
+
+**Data Freshness:** The pretraining data has a cutoff of mid-2022.
+
+**License:** The licensing is the same as LLaMA-1's, restricting the model's use to research purposes only.
+
+**Paper:** For more details, please refer to our paper: [Sabiá: Portuguese Large Language Models](https://arxiv.org/pdf/2304.07880.pdf) 
+
+
+## Few-shot Example
+
+Given that Sabiá-7B was trained solely on a language modeling objective without fine-tuning for instruction following, it is recommended for few-shot tasks rather than zero-shot tasks, like in the example below.
+
+```python
+import torch
+from transformers import LlamaTokenizer, LlamaForCausalLM
+
+tokenizer = LlamaTokenizer.from_pretrained("maritaca-ai/sabia-7b")
+model = LlamaForCausalLM.from_pretrained(
+    "maritaca-ai/sabia-7b",
+    device_map="auto",  # Automatically loads the model in the GPU, if there is one. Requires pip install acelerate
+    low_cpu_mem_usage=True,
+    torch_dtype=torch.bfloat16   # If your GPU does not support bfloat16, change to torch.float16
+)  
+
+prompt = """Classifique a resenha de filme como "positiva" ou "negativa".
+
+Resenha: Gostei muito do filme, é o melhor do ano!
+Classe: positiva
+
+Resenha: O filme deixa muito a desejar.
+Classe: negativa
+
+Resenha: Apesar de longo, valeu o ingresso.
+Classe:"""
+
+input_ids = tokenizer(prompt, return_tensors="pt")
+
+output = model.generate(
+    input_ids["input_ids"].to("cuda"),
+    max_length=1024,
+    eos_token_id=tokenizer.encode("\n"))  # Stop generation when a "\n" token is dectected
+
+# The output contains the input tokens, so we have to skip them.
+output = output[0][len(input_ids["input_ids"][0]):]
+
+print(tokenizer.decode(output, skip_special_tokens=True))
+```
+
+If your GPU does not have enough RAM, try using int8 precision.
+However, expect some degradation in the model output quality when compared to fp16 or bf16.
+```python
+model = LlamaForCausalLM.from_pretrained(
+    "maritaca-ai/sabia-7b",
+    device_map="auto",
+    low_cpu_mem_usage=True,
+    load_in_8bit=True,  # Requires pip install bitsandbytes
+)
+```
+
+## Results in Portuguese
+
+Below we show the results on the Poeta benchmark, which consists of 14 Portuguese datasets.
+
+For more information on the Normalized Preferred Metric (NPM), please refer to our paper.
+
+|Model | NPM |
+|--|--|
+|LLaMA-1-7B| 33.0|
+|LLaMA-2-7B| 43.7|
+|Sabiá-7B| 48.5|
+
+## Results in English 
+
+Below we show the average results on 6 English datasets: PIQA, HellaSwag, WinoGrande, ARC-e, ARC-c, and OpenBookQA.
+
+|Model | NPM |
+|--|--|
+|LLaMA-1-7B| 50.1|
+|Sabiá-7B| 49.0|
+
+
+## Citation
+
+Please use the following bibtex to cite our paper: 
+```
+@InProceedings{10.1007/978-3-031-45392-2_15,
+    author="Pires, Ramon
+    and Abonizio, Hugo
+    and Almeida, Thales Sales
+    and Nogueira, Rodrigo",
+    editor="Naldi, Murilo C.
+    and Bianchi, Reinaldo A. C.",
+    title="Sabi{\'a}: Portuguese Large Language Models",
+    booktitle="Intelligent Systems",
+    year="2023",
+    publisher="Springer Nature Switzerland",
+    address="Cham",
+    pages="226--240",
+    isbn="978-3-031-45392-2"
+}
+```
+
+# [Open Portuguese LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard)
+Detailed results can be found [here](https://huggingface.co/datasets/eduagarcia-temp/llm_pt_leaderboard_raw_results/tree/main/maritaca-ai/sabia-7b)
+
+|          Metric          |  Value  |
+|--------------------------|---------|
+|Average                   |**47.09**|
+|ENEM Challenge (No Images)|    55.07|
+|BLUEX (No Images)         |    47.71|
+|OAB Exams                 |    41.41|
+|Assin2 RTE                |    46.68|
+|Assin2 STS                |     1.89|
+|FaQuAD NLI                |    58.34|
+|HateBR Binary             |    61.93|
+|PT Hate Speech Binary     |    64.13|
+|tweetSentBR               |    46.64|
+
+
+