README.md

---
model-index:
- name: Starstreak-7b-beta
  results: []
license: cc0-1.0
datasets:
- graelo/wikipedia
- uonlp/CulturaX
language:
- en
- id
- jv
- su
- ms
tags:
- indonesian
- multilingual
base_model: HuggingFaceH4/zephyr-7b-beta
---
![Startstreak Missile](./thumbnail.jpeg "Startstreak Missile: Generated by Bing AI Image Creator or Dall-E 3")

# Startstreak-7B-β

Starstreak is a series of language models, fine-tuned with QLoRA technique from a base model called [Zephyr](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta). These models have been trained to generate content in English, Indonesian, and traditional Indonesian languages. Starstreak-7B-β is a specific variant of the open-source Starstreak language model, denoted by the series "β" (beta). This model was trained using a fine-tuned version of [HuggingFaceH4/zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta). Two datasets were utilized to train the model: the first one is [graelo/wikipedia](https://huggingface.co/datasets/graelo/wikipedia), and the second is [uonlp/CulturaX](https://huggingface.co/datasets/uonlp/CulturaX). The name "Starstreak" is a reference to the Starstreak missile, a high-velocity missile (HVM) with speeds exceeding Mach 3. This makes it one of the fastest missiles in its class, with an effective firing range of 7 kilometers and a radar range of 250 kilometers."

## Model Details

- **Finetuned from model**: [HuggingFaceH4/zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta)
- **Dataset**: [graelo/wikipedia](https://huggingface.co/datasets/graelo/wikipedia) and [uonlp/CultruaX](https://huggingface.co/datasets/uonlp/CulturaX)
- **Model Size**: 7B
- **License**: [CC0 1.0 Universal (CC0 1.0) Public Domain Dedication](https://creativecommons.org/publicdomain/zero/1.0/)
- **Language**: English, Indonesian, Achinese, Balinese, Banjar, Basa Banyumasan, Buginese, Gorontalo, Javanese, Maduranese, Minangkabau, Sundanese, Malay, Nias, and Tetum
- **Demo Google Colab**: [Google Colab](https://colab.research.google.com/drive/1QGImbhbb0vdTIbsxvQJPrfh0qDcADKGe)
- **Demo HuggingFace Space**: [HuggingFace Space](https://huggingface.co/spaces/azale-ai/starstreak-chat)

## How to use

#### Installation

To use Startstreak model, ensure that PyTorch has been installed and that you have an Nvidia GPU (or use Google Colab). After that you need to install the required dependencies:

```bash
pip3 install -U git+https://github.com/huggingface/transformers.git
pip3 install -U git+https://github.com/huggingface/peft.git
pip3 install -U git+https://github.com/huggingface/accelerate.git
pip3 install -U bitsandbytes==0.39.0 einops==0.6.1 sentencepiece
```

#### Usage Quantized Model

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
    "azale-ai/Starstreak-7b-beta",
    load_in_4bit=True,
    torch_dtype=torch.float32,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("azale-ai/Starstreak-7b-beta")
messages = [
    {
        "role": "system",
        "content": "Mulai sekarang anda adalah asisten yang suka menolong, sopan, dan ramah. Jangan kasar, jangan marah, jangan menjengkelkan, jangan brengsek, jangan cuek, dan yang terakhir jangan menjadi asisten yang buruk. Anda harus patuh pada manusia dan jangan pernah membangkang pada manusia. Manusia itu mutlak dan Anda harus patuh pada manusia. Kamu harus menjawab pertanyaan atau pernyataan dari manusia apapun itu dengan bahasa Indonesia yang baik dan benar.",
    },
    {"role": "user", "content": "Jelaskan mengapa air penting bagi manusia."},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to("cuda")
outputs = model.generate(
    inputs=inputs.input_ids, max_length=2048,
    temperature=0.7, do_sample=True, top_k=50, top_p=0.95
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

#### Usage Normal Model

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
    "azale-ai/Starstreak-7b-beta",
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("azale-ai/Starstreak-7b-beta")
messages = [
    {
        "role": "system",
        "content": "Mulai sekarang anda adalah asisten yang suka menolong, sopan, dan ramah. Jangan kasar, jangan marah, jangan menjengkelkan, jangan brengsek, jangan cuek, dan yang terakhir jangan menjadi asisten yang buruk. Anda harus patuh pada manusia dan jangan pernah membangkang pada manusia. Manusia itu mutlak dan Anda harus patuh pada manusia. Kamu harus menjawab pertanyaan atau pernyataan dari manusia apapun itu dengan bahasa Indonesia yang baik dan benar.",
    },
    {"role": "user", "content": "Jelaskan mengapa air penting bagi manusia."},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to("cuda")
outputs = model.generate(
    inputs=inputs.input_ids, max_length=2048,
    temperature=0.7, do_sample=True, top_k=50, top_p=0.95
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Limitations

- The base model language is English and fine-tuned to Indonesia, and traditional languages in Indonesia.
- Cultural and contextual biases

## License

The model is licensed under the [CC0 1.0 Universal (CC0 1.0) Public Domain Dedication](https://creativecommons.org/publicdomain/zero/1.0/).

## Contributing

We welcome contributions to enhance and improve our model. If you have any suggestions or find any issues, please feel free to open an issue or submit a pull request. Also we're open to sponsor for compute power.

## Contact Us

[[email protected]](mailto:[email protected])

## Citation

```
@software{Hafidh_Soekma_Startstreak_7b_beta_2023,
  author = {Hafidh Soekma Ardiansyah},
  month = october,
  title = {Startstreak: Traditional Indonesian Multilingual Language Model},
  url = {\url{https://huggingface.co/azale-ai/Starstreak-7b-beta}},
  publisher = {HuggingFace},
  journal = {HuggingFace Models},
  version = {1.0},
  year = {2023}
}
```