File size: 1,763 Bytes
35c81f4 05c9a45 514081f 35c81f4 05c9a45 bbcd968 8ebfd75 05c9a45 8ebfd75 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 |
---
license: cc-by-sa-4.0
language:
- ja
tags:
- japanese
- causal-lm
- quantized
inference: false
---
# OpenCALM-7B - 8bit
[](https://colab.research.google.com/gist/kyo-takano/0c7bf0479158aa137e0ba935dec70461/opencalm-7b-8bit.ipynb)
8-bit quantized version of [OpenCALM-7B by CyberAgent (under CC BY-SA 4.0)](https://huggingface.co/cyberagent/open-calm-7b)
When using this quantized model, please be sure to give credit to the original.
## Setup
```sh
pip install -q -U bitsandbytes
pip install -q -U git+https://github.com/huggingface/transformers.git
pip install -q -U git+https://github.com/huggingface/accelerate.git
```
## Usage
```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
MODEL_ID = "kyo-takano/open-calm-7b-8bit"
model = AutoModelForCausalLM.from_pretrained(MODEL_ID)
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
inputs = tokenizer("AIによって私達の暮らしは、", return_tensors="pt").to(model.device)
with torch.no_grad():
tokens = model.generate(
**inputs,
max_new_tokens=64,
do_sample=True,
temperature=0.7,
top_p=0.9,
repetition_penalty=1.05,
pad_token_id=tokenizer.pad_token_id,
)
output = tokenizer.decode(tokens[0], skip_special_tokens=True)
print(output)
```
## Model Details
- Developed by: CyberAgent, Inc.
- Quantized by: Kyo Takano
- Model type: Transformer-based Language Model
- Language: Japanese
- Library: GPT-NeoX
- License: OpenCALM is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0). When using this model, please provide appropriate credit to **CyberAgent, Inc.**
|