metadata

language:
  - en
  - ja
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation

PLaMo-13B

Model Description

PLaMo-13B-Instruct is an instruct fine-tuned model based on the 8192 context length version of Plamo-13B text-generation model. PLaMo-13B-Instruct is fine-tuned using several publicly available datasets. This model is released under Apache v2.0 license.

PLaMo-13B Release blog (Japanese)

Requirements

numpy
safetensors
sentencepiece
torch
transformers

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("pfnet/plamo-13b", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("pfnet/plamo-13b", trust_remote_code=True)
text = "これからの人工知能技術は"
input_ids = tokenizer(text, return_tensors="pt").input_ids
generated_tokens = model.generate(
    inputs=input_ids,
    max_new_tokens=32,
    do_sample=True,
    top_k=50,
    top_p=0.95,
    temperature=1.0,
)[0]
generated_text = tokenizer.decode(generated_tokens)
print(generated_text)

Model Details

Model size: 13B
Trained tokens: 1.5T tokens (English: 1.32T tokens, Japanese: 0.18T tokens)
Tokenizer: sentencepiece tokenizer trained on a subset of the pretraining datasets.
Context length: 8192
Developed by: Preferred Networkfs, Inc
Model type: Causal decoder-only
Language(s): English, Japanese
License: Apache v2.0

Training Dataset

For the pretraining model, see Plamo-13B.

Bias, Risks, and Limitations

PLaMo-13B-Instruct is a new technology that carries risks with use. Testing conducted to date has been in English and Japanese, and has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, PLaMo-13B’s potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses to user prompts. Therefore, before deploying any applications of PLaMo-13B, developers should perform safety testing and tuning tailored to their specific applications of the model.

How to cite

@online{PLaMoInstruct2023Introducing,
    author    = {Preferred Networks, Inc},
    title     = {PLaMo-13B-Instruct},
    year      = {2023},
    url       = {https://huggingface.co/pfnet/plamo-13b-instruct},
    urldate   = {2023-10-26}
}

References

@article{touvron2023llama,
  title={LLaMA: Open and Efficient Foundation Language Models},
  author={Touvron, Hugo and Lavril, Thibaut and Izacard, Gautier and Martinet, Xavier and Lachaux, Marie-Anne and Lacroix, Timoth{\'e}e and Rozi{\`e}re, Baptiste and Goyal, Naman and Hambro, Eric and Azhar, Faisal and Rodriguez, Aurelien and Joulin, Armand and Grave, Edouard and Lample, Guillaume},
  journal={arXiv preprint arXiv:2302.13971},
  year={2023}
}