Update README.md

README.md

@@ -10,14 +10,12 @@ pipeline_tag: text-generation
 # PLaMo-13B
 
 ## Model Description
-PLaMo-13B is a LLaMA-based 13B model pre-trained on English and Japanese open datasets, developed by Preferred Networks, Inc.
-PLaMo-13B is released under Apache v2.0 license.
+PLaMo-13B-Instruct is an instruct fine-tuned model based on the 8192 context length version of [Plamo-13B](https://huggingface.co/pfnet/plamo-13b) text-generation model. PLaMo-13B-Instruct is fine-tuned using several publicly available datasets.
+This model is released under Apache v2.0 license.
 
 [PLaMo-13B Release blog (Japanese)](https://tech.preferred.jp/ja/blog/llm-plamo/)
 
-## Usage
-
-### Requirements
+## Requirements
 
 - numpy
 - safetensors
@@ -25,14 +23,7 @@ PLaMo-13B is released under Apache v2.0 license.
 - torch
 - transformers
 
-### Use a pipeline as a high-level helper
-```python
-import transformers
-pipeline = transformers.pipeline("text-generation", model="pfnet/plamo-13b", trust_remote_code=True)
-print(pipeline("The future of artificial intelligence technology is ", max_new_tokens=32))
-```
-
-### Load model directly
+## Usage
 ```python
 from transformers import AutoTokenizer, AutoModelForCausalLM
 tokenizer = AutoTokenizer.from_pretrained("pfnet/plamo-13b", trust_remote_code=True)
@@ -55,7 +46,8 @@ print(generated_text)
 
 - Model size: 13B
 - Trained tokens: 1.5T tokens (English: 1.32T tokens, Japanese: 0.18T tokens)
-- Context length: 4096
+- Tokenizer: sentencepiece tokenizer trained on a subset of the pretraining datasets.
+- Context length: 8192
 - Developed by: Preferred Networkfs, Inc
 - Model type: Causal decoder-only
 - Language(s): English, Japanese
@@ -63,35 +55,26 @@ print(generated_text)
 
 ## Training Dataset
 
-### English
-
-- C4 - English
-- Project Gutenberg
-- RedPajama - Arxiv
-- RedPajama - CommonCrawl - English
-- RedPajama - Github
-- RedPajama - StackExchange
-- RedPajama - Wikipedia
-
-### Japanese
+<!-- - [Stanford Alpaca (Japanese translation)](https://huggingface.co/datasets/fujiki/japanese_alpaca_data)-->
+- [databricks-dolly-15k (Japanese translation)](https://huggingface.co/datasets/kunishou/databricks-dolly-15k-ja)
+- [Anthropic HH-RLHF (Japanese translation, subset)](https://huggingface.co/datasets/fujiki/japanese_hh-rlhf-49k)
+- [OpenAssistant Conversations Dataset (Japanese translation, oasst1)](https://huggingface.co/datasets/kunishou/oasst1-89k-ja)
+- [Wikinews subset of Izumi-lab llm-japanese-dataset](https://huggingface.co/datasets/izumi-lab/llm-japanese-dataset)
 
-- mC4 - Japanese
-- Wikipedia - Japanese
+For the pretraining model, see [Plamo-13B](https://huggingface.co/pfnet/plamo-13b).
 
-## Tokenizer
-PLaMo-13B uses sentencepiece tokenizer which is trained on a subset of the datasets for model pre-training.
 
 ## Bias, Risks, and Limitations
-PLaMo-13B is a new technology that carries risks with use. Testing conducted to date has been in English and Japanese, and has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, PLaMo-13B’s potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses to user prompts. Therefore, before deploying any applications of PLaMo-13B, developers should perform safety testing and tuning tailored to their specific applications of the model.
+PLaMo-13B-Instruct is a new technology that carries risks with use. Testing conducted to date has been in English and Japanese, and has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, PLaMo-13B’s potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses to user prompts. Therefore, before deploying any applications of PLaMo-13B, developers should perform safety testing and tuning tailored to their specific applications of the model.
 
 ## How to cite
 ```tex
-@online{PLaMo2023Introducing,
+@online{PLaMoInstruct2023Introducing,
     author    = {Preferred Networks, Inc},
-    title     = {PLaMo-13B},
+    title     = {PLaMo-13B-Instruct},
     year      = {2023},
-    url       = {https://huggingface.co/pfnet/plamo-13b},
-    urldate   = {2023-09-28}
+    url       = {https://huggingface.co/pfnet/plamo-13b-instruct},
+    urldate   = {2023-10-26}
 }
 ```