Update README.md

README.md

@@ -32,28 +32,14 @@ Apache-2.0 (commercial use permitted)
 * [Codebase (mosaicml/llm-foundry repo)](https://github.com/mosaicml/llm-foundry/)
 * Questions: Feel free to contact us via the [MosaicML Community Slack](https://join.slack.com/t/mosaicml-community/shared_invite/zt-w0tiddn9-WGTlRpfjcO9J5jyrMub1dg)!
 
-### Example Dialogue
+### Example Question/Instruction
 
-**SamIAm85**:
->I want you to come up with a tweet based on this summary of the article:
->"Introducing MPT-7B, the latest entry in our MosaicML Foundation Series.
->MPT-7B is a transformer trained from scratch on IT tokens of text and code.
->It is open source, available for commercial use, and it matches the quality of LLaMA-7B.
->MPT-7B was trained on the MosaicML platform in 9.5 days with zero human intervention at a cost of ~$200k.
->Starting today, you can train, finetune, and deploy your own private MPT models, either starting from one of our checkpoints or training from scratch.
->For inspiration, we are also releasing three finetuned models in addition to the base MPT-7B: MPT-7B-Instruct, MPT-7B-Chat, and MPT-7B-Story Writer-65k+, the last of which uses a context length of 65k tokens!
+**Longboi24**
+> What is a quoll?
 
-**MPT-7B-Instruct**:
->Here's my suggestion for a tweet:
->"Exciting news! @MosaicML has launched #MPT_7B - a new standard for open-source, commercially usable LLMs. With training on 17 tokens of text & code, it matches the quality of LLaMA-7B. Now you can train, fine-tune, and deploy your own private MPT models via their platform. Check out the full blog post here: https://bit.ly/3j6UJzZ"
+**MPT-7B-Instruct**
 
-**SamIAm85**:
->Is there anything you want to say to MosaicML about this exciting news?
-
-**MPT-7B-Instruct**:
-> As an Al language model myself, I am very impressed with the advancements made by MosaicML in developing such high-quality open-source LLMs.
-> This new version of MPT-7B is truly impressive and I look forward to seeing what innovative applications developers will create using these powerful tools.
-> Thank you for your hard work and dedication to advancing Al research and development.
+>A Quoll (pronounced “cool”) is one of Australia’s native carnivorous marsupial mammals, which are also known as macropods or wallabies in other parts around Asia and South America
 
 ## How to Use
 
@@ -73,6 +59,21 @@ model = transformers.AutoModelForCausalLM.from_pretrained('mosaicml/mpt-7b-instr
 model.to(device='cuda:0', dtype=torch.bfloat16)
 ```
 
+Although the model was trained with a sequence length of 2048, ALiBi enables users to increase the maximum sequence length during finetuning and/or inference. For example:
+
+```python
+config = transformers.AutoConfig.from_pretrained('mosaicml/mpt-7b', trust_remote_code=True)
+config.update({"max_seq_len": 4096})
+model = transformers.AutoModelForCausalLM.from_pretrained('mosaicml/mpt-7b', config=config, trust_remote_code=True)
+```
+
+This model was trained with the [EleutherAI/gpt-neox-20b](https://huggingface.co/EleutherAI/gpt-neox-20b) tokenizer.
+
+```python
+from transformers import AutoTokenizer
+tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-neox-20b")
+```
+
 ## Model Description
 
 The architecture is a modification of a standard decoder-only transformer.
@@ -104,4 +105,4 @@ This model was finetuned on 440 A100-40GBs for about half a day using the [Mosai
 
 ## Acknowledgements
 
-This model was finetuned by Sam Havens 
+This model was finetuned by Sam Havens and the MosaicML NLP team