| tags: | |
| - deepsparse | |
| # mpt_7b_chat-dense_quant_linearW8A8MatMul8Embeds8LMhead8 | |
| ```python | |
| import deepsparse | |
| from huggingface_hub import snapshot_download | |
| MODEL_PATH = snapshot_download(repo_id="mgoin/mpt-7b-chat-quant") | |
| model = deepsparse.Pipeline.create(task="text-generation", model_path=MODEL_PATH) | |
| model(sequences="Tell me a joke.") | |
| ``` |