💻 Local SmolLMs
Collection
SmolLM models in MLC, ONNX and GGUF format for local applications + in-browser demos
•
14 items
•
Updated
•
46
This is the smollm-360M-instruct-add-basics model in MLC format q0f16
.
The model can be used for projects MLC-LLM and WebLLM.
Here are some examples of using this model in MLC LLM. Before running the examples, please install MLC LLM by following the installation documentation.
In command line, run
mlc_llm chat HF://HuggingFaceTB/smollm-360M-instruct-add-basics-q0f16-MLC
In command line, run
mlc_llm serve HF://HuggingFaceTB/smollm-360M-instruct-add-basics-q0f16-MLC
from mlc_llm import MLCEngine
# Create engine
model = "HF://HuggingFaceTB/smollm-360M-instruct-add-basics-q0f16-MLC"
engine = MLCEngine(model)
# Run chat completion in OpenAI API.
for response in engine.chat.completions.create(
messages=[{"role": "user", "content": "What is the meaning of life?"}],
model=model,
stream=True,
):
for choice in response.choices:
print(choice.delta.content, end="", flush=True)
print("\n")
engine.terminate()
For more information on MLC LLM project, please visit our documentation and GitHub repo.
Base model
HuggingFaceTB/SmolLM-360M