Trying to run this model locally, on my machine
#75
by
abdulrafay97
- opened
Hey,
I am new to this. can anyone help me how can I use this model locally?
Hello!
If you have Python installed, then you can run:
pip install sentence-transformers
to install the Sentence Transformers package which allows easy access to these models.
Then, you can use the model like so:
from sentence_transformers import SentenceTransformer
# 1. Load a pretrained Sentence Transformer model
model = SentenceTransformer("all-MiniLM-L6-v2")
# The sentences to encode
sentences = [
"The weather is lovely today.",
"It's so sunny outside!",
"He drove to the stadium.",
]
# 2. Calculate embeddings by calling model.encode()
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]
# 3. Calculate the embedding similarities
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.6660, 0.1046],
# [0.6660, 1.0000, 0.1411],
# [0.1046, 0.1411, 1.0000]])
Hope this helps!
- Tom Aarsen
Hey @tomaarsen ,
Thank you for your input. But I want to run this model offline on my system. The approach you suggested is still using the API.
The approach I described actually downloads the model weights locally and computes the embeddings on your CPU or GPU.
- Tom Aarsen
@tomaarsen
Oh, I just realized that. Thank you.
However, I believe the model restarts each time. I would like to run it on my own system to avoid the startup delay.