langchain.embeddings.xinference.XinferenceEmbeddings¶
class langchain.embeddings.xinference.XinferenceEmbeddings(server_url: Optional[str] = None, model_uid: Optional[str] = None)[source]¶
Wrapper around xinference embedding models.
To use, you should have the xinference library installed:
pip install xinference
Check out: https://github.com/xorbitsai/inference
To run, you need to start a Xinference supervisor on one server and Xinference workers on the other servers.
Example
To start a local instance of Xinference, run
$ xinference
You can also deploy Xinference in a distributed cluster. Here are the steps:
Starting the supervisor:
$ xinference-supervisor
Starting the worker:
$ xinference-worker
Then, launch a model using command line interface (CLI).
Example:
$ xinference launch -n orca -s 3 -q q4_0
It will return a model UID. Then you can use Xinference Embedding with LangChain.
Example:
from langchain.embeddings import XinferenceEmbeddings
xinference = XinferenceEmbeddings(
    server_url="http://0.0.0.0:9997",
    model_uid = {model_uid} # replace model_uid with the model UID return from launching the model
)
Attributes
client
server_url
URL of the xinference server
model_uid
UID of the launched model
Methods
__init__([server_url, model_uid])
aembed_documents(texts)
Asynchronous Embed search docs.
aembed_query(text)
Asynchronous Embed query text.
embed_documents(texts)
Embed a list of documents using Xinference.
embed_query(text)
Embed a query of documents using Xinference.