Model inference on self-hosted remote hardware.
llms.self_hosted_hugging_face.SelfHostedHuggingFaceLLM
HuggingFace Pipeline API to run on self-hosted remote hardware.
llms.stochasticai.StochasticAI
StochasticAI large language models.
llms.symblai_nebula.Nebula
Nebula Service models.
llms.textgen.TextGen
text-generation-webui models.
llms.titan_takeoff.TitanTakeoff
Create a new model by parsing and validating input data from keyword arguments.
llms.tongyi.Tongyi
Tongyi Qwen large language models.
llms.vertexai.VertexAI
Google Vertex AI large language models.
llms.vertexai.VertexAIModelGarden
Large language models served from Vertex AI Model Garden.
llms.vllm.VLLM
VLLM language model.
llms.vllm.VLLMOpenAI
vLLM OpenAI-compatible API client
llms.writer.Writer
Writer large language models.
llms.xinference.Xinference
Wrapper for accessing Xinference's large-scale model inference service.
Functions¶
llms.aviary.get_completions(model, prompt[, ...])
Get completions from Aviary models.
llms.aviary.get_models()
List available models
llms.base.create_base_retry_decorator(...[, ...])
Create a retry decorator for a given LLM and provided list of error types.
llms.base.get_prompts(params, prompts)
Get prompts that are already cached.
llms.base.update_cache(existing_prompts, ...)
Update the cache and get the LLM output.
llms.cohere.acompletion_with_retry(llm, **kwargs)
Use tenacity to retry the completion call.