langchain.llms.octoai_endpoint.OctoAIEndpoint¶
class langchain.llms.octoai_endpoint.OctoAIEndpoint[source]¶
Bases: LLM
OctoAI LLM Endpoints.
OctoAIEndpoint is a class to interact with OctoAICompute Service large language model endpoints.
To use, you should have the octoai python package installed, and the
environment variable OCTOAI_API_TOKEN set with your API token, or pass
it as a named parameter to the constructor.
Example
from langchain.llms.octoai_endpoint  import OctoAIEndpoint
OctoAIEndpoint(
    octoai_api_token="octoai-api-key",
    endpoint_url="https://mpt-7b-demo-kk0powt97tmb.octoai.cloud/generate",
    model_kwargs={
        "max_new_tokens": 200,
        "temperature": 0.75,
        "top_p": 0.95,
        "repetition_penalty": 1,
        "seed": None,
        "stop": [],
    },
)
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param cache: Optional[bool] = None¶
param callback_manager: Optional[BaseCallbackManager] = None¶
param callbacks: Callbacks = None¶
param endpoint_url: Optional[str] = None¶
Endpoint URL to use.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model_kwargs: Optional[dict] = None¶
Key word arguments to pass to the model.
param octoai_api_token: Optional[str] = None¶
OCTOAI API Token