langchain.chat_models.anyscale.ChatAnyscale¶
class langchain.chat_models.anyscale.ChatAnyscale[source]¶
Bases: ChatOpenAI
Anyscale Chat large language models.
To use, you should have the openai python package installed, and the
environment variable ANYSCALE_API_KEY set with your API key.
Alternatively, you can use the anyscale_api_key keyword argument.
Any parameters that are valid to be passed to the openai.create call can be passed
in, even if not explicitly saved on this class.
Example
from langchain.chat_models import ChatAnyscale
chat = ChatAnyscale(model_name="meta-llama/Llama-2-7b-chat-hf")
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param anyscale_api_base: str = 'https://api.endpoints.anyscale.com/v1'¶
Base URL path for API requests,
leave blank if not using a proxy or service emulator.
param anyscale_api_key: Optional[str] = None¶
AnyScale Endpoints API keys.
param anyscale_proxy: Optional[str] = None¶
To support explicit proxy for Anyscale.
param available_models: Optional[Set[str]] = None¶
Available models from Anyscale API.
param cache: Optional[bool] = None¶
Whether to cache the response.
param callback_manager: Optional[BaseCallbackManager] = None¶
Callback manager to add to the run trace.
param callbacks: Callbacks = None¶
Callbacks to add to the run trace.
param max_retries: int = 6¶
Maximum number of retries to make when generating.
param max_tokens: Optional[int] = None¶
Maximum number of tokens to generate.
param metadata: Optional[Dict[str, Any]] = None¶