param truncate: Optional[int] = None¶
Truncate inputs tokens to the given size
param typical_p: Optional[float] = 0.95¶
Typical Decoding mass. See [Typical Decoding for Natural Language
Generation](https://arxiv.org/abs/2202.00666) for more information.
param verbose: bool [Optional]¶
Whether to print out response text.
param watermark: bool = False¶
Watermarking with [A Watermark for Large Language Models]
(https://arxiv.org/abs/2301.10226)
__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str¶
Check Cache and run the LLM on the given prompt and input.
async abatch(inputs: List[Union[PromptValue, str, List[BaseMessage]]], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Any) → List[str]¶
Default implementation of abatch, which calls ainvoke N times.
Subclasses should override this method if they can batch more efficiently.