langchain.llms.textgen.TextGen¶
class langchain.llms.textgen.TextGen[source]¶
Bases: LLM
text-generation-webui models.
To use, you should have the text-generation-webui installed, a model loaded,
and –api added as a command-line option.
Suggested installation, use one-click installer for your OS:
https://github.com/oobabooga/text-generation-webui#one-click-installers
Parameters below taken from text-generation-webui api example:
https://github.com/oobabooga/text-generation-webui/blob/main/api-examples/api-example.py
Example
from langchain.llms import TextGen
llm = TextGen(model_url="http://localhost:8500")
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param add_bos_token: bool = True¶
Add the bos_token to the beginning of prompts.
Disabling this can make the replies more creative.
param ban_eos_token: bool = False¶
Ban the eos_token. Forces the model to never end the generation prematurely.
param cache: Optional[bool] = None¶
param callback_manager: Optional[BaseCallbackManager] = None¶
param callbacks: Callbacks = None¶
param do_sample: bool = True¶
Do sample
param early_stopping: bool = False¶
Early stopping
param epsilon_cutoff: Optional[float] = 0¶
Epsilon cutoff
param eta_cutoff: Optional[float] = 0¶
ETA cutoff
param length_penalty: Optional[float] = 1¶
Length Penalty
param max_new_tokens: Optional[int] = 250¶
The maximum number of tokens to generate.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.