Metadata to add to the run trace.
param min_length: Optional[int] = 0¶
Minimum generation length in tokens.
param model_url: str [Required]¶
The full URL to the textgen webui including http[s]://host:port
param no_repeat_ngram_size: Optional[int] = 0¶
If not set to 0, specifies the length of token sets that are completely blocked
from repeating at all. Higher values = blocks larger phrases,
lower values = blocks words or letters from repeating.
Only 0 or high values are a good idea in most cases.
param num_beams: Optional[int] = 1¶
Number of beams
param penalty_alpha: Optional[float] = 0¶
Penalty Alpha
param preset: Optional[str] = None¶
The preset to use in the textgen webui
param repetition_penalty: Optional[float] = 1.18¶
Exponential penalty factor for repeating prior tokens. 1 means no penalty,
higher value = less repetition, lower value = more repetition.
param seed: int = -1¶
Seed (-1 for random)
param skip_special_tokens: bool = True¶
Skip special tokens. Some specific models need this unset.
param stopping_strings: Optional[List[str]] = []¶
A list of strings to stop generation when encountered.
param streaming: bool = False¶
Whether to stream the results, token by token.
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param temperature: Optional[float] = 1.3¶
Primary factor to control randomness of outputs. 0 = deterministic
(only the most likely token is used). Higher value = more randomness.
param top_k: Optional[float] = 40¶
Similar to top_p, but select instead only the top_k most likely tokens.