langchain.llms.ctranslate2.CTranslate2¶
class langchain.llms.ctranslate2.CTranslate2[source]¶
Bases: BaseLLM
CTranslate2 language model.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param cache: Optional[bool] = None¶
param callback_manager: Optional[BaseCallbackManager] = None¶
param callbacks: Callbacks = None¶
param compute_type: Union[str, Dict[str, str]] = 'default'¶
Model computation type or a dictionary mapping a device name to the computation type
(possible values are: default, auto, int8, int8_float32, int8_float16,
int8_bfloat16, int16, float16, bfloat16, float32).
param ctranslate2_kwargs: Dict[str, Any] [Optional]¶
Holds any model parameters valid for ctranslate2.Generator call not
explicitly specified.
param device: str = 'cpu'¶
Device to use (possible values are: cpu, cuda, auto).
param device_index: Union[int, List[int]] = 0¶
Device IDs where to place this generator on.
param max_length: int = 512¶
Maximum generation length.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model_path: str = ''¶
Path to the CTranslate2 model directory.
param sampling_temperature: float = 1¶
Sampling temperature to generate more random samples.
param sampling_topk: int = 1¶
Randomly sample predictions from the top K candidates.
param sampling_topp: float = 1¶
Keep the most probable tokens whose cumulative probability exceeds this value.