langchain.llms.llamacpp.LlamaCpp¶
class langchain.llms.llamacpp.LlamaCpp[source]¶
Bases: LLM
llama.cpp model.
To use, you should have the llama-cpp-python library installed, and provide the
path to the Llama model as a named parameter to the constructor.
Check out: https://github.com/abetlen/llama-cpp-python
Example
from langchain.llms import LlamaCpp
llm = LlamaCpp(model_path="/path/to/llama/model")
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param cache: Optional[bool] = None¶
param callback_manager: Optional[BaseCallbackManager] = None¶
param callbacks: Callbacks = None¶
param echo: Optional[bool] = False¶
Whether to echo the prompt.
param f16_kv: bool = True¶
Use half-precision for key/value cache.
param grammar: Optional[Union[str, LlamaGrammar]] = None¶
grammar: formal grammar for constraining model outputs. For instance, the grammar
can be used to force the model to generate valid JSON or to speak exclusively in
emojis. At most one of grammar_path and grammar should be passed in.
param grammar_path: Optional[Union[str, Path]] = None¶
grammar_path: Path to the .gbnf file that defines formal grammars
for constraining model outputs. For instance, the grammar can be used
to force the model to generate valid JSON or to speak exclusively in emojis. At most
one of grammar_path and grammar should be passed in.
param last_n_tokens_size: Optional[int] = 64¶
The number of tokens to look back when applying the repeat_penalty.
param logits_all: bool = False¶