langchain.embeddings.llamacpp.LlamaCppEmbeddings¶
class langchain.embeddings.llamacpp.LlamaCppEmbeddings[source]¶
Bases: BaseModel, Embeddings
llama.cpp embedding models.
To use, you should have the llama-cpp-python library installed, and provide the
path to the Llama model as a named parameter to the constructor.
Check out: https://github.com/abetlen/llama-cpp-python
Example
from langchain.embeddings import LlamaCppEmbeddings
llama = LlamaCppEmbeddings(model_path="/path/to/model.bin")
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param f16_kv: bool = False¶
Use half-precision for key/value cache.
param logits_all: bool = False¶
Return logits for all tokens, not just the last token.
param model_path: str [Required]¶
param n_batch: Optional[int] = 8¶
Number of tokens to process in parallel.
Should be a number between 1 and n_ctx.
param n_ctx: int = 512¶
Token context window.
param n_gpu_layers: Optional[int] = None¶
Number of layers to be loaded into gpu memory. Default None.
param n_parts: int = -1¶
Number of parts to split the model into.
If -1, the number of parts is automatically determined.
param n_threads: Optional[int] = None¶
Number of threads to use. If None, the number
of threads is automatically determined.
param seed: int = -1¶
Seed. If -1, a random seed is used.
param use_mlock: bool = False¶
Force system to keep model in RAM.
param verbose: bool = True¶