langchain.vectorstores.matching_engine.MatchingEngine¶
class langchain.vectorstores.matching_engine.MatchingEngine(project_id: str, index: MatchingEngineIndex, endpoint: MatchingEngineIndexEndpoint, embedding: Embeddings, gcs_client: storage.Client, gcs_bucket_name: str, credentials: Optional[Credentials] = None)[source]¶
Google Vertex AI Matching Engine vector store.
While the embeddings are stored in the Matching Engine, the embedded
documents will be stored in GCS.
An existing Index and corresponding Endpoint are preconditions for
using this module.
See usage in docs/modules/indexes/vectorstores/examples/matchingengine.ipynb
Note that this implementation is mostly meant for reading if you are
planning to do a real time implementation. While reading is a real time
operation, updating the index takes close to one hour.
Vertex Matching Engine implementation of the vector store.
While the embeddings are stored in the Matching Engine, the embedded
documents will be stored in GCS.
An existing Index and corresponding Endpoint are preconditions for
using this module.
See usage in
docs/modules/indexes/vectorstores/examples/matchingengine.ipynb.
Note that this implementation is mostly meant for reading if you are
planning to do a real time implementation. While reading is a real time
operation, updating the index takes close to one hour.
project_id¶
The GCS project id.
index¶
The created index class. See
~:func:MatchingEngine.from_components.
endpoint¶
The created endpoint class. See
~:func:MatchingEngine.from_components.
embedding¶
A Embeddings that will be used for
embedding the text sent. If none is sent, then the
multilingual Tensorflow Universal Sentence Encoder will be used.
gcs_client¶
The GCS client.
gcs_bucket_name¶
The GCS bucket name.
credentials¶


gcs_bucket_name¶
The GCS bucket name.
credentials¶
Created GCP credentials.
Type
Optional
Attributes
embeddings
Access the query embedding object if available.
Methods
__init__(project_id, index, endpoint, ...[, ...])
Vertex Matching Engine implementation of the vector store.
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_texts(texts, embedding[, metadatas])
Return VectorStore initialized from texts and embeddings.
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs most similar to query.
delete([ids])
Delete by vector ID or other criteria.


delete([ids])
Delete by vector ID or other criteria.
from_components(project_id, region, ...[, ...])
Takes the object creation out of the constructor.
from_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
from_texts(texts, embedding[, metadatas])
Use from components instead.
max_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
max_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
search(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
similarity_search(query[, k])
Return docs most similar to query.
similarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
similarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
similarity_search_with_score(*args, **kwargs)
Run similarity search with distance.
__init__(project_id: str, index: MatchingEngineIndex, endpoint: MatchingEngineIndexEndpoint, embedding: Embeddings, gcs_client: storage.Client, gcs_bucket_name: str, credentials: Optional[Credentials] = None)[source]¶
Vertex Matching Engine implementation of the vector store.
While the embeddings are stored in the Matching Engine, the embedded
documents will be stored in GCS.
An existing Index and corresponding Endpoint are preconditions for
using this module.
See usage in
docs/modules/indexes/vectorstores/examples/matchingengine.ipynb.
Note that this implementation is mostly meant for reading if you are
planning to do a real time implementation. While reading is a real time
operation, updating the index takes close to one hour.


operation, updating the index takes close to one hour.
project_id¶
The GCS project id.
index¶
The created index class. See
~:func:MatchingEngine.from_components.
endpoint¶
The created endpoint class. See
~:func:MatchingEngine.from_components.
embedding¶
A Embeddings that will be used for
embedding the text sent. If none is sent, then the
multilingual Tensorflow Universal Sentence Encoder will be used.
gcs_client¶
The GCS client.
gcs_bucket_name¶
The GCS bucket name.
credentials¶
Created GCP credentials.
Type
Optional
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str]¶
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
add_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str][source]¶
Run more texts through the embeddings and add to the vectorstore.
Parameters
texts – Iterable of strings to add to the vectorstore.


Parameters
texts – Iterable of strings to add to the vectorstore.
metadatas – Optional list of metadatas associated with the texts.
kwargs – vectorstore specific parameters.
Returns
List of ids from adding the texts into the vectorstore.
async classmethod afrom_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
async classmethod afrom_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, **kwargs: Any) → VST¶
Return VectorStore initialized from texts and embeddings.
async amax_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
async amax_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs: Any) → VectorStoreRetriever¶
Return VectorStoreRetriever initialized from this VectorStore.
Parameters
search_type (Optional[str]) – Defines the type of search that
the Retriever should perform.
Can be “similarity” (default), “mmr”, or
“similarity_score_threshold”.
search_kwargs (Optional[Dict]) – Keyword arguments to pass to the
search function. Can include things like:
k: Amount of documents to return (Default: 4)
score_threshold: Minimum relevance threshold
for similarity_score_threshold


score_threshold: Minimum relevance threshold
for similarity_score_threshold
fetch_k: Amount of documents to pass to MMR algorithm (Default: 20)
lambda_mult: Diversity of results returned by MMR;
1 for minimum diversity and 0 for maximum. (Default: 0.5)
filter: Filter by document metadata
Returns
Retriever class for VectorStore.
Return type
VectorStoreRetriever
Examples:
# Retrieve more documents with higher diversity
# Useful if your dataset has many similar documents
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 6, 'lambda_mult': 0.25}
)
# Fetch more documents for the MMR algorithm to consider
# But only return the top 5
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 5, 'fetch_k': 50}
)
# Only retrieve documents that have a relevance score
# Above a certain threshold
docsearch.as_retriever(
    search_type="similarity_score_threshold",
    search_kwargs={'score_threshold': 0.8}
)
# Only get the single most similar document from the dataset
docsearch.as_retriever(search_kwargs={'k': 1})
# Use a filter to only retrieve documents from a specific paper
docsearch.as_retriever(
    search_kwargs={'filter': {'paper_title':'GPT-4 Technical Report'}}
)
async asearch(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
async asimilarity_search(query: str, k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to query.


Return docs most similar to query.
async asimilarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to embedding vector.
async asimilarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs most similar to query.
delete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids – List of ids to delete.
**kwargs – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
classmethod from_components(project_id: str, region: str, gcs_bucket_name: str, index_id: str, endpoint_id: str, credentials_path: Optional[str] = None, embedding: Optional[Embeddings] = None) → MatchingEngine[source]¶
Takes the object creation out of the constructor.
Parameters
project_id – The GCP project id.
region – The default location making the API calls. It must have
regional. (the same location as the GCS bucket and must be) –
gcs_bucket_name – The location where the vectors will be stored in
created. (order for the index to be) –
index_id – The id of the created index.
endpoint_id – The id of the created endpoint.
credentials_path – (Optional) The path of the Google credentials on
system. (the local file) –
embedding – The Embeddings that will be used for
texts. (embedding the) –
Returns
A configured MatchingEngine with the texts added to the index.


Returns
A configured MatchingEngine with the texts added to the index.
classmethod from_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
classmethod from_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, **kwargs: Any) → MatchingEngine[source]¶
Use from components instead.
max_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
query – Text to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
max_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.


fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
search(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
similarity_search(query: str, k: int = 4, **kwargs: Any) → List[Document][source]¶
Return docs most similar to query.
Parameters
query – The string that will be used to search for similar documents.
k – The amount of neighbors that will be retrieved.
Returns
A list of k matching documents.
similarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to embedding vector.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
Returns
List of Documents most similar to the query vector.
similarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query – input text
k – Number of Documents to return. Defaults to 4.
**kwargs – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)


Returns
List of Tuples of (doc, similarity_score)
similarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance.
Examples using MatchingEngine¶
Google Vertex AI MatchingEngine


langchain.vectorstores.elasticsearch.BaseRetrievalStrategy¶
class langchain.vectorstores.elasticsearch.BaseRetrievalStrategy[source]¶
Base class for Elasticsearch retrieval strategies.
Methods
__init__()
before_index_setup(client, text_field, ...)
Executes before the index is created.
index(dims_length, vector_query_field, ...)
Executes when the index is created.
query(query_vector, query, *, k, fetch_k, ...)
Executes when a search is performed on the store.
require_inference()
Returns whether or not the strategy requires inference to be performed on the text before it is added to the index.
__init__()¶
before_index_setup(client: Elasticsearch, text_field: str, vector_query_field: str) → None[source]¶
Executes before the index is created. Used for setting up
any required Elasticsearch resources like a pipeline.
Parameters
client – The Elasticsearch client.
text_field – The field containing the text data in the index.
vector_query_field – The field containing the vector
representations in the index.
abstract index(dims_length: Optional[int], vector_query_field: str, similarity: Optional[DistanceStrategy]) → Dict[source]¶
Executes when the index is created.
Parameters
dims_length – Numeric length of the embedding vectors,
or None if not using vector-based query.
vector_query_field – The field containing the vector
representations in the index.
similarity – The similarity strategy to use,
or None if not using one.
Returns
The Elasticsearch settings and mappings for the strategy.
Return type
Dict
abstract query(query_vector: Optional[List[float]], query: Optional[str], *, k: int, fetch_k: int, vector_query_field: str, text_field: str, filter: List[dict], similarity: Optional[DistanceStrategy]) → Dict[source]¶


Executes when a search is performed on the store.
Parameters
query_vector – The query vector,
or None if not using vector-based query.
query – The text query, or None if not using text-based query.
k – The total number of results to retrieve.
fetch_k – The number of results to fetch initially.
vector_query_field – The field containing the vector
representations in the index.
text_field – The field containing the text data in the index.
filter – List of filter clauses to apply to the query.
similarity – The similarity strategy to use, or None if not using one.
Returns
The Elasticsearch query body.
Return type
Dict
require_inference() → bool[source]¶
Returns whether or not the strategy requires inference
to be performed on the text before it is added to the index.
Returns
Whether or not the strategy requires inference
to be performed on the text before it is added to the index.
Return type
bool


langchain.vectorstores.typesense.Typesense¶
class langchain.vectorstores.typesense.Typesense(typesense_client: Client, embedding: Embeddings, *, typesense_collection_name: Optional[str] = None, text_key: str = 'text')[source]¶
Typesense vector store.
To use, you should have the typesense python package installed.
Example
from langchain.embedding.openai import OpenAIEmbeddings
from langchain.vectorstores import Typesense
import typesense
node = {
    "host": "localhost",  # For Typesense Cloud use xxx.a1.typesense.net
    "port": "8108",       # For Typesense Cloud use 443
    "protocol": "http"    # For Typesense Cloud use https
}
typesense_client = typesense.Client(
    {
      "nodes": [node],
      "api_key": "<API_KEY>",
      "connection_timeout_seconds": 2
    }
)
typesense_collection_name = "langchain-memory"
embedding = OpenAIEmbeddings()
vectorstore = Typesense(
    typesense_client=typesense_client,
    embedding=embedding,
    typesense_collection_name=typesense_collection_name,
    text_key="text",
)
Initialize with Typesense client.
Attributes
embeddings
Access the query embedding object if available.
Methods
__init__(typesense_client, embedding, *[, ...])
Initialize with Typesense client.
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.


Run more documents through the embeddings and add to the vectorstore.
add_texts(texts[, metadatas, ids])
Run more texts through the embedding and add to the vectorstore.
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_texts(texts, embedding[, metadatas])
Return VectorStore initialized from texts and embeddings.
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs most similar to query.
delete([ids])
Delete by vector ID or other criteria.
from_client_params(embedding, *[, host, ...])
Initialize Typesense directly from client parameters.
from_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
from_texts(texts, embedding[, metadatas, ...])
Construct Typesense wrapper from raw text.
max_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
max_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
search(query, search_type, **kwargs)


search(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
similarity_search(query[, k, filter])
Return typesense documents most similar to query.
similarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
similarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
similarity_search_with_score(query[, k, filter])
Return typesense documents most similar to query, along with scores.
__init__(typesense_client: Client, embedding: Embeddings, *, typesense_collection_name: Optional[str] = None, text_key: str = 'text')[source]¶
Initialize with Typesense client.
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str]¶
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
add_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, **kwargs: Any) → List[str][source]¶


Run more texts through the embedding and add to the vectorstore.
Parameters
texts – Iterable of strings to add to the vectorstore.
metadatas – Optional list of metadatas associated with the texts.
ids – Optional list of ids to associate with the texts.
Returns
List of ids from adding the texts into the vectorstore.
async classmethod afrom_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
async classmethod afrom_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, **kwargs: Any) → VST¶
Return VectorStore initialized from texts and embeddings.
async amax_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
async amax_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs: Any) → VectorStoreRetriever¶
Return VectorStoreRetriever initialized from this VectorStore.
Parameters
search_type (Optional[str]) – Defines the type of search that
the Retriever should perform.
Can be “similarity” (default), “mmr”, or
“similarity_score_threshold”.
search_kwargs (Optional[Dict]) – Keyword arguments to pass to the
search function. Can include things like:
k: Amount of documents to return (Default: 4)


k: Amount of documents to return (Default: 4)
score_threshold: Minimum relevance threshold
for similarity_score_threshold
fetch_k: Amount of documents to pass to MMR algorithm (Default: 20)
lambda_mult: Diversity of results returned by MMR;
1 for minimum diversity and 0 for maximum. (Default: 0.5)
filter: Filter by document metadata
Returns
Retriever class for VectorStore.
Return type
VectorStoreRetriever
Examples:
# Retrieve more documents with higher diversity
# Useful if your dataset has many similar documents
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 6, 'lambda_mult': 0.25}
)
# Fetch more documents for the MMR algorithm to consider
# But only return the top 5
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 5, 'fetch_k': 50}
)
# Only retrieve documents that have a relevance score
# Above a certain threshold
docsearch.as_retriever(
    search_type="similarity_score_threshold",
    search_kwargs={'score_threshold': 0.8}
)
# Only get the single most similar document from the dataset
docsearch.as_retriever(search_kwargs={'k': 1})
# Use a filter to only retrieve documents from a specific paper
docsearch.as_retriever(
    search_kwargs={'filter': {'paper_title':'GPT-4 Technical Report'}}
)
async asearch(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
async asimilarity_search(query: str, k: int = 4, **kwargs: Any) → List[Document]¶


Return docs most similar to query.
async asimilarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to embedding vector.
async asimilarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs most similar to query.
delete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids – List of ids to delete.
**kwargs – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
classmethod from_client_params(embedding: Embeddings, *, host: str = 'localhost', port: Union[str, int] = '8108', protocol: str = 'http', typesense_api_key: Optional[str] = None, connection_timeout_seconds: int = 2, **kwargs: Any) → Typesense[source]¶
Initialize Typesense directly from client parameters.
Example
from langchain.embedding.openai import OpenAIEmbeddings
from langchain.vectorstores import Typesense
# Pass in typesense_api_key as kwarg or set env var "TYPESENSE_API_KEY".
vectorstore = Typesense(
    OpenAIEmbeddings(),
    host="localhost",
    port="8108",
    protocol="http",
    typesense_collection_name="langchain-memory",
)
classmethod from_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.


Return VectorStore initialized from documents and embeddings.
classmethod from_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, typesense_client: Optional[Client] = None, typesense_client_params: Optional[dict] = None, typesense_collection_name: Optional[str] = None, text_key: str = 'text', **kwargs: Any) → Typesense[source]¶
Construct Typesense wrapper from raw text.
max_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
query – Text to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
max_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.


k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
search(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
similarity_search(query: str, k: int = 10, filter: Optional[str] = '', **kwargs: Any) → List[Document][source]¶
Return typesense documents most similar to query.
Parameters
query – Text to look up documents similar to.
k – Number of Documents to return. Defaults to 10.
Minimum 10 results would be returned.
filter – typesense filter_by expression to filter documents on
Returns
List of Documents most similar to the query and score for each
similarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to embedding vector.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
Returns
List of Documents most similar to the query vector.
similarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query – input text
k – Number of Documents to return. Defaults to 4.


query – input text
k – Number of Documents to return. Defaults to 4.
**kwargs – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
similarity_search_with_score(query: str, k: int = 10, filter: Optional[str] = '') → List[Tuple[Document, float]][source]¶
Return typesense documents most similar to query, along with scores.
Parameters
query – Text to look up documents similar to.
k – Number of Documents to return. Defaults to 10.
Minimum 10 results would be returned.
filter – typesense filter_by expression to filter documents on
Returns
List of Documents most similar to the query and score for each
Examples using Typesense¶
Typesense


langchain.vectorstores.marqo.Marqo¶
class langchain.vectorstores.marqo.Marqo(client: marqo.Client, index_name: str, add_documents_settings: Optional[Dict[str, Any]] = None, searchable_attributes: Optional[List[str]] = None, page_content_builder: Optional[Callable[[Dict[str, Any]], str]] = None)[source]¶
Marqo vector store.
Marqo indexes have their own models associated with them to generate your
embeddings. This means that you can selected from a range of different models
and also use CLIP models to create multimodal indexes
with images and text together.
Marqo also supports more advanced queries with multiple weighted terms, see See
https://docs.marqo.ai/latest/#searching-using-weights-in-queries.
This class can flexibly take strings or dictionaries for weighted queries
in its similarity search methods.
To use, you should have the marqo python package installed, you can do this with
pip install marqo.
Example
import marqo
from langchain.vectorstores import Marqo
client = marqo.Client(url=os.environ["MARQO_URL"], ...)
vectorstore = Marqo(client, index_name)
Initialize with Marqo client.
Attributes
embeddings
Access the query embedding object if available.
Methods
__init__(client, index_name[, ...])
Initialize with Marqo client.
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_texts(texts[, metadatas])


add_texts(texts[, metadatas])
Upload texts with metadata (properties) to Marqo.
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_texts(texts, embedding[, metadatas])
Return VectorStore initialized from texts and embeddings.
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs most similar to query.
bulk_similarity_search(queries[, k])
Search the marqo index for the most similar documents in bulk with multiple queries.
bulk_similarity_search_with_score(queries[, k])
Return documents from Marqo that are similar to the query as well as their scores using a batch of queries.
delete([ids])
Delete by vector ID or other criteria.
from_documents(documents[, embedding])
Return VectorStore initialized from documents.
from_texts(texts[, embedding, metadatas, ...])
Return Marqo initialized from texts.
get_indexes()
Helper to see your available indexes in marqo, useful if the from_texts method was used without an index name specified
get_number_of_documents()
Helper to see the number of documents in the index


get_number_of_documents()
Helper to see the number of documents in the index
marqo_bulk_similarity_search(queries[, k])
Return documents from Marqo using a bulk search, exposes Marqo's output directly
marqo_similarity_search(query[, k])
Return documents from Marqo exposing Marqo's output directly
max_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
max_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
search(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
similarity_search(query[, k])
Search the marqo index for the most similar documents.
similarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
similarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
similarity_search_with_score(query[, k])
Return documents from Marqo that are similar to the query as well as their scores.
__init__(client: marqo.Client, index_name: str, add_documents_settings: Optional[Dict[str, Any]] = None, searchable_attributes: Optional[List[str]] = None, page_content_builder: Optional[Callable[[Dict[str, Any]], str]] = None)[source]¶
Initialize with Marqo client.
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]


Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str]¶
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
add_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str][source]¶
Upload texts with metadata (properties) to Marqo.
You can either have marqo generate ids for each document or you can provide
your own by including a “_id” field in the metadata objects.
Parameters
texts (Iterable[str]) – am iterator of texts - assumed to preserve an
metadatas. (order that matches the) –
metadatas (Optional[List[dict]], optional) – a list of metadatas.
Raises
ValueError – if metadatas is provided and the number of metadatas differs
from the number of texts. –
Returns
The list of ids that were added.
Return type
List[str]
async classmethod afrom_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
async classmethod afrom_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, **kwargs: Any) → VST¶
Return VectorStore initialized from texts and embeddings.


Return VectorStore initialized from texts and embeddings.
async amax_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
async amax_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs: Any) → VectorStoreRetriever¶
Return VectorStoreRetriever initialized from this VectorStore.
Parameters
search_type (Optional[str]) – Defines the type of search that
the Retriever should perform.
Can be “similarity” (default), “mmr”, or
“similarity_score_threshold”.
search_kwargs (Optional[Dict]) – Keyword arguments to pass to the
search function. Can include things like:
k: Amount of documents to return (Default: 4)
score_threshold: Minimum relevance threshold
for similarity_score_threshold
fetch_k: Amount of documents to pass to MMR algorithm (Default: 20)
lambda_mult: Diversity of results returned by MMR;
1 for minimum diversity and 0 for maximum. (Default: 0.5)
filter: Filter by document metadata
Returns
Retriever class for VectorStore.
Return type
VectorStoreRetriever
Examples:
# Retrieve more documents with higher diversity
# Useful if your dataset has many similar documents
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 6, 'lambda_mult': 0.25}
)


)
# Fetch more documents for the MMR algorithm to consider
# But only return the top 5
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 5, 'fetch_k': 50}
)
# Only retrieve documents that have a relevance score
# Above a certain threshold
docsearch.as_retriever(
    search_type="similarity_score_threshold",
    search_kwargs={'score_threshold': 0.8}
)
# Only get the single most similar document from the dataset
docsearch.as_retriever(search_kwargs={'k': 1})
# Use a filter to only retrieve documents from a specific paper
docsearch.as_retriever(
    search_kwargs={'filter': {'paper_title':'GPT-4 Technical Report'}}
)
async asearch(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
async asimilarity_search(query: str, k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to query.
async asimilarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to embedding vector.
async asimilarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs most similar to query.
bulk_similarity_search(queries: Iterable[Union[str, Dict[str, float]]], k: int = 4, **kwargs: Any) → List[List[Document]][source]¶
Search the marqo index for the most similar documents in bulk with multiple
queries.
Parameters


queries.
Parameters
queries (Iterable[Union[str, Dict[str, float]]]) – An iterable of queries to
bulk (execute in) –
of (queries in the list can be strings or dictionaries) –
queries. (weighted) –
k (int, optional) – The number of documents to return for each query.
4. (Defaults to) –
Returns
A list of results for each query.
Return type
List[List[Document]]
bulk_similarity_search_with_score(queries: Iterable[Union[str, Dict[str, float]]], k: int = 4, **kwargs: Any) → List[List[Tuple[Document, float]]][source]¶
Return documents from Marqo that are similar to the query as well as
their scores using a batch of queries.
Parameters
query (Iterable[Union[str, Dict[str, float]]]) – An iterable of queries
bulk (to execute in) –
dictionaries (queries in the list can be strings or) –
queries. (of weighted) –
k (int, optional) – The number of documents to return. Defaults to 4.
Returns
A list of lists of the matching
documents and their scores for each query
Return type
List[Tuple[Document, float]]
delete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids – List of ids to delete.
**kwargs – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
classmethod from_documents(documents: List[Document], embedding: Optional[Embeddings] = None, **kwargs: Any) → Marqo[source]¶


Return VectorStore initialized from documents. Note that Marqo does not
need embeddings, we retain the parameter to adhere to the Liskov substitution
principle.
Parameters
documents (List[Document]) – Input documents
embedding (Any, optional) – Embeddings (not required). Defaults to None.
Returns
A Marqo vectorstore
Return type
VectorStore
classmethod from_texts(texts: List[str], embedding: Any = None, metadatas: Optional[List[dict]] = None, index_name: str = '', url: str = 'http://localhost:8882', api_key: str = '', add_documents_settings: Optional[Dict[str, Any]] = None, searchable_attributes: Optional[List[str]] = None, page_content_builder: Optional[Callable[[Dict[str, str]], str]] = None, index_settings: Optional[Dict[str, Any]] = None, verbose: bool = True, **kwargs: Any) → Marqo[source]¶
Return Marqo initialized from texts. Note that Marqo does not need
embeddings, we retain the parameter to adhere to the Liskov
substitution principle.
This is a quick way to get started with marqo - simply provide your texts and
metadatas and this will create an instance of the data store and index the
provided data.
To know the ids of your documents with this approach you will need to include
them in under the key “_id” in your metadatas for each text
Example:
.. code-block:: python
from langchain.vectorstores import Marqo
datastore = Marqo(texts=[‘text’], index_name=’my-first-index’,
url=’http://localhost:8882’)
Parameters
texts (List[str]) – A list of texts to index into marqo upon creation.


texts (List[str]) – A list of texts to index into marqo upon creation.
embedding (Any, optional) – Embeddings (not required). Defaults to None.
index_name (str, optional) – The name of the index to use, if none is
None. (accompany the texts. Defaults to) –
url (str, optional) – The URL for Marqo. Defaults to “http://localhost:8882”.
api_key (str, optional) – The API key for Marqo. Defaults to “”.
metadatas (Optional[List[dict]], optional) – A list of metadatas, to
None. –
Can (this is only used when a new index is being created. Defaults to "cpu".) –
"cuda". (be "cpu" or) –
add_documents_settings (Optional[Dict[str, Any]], optional) – Settings
documents (for adding) –
see –
https – //docs.marqo.ai/0.0.16/API-Reference/documents/#query-parameters.
{}. (Defaults to) –
index_settings (Optional[Dict[str, Any]], optional) – Index settings if
exist (the index doesn't) –
see –
https – //docs.marqo.ai/0.0.16/API-Reference/indexes/#index-defaults-object.
{}. –
Returns
An instance of the Marqo vector store
Return type
Marqo
get_indexes() → List[Dict[str, str]][source]¶
Helper to see your available indexes in marqo, useful if the
from_texts method was used without an index name specified
Returns
The list of indexes
Return type
List[Dict[str, str]]
get_number_of_documents() → int[source]¶


List[Dict[str, str]]
get_number_of_documents() → int[source]¶
Helper to see the number of documents in the index
Returns
The number of documents
Return type
int
marqo_bulk_similarity_search(queries: Iterable[Union[str, Dict[str, float]]], k: int = 4) → Dict[str, List[Dict[str, List[Dict[str, str]]]]][source]¶
Return documents from Marqo using a bulk search, exposes Marqo’s
output directly
Parameters
queries (Iterable[Union[str, Dict[str, float]]]) – A list of queries.
k (int, optional) – The number of documents to return for each query.
4. (Defaults to) –
Returns
A bulk search results
object
Return type
Dict[str, Dict[List[Dict[str, Dict[str, Any]]]]]
marqo_similarity_search(query: Union[str, Dict[str, float]], k: int = 4) → Dict[str, List[Dict[str, str]]][source]¶
Return documents from Marqo exposing Marqo’s output directly
Parameters
query (str) – The query to search with.
k (int, optional) – The number of documents to return. Defaults to 4.
Returns
This hits from marqo.
Return type
List[Dict[str, Any]]
max_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
query – Text to look up documents similar to.
k – Number of Documents to return. Defaults to 4.


k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
max_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
search(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
similarity_search(query: Union[str, Dict[str, float]], k: int = 4, **kwargs: Any) → List[Document][source]¶
Search the marqo index for the most similar documents.
Parameters
query (Union[str, Dict[str, float]]) – The query for the search, either
query. (as a string or a weighted) –


query. (as a string or a weighted) –
k (int, optional) – The number of documents to return. Defaults to 4.
Returns
k documents ordered from best to worst match.
Return type
List[Document]
similarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to embedding vector.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
Returns
List of Documents most similar to the query vector.
similarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query – input text
k – Number of Documents to return. Defaults to 4.
**kwargs – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
similarity_search_with_score(query: Union[str, Dict[str, float]], k: int = 4) → List[Tuple[Document, float]][source]¶
Return documents from Marqo that are similar to the query as well
as their scores.
Parameters
query (str) – The query to search with, either as a string or a weighted
query. –
k (int, optional) – The number of documents to return. Defaults to 4.
Returns
The matching documents and their scores,
ordered by descending score.
Return type
List[Tuple[Document, float]]


ordered by descending score.
Return type
List[Tuple[Document, float]]
Examples using Marqo¶
Marqo


langchain.vectorstores.annoy.dependable_annoy_import¶
langchain.vectorstores.annoy.dependable_annoy_import() → Any[source]¶
Import annoy if available, otherwise raise error.


langchain.vectorstores.llm_rails.ModelChoices¶
class langchain.vectorstores.llm_rails.ModelChoices(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]¶
embedding_english_v1 = 'embedding-english-v1'¶
embedding_multi_v1 = 'embedding-multi-v1'¶


langchain.vectorstores.clickhouse.has_mul_sub_str¶
langchain.vectorstores.clickhouse.has_mul_sub_str(s: str, *args: Any) → bool[source]¶
Check if a string contains multiple substrings.
:param s: string to check.
:param *args: substrings to check.
Returns
True if all substrings are in the string, False otherwise.


langchain.vectorstores.rocksetdb.Rockset¶
class langchain.vectorstores.rocksetdb.Rockset(client: Any, embeddings: Embeddings, collection_name: str, text_key: str, embedding_key: str, workspace: str = 'commons')[source]¶
Rockset vector store.
To use, you should have the rockset python package installed. Note that to use
this, the collection being used must already exist in your Rockset instance.
You must also ensure you use a Rockset ingest transformation to apply
VECTOR_ENFORCE on the column being used to store embedding_key in the
collection.
See: https://rockset.com/blog/introducing-vector-search-on-rockset/ for more details
Everything below assumes commons Rockset workspace.
Example
from langchain.vectorstores import Rockset
from langchain.embeddings.openai import OpenAIEmbeddings
import rockset
# Make sure you use the right host (region) for your Rockset instance
# and APIKEY has both read-write access to your collection.
rs = rockset.RocksetClient(host=rockset.Regions.use1a1, api_key="***")
collection_name = "langchain_demo"
embeddings = OpenAIEmbeddings()
vectorstore = Rockset(rs, collection_name, embeddings,
    "description", "description_embedding")
Initialize with Rockset client.
:param client: Rockset client object
:param collection: Rockset collection to insert docs / query
:param embeddings: Langchain Embeddings object to use to generate
embedding for given text.
Parameters
text_key – column in Rockset collection to use to store the text
embedding_key – column in Rockset collection to use to store the embedding.
Note: We must apply VECTOR_ENFORCE() on this column via
Rockset ingest transformation.
Attributes
embeddings
Access the query embedding object if available.
Methods


Attributes
embeddings
Access the query embedding object if available.
Methods
__init__(client, embeddings, ...[, workspace])
Initialize with Rockset client. :param client: Rockset client object :param collection: Rockset collection to insert docs / query :param embeddings: Langchain Embeddings object to use to generate                    embedding for given text. :param text_key: column in Rockset collection to use to store the text :param embedding_key: column in Rockset collection to use to store the embedding. Note: We must apply VECTOR_ENFORCE() on this column via                       Rockset ingest transformation.
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_texts(texts[, metadatas, ids, batch_size])
Run more texts through the embeddings and add to the vectorstore
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_texts(texts, embedding[, metadatas])
Return VectorStore initialized from texts and embeddings.
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.


asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs most similar to query.
delete([ids])
Delete by vector ID or other criteria.
delete_texts(ids)
Delete a list of docs from the Rockset collection
from_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
from_texts(texts, embedding[, metadatas, ...])
Create Rockset wrapper with existing texts.
max_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
max_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
search(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
similarity_search(query[, k, distance_func, ...])
Same as similarity_search_with_relevance_scores but doesn't return the scores.
similarity_search_by_vector(embedding[, k, ...])
Accepts a query_embedding (vector), and returns documents with similar embeddings.
similarity_search_by_vector_with_relevance_scores(...)
Accepts a query_embedding (vector), and returns documents with similar embeddings along with their relevance scores.
similarity_search_with_relevance_scores(query)
Perform a similarity search with Rockset
similarity_search_with_score(*args, **kwargs)
Run similarity search with distance.
__init__(client: Any, embeddings: Embeddings, collection_name: str, text_key: str, embedding_key: str, workspace: str = 'commons')[source]¶
Initialize with Rockset client.
:param client: Rockset client object


Initialize with Rockset client.
:param client: Rockset client object
:param collection: Rockset collection to insert docs / query
:param embeddings: Langchain Embeddings object to use to generate
embedding for given text.
Parameters
text_key – column in Rockset collection to use to store the text
embedding_key – column in Rockset collection to use to store the embedding.
Note: We must apply VECTOR_ENFORCE() on this column via
Rockset ingest transformation.
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str]¶
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
add_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, batch_size: int = 32, **kwargs: Any) → List[str][source]¶
Run more texts through the embeddings and add to the vectorstore
Args:
texts: Iterable of strings to add to the vectorstore.
metadatas: Optional list of metadatas associated with the texts.
ids: Optional list of ids to associate with the texts.


ids: Optional list of ids to associate with the texts.
batch_size: Send documents in batches to rockset.
Returns
List of ids from adding the texts into the vectorstore.
async classmethod afrom_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
async classmethod afrom_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, **kwargs: Any) → VST¶
Return VectorStore initialized from texts and embeddings.
async amax_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
async amax_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs: Any) → VectorStoreRetriever¶
Return VectorStoreRetriever initialized from this VectorStore.
Parameters
search_type (Optional[str]) – Defines the type of search that
the Retriever should perform.
Can be “similarity” (default), “mmr”, or
“similarity_score_threshold”.
search_kwargs (Optional[Dict]) – Keyword arguments to pass to the
search function. Can include things like:
k: Amount of documents to return (Default: 4)
score_threshold: Minimum relevance threshold
for similarity_score_threshold
fetch_k: Amount of documents to pass to MMR algorithm (Default: 20)


fetch_k: Amount of documents to pass to MMR algorithm (Default: 20)
lambda_mult: Diversity of results returned by MMR;
1 for minimum diversity and 0 for maximum. (Default: 0.5)
filter: Filter by document metadata
Returns
Retriever class for VectorStore.
Return type
VectorStoreRetriever
Examples:
# Retrieve more documents with higher diversity
# Useful if your dataset has many similar documents
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 6, 'lambda_mult': 0.25}
)
# Fetch more documents for the MMR algorithm to consider
# But only return the top 5
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 5, 'fetch_k': 50}
)
# Only retrieve documents that have a relevance score
# Above a certain threshold
docsearch.as_retriever(
    search_type="similarity_score_threshold",
    search_kwargs={'score_threshold': 0.8}
)
# Only get the single most similar document from the dataset
docsearch.as_retriever(search_kwargs={'k': 1})
# Use a filter to only retrieve documents from a specific paper
docsearch.as_retriever(
    search_kwargs={'filter': {'paper_title':'GPT-4 Technical Report'}}
)
async asearch(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
async asimilarity_search(query: str, k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to query.


Return docs most similar to query.
async asimilarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to embedding vector.
async asimilarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs most similar to query.
delete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids – List of ids to delete.
**kwargs – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
delete_texts(ids: List[str]) → None[source]¶
Delete a list of docs from the Rockset collection
classmethod from_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
classmethod from_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, client: Any = None, collection_name: str = '', text_key: str = '', embedding_key: str = '', ids: Optional[List[str]] = None, batch_size: int = 32, **kwargs: Any) → Rockset[source]¶
Create Rockset wrapper with existing texts.
This is intended as a quicker way to get started.
max_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.


Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
query – Text to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
max_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
search(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
similarity_search(query: str, k: int = 4, distance_func: DistanceFunction = DistanceFunction.COSINE_SIM, where_str: Optional[str] = None, **kwargs: Any) → List[Document][source]¶


Same as similarity_search_with_relevance_scores but
doesn’t return the scores.
similarity_search_by_vector(embedding: List[float], k: int = 4, distance_func: DistanceFunction = DistanceFunction.COSINE_SIM, where_str: Optional[str] = None, **kwargs: Any) → List[Document][source]¶
Accepts a query_embedding (vector), and returns documents with
similar embeddings.
similarity_search_by_vector_with_relevance_scores(embedding: List[float], k: int = 4, distance_func: DistanceFunction = DistanceFunction.COSINE_SIM, where_str: Optional[str] = None, **kwargs: Any) → List[Tuple[Document, float]][source]¶
Accepts a query_embedding (vector), and returns documents with
similar embeddings along with their relevance scores.
similarity_search_with_relevance_scores(query: str, k: int = 4, distance_func: DistanceFunction = DistanceFunction.COSINE_SIM, where_str: Optional[str] = None, **kwargs: Any) → List[Tuple[Document, float]][source]¶
Perform a similarity search with Rockset
Parameters
query (str) – Text to look up documents similar to.
distance_func (DistanceFunction) – how to compute distance between two
vectors in Rockset.
k (int, optional) – Top K neighbors to retrieve. Defaults to 4.
where_str (Optional[str], optional) – Metadata filters supplied as a
SQL where condition string. Defaults to None.
eg. “price<=70.0 AND brand=’Nintendo’”
NOTE – Please do not let end-user to fill this and always be aware
of SQL injection.
Returns
List of documents with their relevance score
Return type
List[Tuple[Document, float]]


List of documents with their relevance score
Return type
List[Tuple[Document, float]]
similarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance.
Examples using Rockset¶
Rockset


langchain.vectorstores.faiss.FAISS¶
class langchain.vectorstores.faiss.FAISS(embedding_function: Callable, index: Any, docstore: Docstore, index_to_docstore_id: Dict[int, str], relevance_score_fn: Optional[Callable[[float], float]] = None, normalize_L2: bool = False, distance_strategy: DistanceStrategy = DistanceStrategy.EUCLIDEAN_DISTANCE)[source]¶
Meta Faiss vector store.
To use, you must have the faiss python package installed.
Example
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores import FAISS
embeddings = OpenAIEmbeddings()
texts = ["FAISS is an important library", "LangChain supports FAISS"]
faiss = FAISS.from_texts(texts, embeddings)
Initialize with necessary components.
Attributes
embeddings
Access the query embedding object if available.
Methods
__init__(embedding_function, index, ...[, ...])
Initialize with necessary components.
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_embeddings(text_embeddings[, metadatas, ids])
Run more texts through the embeddings and add to the vectorstore.
add_texts(texts[, metadatas, ids])
Run more texts through the embeddings and add to the vectorstore.
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_texts(texts, embedding[, metadatas])
Return VectorStore initialized from texts and embeddings.


Return VectorStore initialized from texts and embeddings.
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs most similar to query.
delete([ids])
Delete by ID.
deserialize_from_bytes(serialized, ...)
Deserialize FAISS index, docstore, and index_to_docstore_id from bytes.
from_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
from_embeddings(text_embeddings, embedding)
Construct FAISS wrapper from raw documents.
from_texts(texts, embedding[, metadatas, ids])
Construct FAISS wrapper from raw documents.
load_local(folder_path, embeddings[, index_name])
Load FAISS index, docstore, and index_to_docstore_id from disk.
max_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
max_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
max_marginal_relevance_search_with_score_by_vector(...)
Return docs and their similarity scores selected using the maximal marginal
merge_from(target)
Merge another FAISS object with the current one.
save_local(folder_path[, index_name])


save_local(folder_path[, index_name])
Save FAISS index, docstore, and index_to_docstore_id to disk.
search(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
serialize_to_bytes()
Serialize FAISS index, docstore, and index_to_docstore_id to bytes.
similarity_search(query[, k, filter, fetch_k])
Return docs most similar to query.
similarity_search_by_vector(embedding[, k, ...])
Return docs most similar to embedding vector.
similarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
similarity_search_with_score(query[, k, ...])
Return docs most similar to query.
similarity_search_with_score_by_vector(embedding)
Return docs most similar to query.
__init__(embedding_function: Callable, index: Any, docstore: Docstore, index_to_docstore_id: Dict[int, str], relevance_score_fn: Optional[Callable[[float], float]] = None, normalize_L2: bool = False, distance_strategy: DistanceStrategy = DistanceStrategy.EUCLIDEAN_DISTANCE)[source]¶
Initialize with necessary components.
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str]¶
Run more texts through the embeddings and add to the vectorstore.


Run more texts through the embeddings and add to the vectorstore.
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
add_embeddings(text_embeddings: Iterable[Tuple[str, List[float]]], metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, **kwargs: Any) → List[str][source]¶
Run more texts through the embeddings and add to the vectorstore.
Parameters
text_embeddings – Iterable pairs of string and embedding to
add to the vectorstore.
metadatas – Optional list of metadatas associated with the texts.
ids – Optional list of unique IDs.
Returns
List of ids from adding the texts into the vectorstore.
add_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, **kwargs: Any) → List[str][source]¶
Run more texts through the embeddings and add to the vectorstore.
Parameters
texts – Iterable of strings to add to the vectorstore.
metadatas – Optional list of metadatas associated with the texts.
ids – Optional list of unique IDs.
Returns
List of ids from adding the texts into the vectorstore.
async classmethod afrom_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
async classmethod afrom_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, **kwargs: Any) → VST¶


Return VectorStore initialized from texts and embeddings.
async amax_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
async amax_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs: Any) → VectorStoreRetriever¶
Return VectorStoreRetriever initialized from this VectorStore.
Parameters
search_type (Optional[str]) – Defines the type of search that
the Retriever should perform.
Can be “similarity” (default), “mmr”, or
“similarity_score_threshold”.
search_kwargs (Optional[Dict]) – Keyword arguments to pass to the
search function. Can include things like:
k: Amount of documents to return (Default: 4)
score_threshold: Minimum relevance threshold
for similarity_score_threshold
fetch_k: Amount of documents to pass to MMR algorithm (Default: 20)
lambda_mult: Diversity of results returned by MMR;
1 for minimum diversity and 0 for maximum. (Default: 0.5)
filter: Filter by document metadata
Returns
Retriever class for VectorStore.
Return type
VectorStoreRetriever
Examples:
# Retrieve more documents with higher diversity
# Useful if your dataset has many similar documents
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 6, 'lambda_mult': 0.25}
)


)
# Fetch more documents for the MMR algorithm to consider
# But only return the top 5
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 5, 'fetch_k': 50}
)
# Only retrieve documents that have a relevance score
# Above a certain threshold
docsearch.as_retriever(
    search_type="similarity_score_threshold",
    search_kwargs={'score_threshold': 0.8}
)
# Only get the single most similar document from the dataset
docsearch.as_retriever(search_kwargs={'k': 1})
# Use a filter to only retrieve documents from a specific paper
docsearch.as_retriever(
    search_kwargs={'filter': {'paper_title':'GPT-4 Technical Report'}}
)
async asearch(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
async asimilarity_search(query: str, k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to query.
async asimilarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to embedding vector.
async asimilarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs most similar to query.
delete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool][source]¶
Delete by ID. These are the IDs in the vectorstore.
Parameters
ids – List of ids to delete.
Returns
True if deletion is successful,
False otherwise, None if not implemented.


Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
classmethod deserialize_from_bytes(serialized: bytes, embeddings: Embeddings, **kwargs: Any) → FAISS[source]¶
Deserialize FAISS index, docstore, and index_to_docstore_id from bytes.
classmethod from_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
classmethod from_embeddings(text_embeddings: Iterable[Tuple[str, List[float]]], embedding: Embeddings, metadatas: Optional[Iterable[dict]] = None, ids: Optional[List[str]] = None, **kwargs: Any) → FAISS[source]¶
Construct FAISS wrapper from raw documents.
This is a user friendly interface that:
Embeds documents.
Creates an in memory docstore
Initializes the FAISS database
This is intended to be a quick way to get started.
Example
from langchain.vectorstores import FAISS
from langchain.embeddings import OpenAIEmbeddings
embeddings = OpenAIEmbeddings()
text_embeddings = embeddings.embed_documents(texts)
text_embedding_pairs = zip(texts, text_embeddings)
faiss = FAISS.from_embeddings(text_embedding_pairs, embeddings)
classmethod from_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, **kwargs: Any) → FAISS[source]¶
Construct FAISS wrapper from raw documents.
This is a user friendly interface that:
Embeds documents.
Creates an in memory docstore
Initializes the FAISS database
This is intended to be a quick way to get started.
Example
from langchain.vectorstores import FAISS
from langchain.embeddings import OpenAIEmbeddings


from langchain.vectorstores import FAISS
from langchain.embeddings import OpenAIEmbeddings
embeddings = OpenAIEmbeddings()
faiss = FAISS.from_texts(texts, embeddings)
classmethod load_local(folder_path: str, embeddings: Embeddings, index_name: str = 'index', **kwargs: Any) → FAISS[source]¶
Load FAISS index, docstore, and index_to_docstore_id from disk.
Parameters
folder_path – folder path to load index, docstore,
and index_to_docstore_id from.
embeddings – Embeddings to use when generating queries
index_name – for saving with a specific index file name
max_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, filter: Optional[Dict[str, Any]] = None, **kwargs: Any) → List[Document][source]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
query – Text to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch before filtering (if needed) to
pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.


Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
max_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, filter: Optional[Dict[str, Any]] = None, **kwargs: Any) → List[Document][source]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch before filtering to
pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
max_marginal_relevance_search_with_score_by_vector(embedding: List[float], *, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, filter: Optional[Dict[str, Any]] = None) → List[Tuple[Document, float]][source]¶
Return docs and their similarity scores selected using the maximal marginalrelevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch before filtering to
pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding


of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns
List of Documents and similarity scores selected by maximal marginalrelevance and score for each.
merge_from(target: FAISS) → None[source]¶
Merge another FAISS object with the current one.
Add the target FAISS to the current one.
Parameters
target – FAISS object you wish to merge into the current one
Returns
None.
save_local(folder_path: str, index_name: str = 'index') → None[source]¶
Save FAISS index, docstore, and index_to_docstore_id to disk.
Parameters
folder_path – folder path to save index, docstore,
and index_to_docstore_id to.
index_name – for saving with a specific index file name
search(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
serialize_to_bytes() → bytes[source]¶
Serialize FAISS index, docstore, and index_to_docstore_id to bytes.
similarity_search(query: str, k: int = 4, filter: Optional[Dict[str, Any]] = None, fetch_k: int = 20, **kwargs: Any) → List[Document][source]¶
Return docs most similar to query.
Parameters
query – Text to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
filter – (Optional[Dict[str, str]]): Filter by metadata. Defaults to None.
fetch_k – (Optional[int]) Number of Documents to fetch before filtering.
Defaults to 20.
Returns
List of Documents most similar to the query.


Defaults to 20.
Returns
List of Documents most similar to the query.
similarity_search_by_vector(embedding: List[float], k: int = 4, filter: Optional[Dict[str, Any]] = None, fetch_k: int = 20, **kwargs: Any) → List[Document][source]¶
Return docs most similar to embedding vector.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
fetch_k – (Optional[int]) Number of Documents to fetch before filtering.
Defaults to 20.
Returns
List of Documents most similar to the embedding.
similarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query – input text
k – Number of Documents to return. Defaults to 4.
**kwargs – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
similarity_search_with_score(query: str, k: int = 4, filter: Optional[Dict[str, Any]] = None, fetch_k: int = 20, **kwargs: Any) → List[Tuple[Document, float]][source]¶
Return docs most similar to query.
Parameters
query – Text to look up documents similar to.
k – Number of Documents to return. Defaults to 4.


k – Number of Documents to return. Defaults to 4.
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
fetch_k – (Optional[int]) Number of Documents to fetch before filtering.
Defaults to 20.
Returns
List of documents most similar to the query text with
L2 distance in float. Lower score represents more similarity.
similarity_search_with_score_by_vector(embedding: List[float], k: int = 4, filter: Optional[Dict[str, Any]] = None, fetch_k: int = 20, **kwargs: Any) → List[Tuple[Document, float]][source]¶
Return docs most similar to query.
Parameters
embedding – Embedding vector to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
filter (Optional[Dict[str, Any]]) – Filter by metadata. Defaults to None.
fetch_k – (Optional[int]) Number of Documents to fetch before filtering.
Defaults to 20.
**kwargs – kwargs to be passed to similarity search. Can include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of documents most similar to the query text and L2 distance
in float for each. Lower score represents more similarity.
Examples using FAISS¶
Cohere Reranker
Facebook Faiss
Document Comparison
Faiss
Loading documents from a YouTube url
Conversational Retrieval Agent
Agents
AutoGPT
BabyAGI User Guide
BabyAGI with Tools
!pip install bs4
Plug-and-Plai
Custom Agent with PlugIn Retrieval
Generative Agents in LangChain
Set env var OPENAI_API_KEY or load from a .env file
SQL
Caching
Ensemble Retriever
Custom agent with tool retrieval


SQL
Caching
Ensemble Retriever
Custom agent with tool retrieval
Select by maximal marginal relevance (MMR)
First we add a step to load memory


langchain.vectorstores.pgembedding.QueryResult¶
class langchain.vectorstores.pgembedding.QueryResult[source]¶
Result from a query.
Attributes
EmbeddingStore
distance
Methods
__init__()
__init__()¶


langchain.vectorstores.pinecone.Pinecone¶
class langchain.vectorstores.pinecone.Pinecone(index: Any, embedding: Union[Embeddings, Callable], text_key: str, namespace: Optional[str] = None, distance_strategy: Optional[DistanceStrategy] = DistanceStrategy.COSINE)[source]¶
Pinecone vector store.
To use, you should have the pinecone-client python package installed.
Example
from langchain.vectorstores import Pinecone
from langchain.embeddings.openai import OpenAIEmbeddings
import pinecone
# The environment should be the one specified next to the API key
# in your Pinecone console
pinecone.init(api_key="***", environment="...")
index = pinecone.Index("langchain-demo")
embeddings = OpenAIEmbeddings()
vectorstore = Pinecone(index, embeddings.embed_query, "text")
Initialize with Pinecone client.
Attributes
embeddings
Access the query embedding object if available.
Methods
__init__(index, embedding, text_key[, ...])
Initialize with Pinecone client.
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_texts(texts[, metadatas, ids, ...])
Run more texts through the embeddings and add to the vectorstore.
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_texts(texts, embedding[, metadatas])
Return VectorStore initialized from texts and embeddings.


Return VectorStore initialized from texts and embeddings.
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs most similar to query.
delete([ids, delete_all, namespace, filter])
Delete by vector IDs or filter.
from_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
from_existing_index(index_name, embedding[, ...])
Load pinecone vectorstore from index name.
from_texts(texts, embedding[, metadatas, ...])
Construct Pinecone wrapper from raw documents.
get_pinecone_index(index_name[, pool_threads])
Return a Pinecone Index instance.
max_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
max_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
search(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
similarity_search(query[, k, filter, namespace])
Return pinecone documents most similar to query.
similarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.


Return docs most similar to embedding vector.
similarity_search_by_vector_with_score(...)
Return pinecone documents most similar to embedding, along with scores.
similarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
similarity_search_with_score(query[, k, ...])
Return pinecone documents most similar to query, along with scores.
__init__(index: Any, embedding: Union[Embeddings, Callable], text_key: str, namespace: Optional[str] = None, distance_strategy: Optional[DistanceStrategy] = DistanceStrategy.COSINE)[source]¶
Initialize with Pinecone client.
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str]¶
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
add_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, namespace: Optional[str] = None, batch_size: int = 32, embedding_chunk_size: int = 1000, **kwargs: Any) → List[str][source]¶


Run more texts through the embeddings and add to the vectorstore.
Upsert optimization is done by chunking the embeddings and upserting them.
This is done to avoid memory issues and optimize using HTTP based embeddings.
For OpenAI embeddings, use pool_threads>4 when constructing the pinecone.Index,
embedding_chunk_size>1000 and batch_size~64 for best performance.
:param texts: Iterable of strings to add to the vectorstore.
:param metadatas: Optional list of metadatas associated with the texts.
:param ids: Optional list of ids to associate with the texts.
:param namespace: Optional pinecone namespace to add the texts to.
:param batch_size: Batch size to use when adding the texts to the vectorstore.
:param embedding_chunk_size: Chunk size to use when embedding the texts.
Returns
List of ids from adding the texts into the vectorstore.
async classmethod afrom_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
async classmethod afrom_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, **kwargs: Any) → VST¶
Return VectorStore initialized from texts and embeddings.
async amax_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
async amax_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.


Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs: Any) → VectorStoreRetriever¶
Return VectorStoreRetriever initialized from this VectorStore.
Parameters
search_type (Optional[str]) – Defines the type of search that
the Retriever should perform.
Can be “similarity” (default), “mmr”, or
“similarity_score_threshold”.
search_kwargs (Optional[Dict]) – Keyword arguments to pass to the
search function. Can include things like:
k: Amount of documents to return (Default: 4)
score_threshold: Minimum relevance threshold
for similarity_score_threshold
fetch_k: Amount of documents to pass to MMR algorithm (Default: 20)
lambda_mult: Diversity of results returned by MMR;
1 for minimum diversity and 0 for maximum. (Default: 0.5)
filter: Filter by document metadata
Returns
Retriever class for VectorStore.
Return type
VectorStoreRetriever
Examples:
# Retrieve more documents with higher diversity
# Useful if your dataset has many similar documents
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 6, 'lambda_mult': 0.25}
)
# Fetch more documents for the MMR algorithm to consider
# But only return the top 5
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 5, 'fetch_k': 50}
)
# Only retrieve documents that have a relevance score
# Above a certain threshold
docsearch.as_retriever(
    search_type="similarity_score_threshold",
    search_kwargs={'score_threshold': 0.8}
)
# Only get the single most similar document from the dataset


)
# Only get the single most similar document from the dataset
docsearch.as_retriever(search_kwargs={'k': 1})
# Use a filter to only retrieve documents from a specific paper
docsearch.as_retriever(
    search_kwargs={'filter': {'paper_title':'GPT-4 Technical Report'}}
)
async asearch(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
async asimilarity_search(query: str, k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to query.
async asimilarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to embedding vector.
async asimilarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs most similar to query.
delete(ids: Optional[List[str]] = None, delete_all: Optional[bool] = None, namespace: Optional[str] = None, filter: Optional[dict] = None, **kwargs: Any) → None[source]¶
Delete by vector IDs or filter.
:param ids: List of ids to delete.
:param filter: Dictionary of conditions to filter vectors to delete.
classmethod from_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
classmethod from_existing_index(index_name: str, embedding: Embeddings, text_key: str = 'text', namespace: Optional[str] = None, pool_threads: int = 4) → Pinecone[source]¶
Load pinecone vectorstore from index name.


Load pinecone vectorstore from index name.
classmethod from_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, batch_size: int = 32, text_key: str = 'text', namespace: Optional[str] = None, index_name: Optional[str] = None, upsert_kwargs: Optional[dict] = None, pool_threads: int = 4, embeddings_chunk_size: int = 1000, **kwargs: Any) → Pinecone[source]¶
Construct Pinecone wrapper from raw documents.
This is a user friendly interface that:
Embeds documents.
Adds the documents to a provided Pinecone index
This is intended to be a quick way to get started.
The pool_threads affects the speed of the upsert operations.
.. rubric:: Example
from langchain.vectorstores import Pinecone
from langchain.embeddings import OpenAIEmbeddings
import pinecone
# The environment should be the one specified next to the API key
# in your Pinecone console
pinecone.init(api_key="***", environment="...")
embeddings = OpenAIEmbeddings()
pinecone = Pinecone.from_texts(
    texts,
    embeddings,
    index_name="langchain-demo"
)
classmethod get_pinecone_index(index_name: Optional[str], pool_threads: int = 4) → Index[source]¶
Return a Pinecone Index instance.
Parameters
index_name – Name of the index to use.
pool_threads – Number of threads to use for index upsert.
Returns
Pinecone Index instance.


Returns
Pinecone Index instance.
max_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, filter: Optional[dict] = None, namespace: Optional[str] = None, **kwargs: Any) → List[Document][source]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
query – Text to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
max_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, filter: Optional[dict] = None, namespace: Optional[str] = None, **kwargs: Any) → List[Document][source]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns


Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
search(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
similarity_search(query: str, k: int = 4, filter: Optional[dict] = None, namespace: Optional[str] = None, **kwargs: Any) → List[Document][source]¶
Return pinecone documents most similar to query.
Parameters
query – Text to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
filter – Dictionary of argument(s) to filter on metadata
namespace – Namespace to search in. Default will search in ‘’ namespace.
Returns
List of Documents most similar to the query and score for each
similarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to embedding vector.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
Returns
List of Documents most similar to the query vector.
similarity_search_by_vector_with_score(embedding: List[float], *, k: int = 4, filter: Optional[dict] = None, namespace: Optional[str] = None) → List[Tuple[Document, float]][source]¶
Return pinecone documents most similar to embedding, along with scores.
similarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query – input text


0 is dissimilar, 1 is most similar.
Parameters
query – input text
k – Number of Documents to return. Defaults to 4.
**kwargs – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
similarity_search_with_score(query: str, k: int = 4, filter: Optional[dict] = None, namespace: Optional[str] = None) → List[Tuple[Document, float]][source]¶
Return pinecone documents most similar to query, along with scores.
Parameters
query – Text to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
filter – Dictionary of argument(s) to filter on metadata
namespace – Namespace to search in. Default will search in ‘’ namespace.
Returns
List of Documents most similar to the query and score for each
Examples using Pinecone¶
Pinecone


langchain.vectorstores.lancedb.LanceDB¶
class langchain.vectorstores.lancedb.LanceDB(connection: Any, embedding: Embeddings, vector_key: Optional[str] = 'vector', id_key: Optional[str] = 'id', text_key: Optional[str] = 'text')[source]¶
LanceDB vector store.
To use, you should have lancedb python package installed.
Example
db = lancedb.connect('./lancedb')
table = db.open_table('my_table')
vectorstore = LanceDB(table, embedding_function)
vectorstore.add_texts(['text1', 'text2'])
result = vectorstore.similarity_search('text1')
Initialize with Lance DB connection
Attributes
embeddings
Access the query embedding object if available.
Methods
__init__(connection, embedding[, ...])
Initialize with Lance DB connection
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_texts(texts[, metadatas, ids])
Turn texts into embedding and add it to the database
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_texts(texts, embedding[, metadatas])
Return VectorStore initialized from texts and embeddings.
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)