Initialize the cache with all relevant parameters.
:param session: an open Cassandra session
:type session: cassandra.cluster.Session
:param keyspace: the keyspace to use for storing the cache
:type keyspace: str
:param embedding: Embedding provider for semantic
encoding and search.
Parameters
table_name (str) – name of the Cassandra (vector) table
to use as cache
distance_metric (str, 'dot') – which measure to adopt for
similarity searches
score_threshold (optional float) – numeric value to use as
cutoff for the similarity searches
ttl_seconds (optional int) – time-to-live for cache entries
(default: None, i.e. forever)
The default score threshold is tuned to the default metric.
Tune it carefully yourself if switching to another distance metric.
clear(**kwargs: Any) → None[source]¶
Clear the whole semantic cache.
delete_by_document_id(document_id: str) → None[source]¶
Given this is a “similarity search” cache, an invalidation pattern
that makes sense is first a lookup to get an ID, and then deleting
with that ID. This is for the second step.
lookup(prompt: str, llm_string: str) → Optional[Sequence[Generation]][source]¶
Look up based on prompt and llm_string.
lookup_with_id(prompt: str, llm_string: str) → Optional[Tuple[str, Sequence[Generation]]][source]¶
Look up based on prompt and llm_string.
If there are hits, return (document_id, cached_entry)
lookup_with_id_through_llm(prompt: str, llm: LLM, stop: Optional[List[str]] = None) → Optional[Tuple[str, Sequence[Generation]]][source]¶
update(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None[source]¶