Instantiate a prompt cache using Momento as a backend.
Note: to instantiate the cache client passed to MomentoCache,
you must have a Momento account. See https://gomomento.com/.
Parameters
cache_client (CacheClient) – The Momento cache client.
cache_name (str) – The name of the cache to use to store the data.
ttl (Optional[timedelta], optional) – The time to live for the cache items.
Defaults to None, ie use the client default TTL.
ensure_cache_exists (bool, optional) – Create the cache if it doesn’t
exist. Defaults to True.
Raises
ImportError – Momento python package is not installed.
TypeError – cache_client is not of type momento.CacheClientObject
ValueError – ttl is non-null and non-negative
clear(**kwargs: Any) → None[source]¶
Clear the cache.
Raises
SdkException – Momento service or network error
classmethod from_client_params(cache_name: str, ttl: timedelta, *, configuration: Optional[momento.config.Configuration] = None, auth_token: Optional[str] = None, **kwargs: Any) → MomentoCache[source]¶
Construct cache from CacheClient parameters.
lookup(prompt: str, llm_string: str) → Optional[Sequence[Generation]][source]¶
Lookup llm generations in cache by prompt and associated model and settings.
Parameters
prompt (str) – The prompt run through the language model.
llm_string (str) – The language model version and settings.
Raises
SdkException – Momento service or network error
Returns
A list of language model generations.
Return type
Optional[RETURN_VAL_TYPE]
update(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None[source]¶
Store llm generations in cache.
Parameters