<commit_before>---
layout: post
title:  LRU-Cache Policy
categories: XAP97
parent: memory-management-facilities.html
weight: 200
---

{% summary %}The LRU Cache policy{% endsummary %}

# Overview

When running in LRU cache policy mode, the space evicts the "oldest" objects from its memory. "Oldest" objects are determined by the time they were written, updated or read in the space. In a persistent space mode, evicting a space object means that a space object is  simply be removed from the space memory, but is still be available through the underlying RDBMS. The space reloads this object back into the space memory only if it was requested by a specific read operation.

The space memory manager uses a dedicated thread called **Evictor** - this thread handles the eviction of objects and identifying memory shortage events. In general, eviction can be done using:

- **Max amount of space objects** - evicts objects one by one. Does not use batches. Very moderate mechanism. Turned on by default when running in LRU mode.
- **Available memory** - eviction is done in batches.

Evicting an object from the space requires the space engine to lock the LRU chain during the object removal, and to update the relevant indexes. This means that the eviction based on **available memory** that is done in batches, might impact the space responsiveness to client requests. Still, you might need to use this in case you can't estimate the amount of objects within the space.

# When LRU Cache Policy Should be Used

For applications that use a very large backend database, leveraging the space as a frontend distributed cache using the `LRU` cache policy can speed up read activity while keeping some of the data in memory. In this case - the motivation is to **access the database in the most optimal manner** when performing data access operations against the space to reduce the load on the database as much as you can.

When using `read`, `readById` or `readByIds` operations performing a lookup for a single specific object(s), that cannot be found within the space (this is often refered to as a "cache miss"), the database access by the space is very optimal. Only one row is retrieved from the database per object lookup via the Space Data Source implementation. This will be conducted from the relevant partition only.

But when performing queries using `readMultiple` or `GSIterator` with a template or a `SQLQuery`, the returned result set that may involve relatively large amount of objects. In such case, a space running with the `LRU` cache policy is very likely to retrieve large amount of data from the database:

- When using `readMultiple` having `Integer.MAX_VALUE` as the `max_objects` parameter, every partition will access the database (parallel database access). This may overload the database.
- When using `readMultiple` having `max_objects` < `Integer.MAX_VALUE` the database might be accessed even if there are enough objects matching the query criteria across all the space partitions.
- When loading data from the database, a data eviction process may be triggered. This may impact the performance.
- Database access may involve reading objects that will not be loaded into the space (result set with non-matching routing value that are filtred out by the partition).

In order to minimize the potential database overhead above, the `LRU` cache policy is recommended mostly for applications that use the `read`, `readById` or `readByIds` operations.

Constructing read operations using `SQLQuery` with `Order by`, `Group by` is not recommended either, as it reqires traversing the entire data set in many cases. In such cases, the preffered option is to perform the query directly against the database and select only the IDs, and then retreive the actual data from Space using the `readById` or `readByIds` operations. Then working with a partitioned space, it is recommended to store the routing value in a separate database column, and select it as so that the `readById` or `readByIds` operations will be more efficient.

# 'MEMORY_ONLY_SEARCH' Modifier

The `MEMORY_ONLY_SEARCH` modifier may be used to instruct the space to limit the query to the memory only and not access the database. This will be give flexibility to minimize the database load in cases where this modifier can be applied. 

This is very useful When running with the `LRU` cache policy and deploying a [Polling Container](polling-container.html). By default in such case, all `read`, `take` and `clear` operations will be conducted **both** against the space and the underlying persistent store. By using the `MEMORY_ONLY_SEARCH` modifier, the operations invokved by the polling container will **only access the space** for processing the search, without interacting with the persistent store. 

Here's a polling container example using the `MEMORY_ONLY_SEARCH` modifier:

{% highlight java %}
SingleReadReceiveOperationHandler receiveOperationHandler = new SingleReadReceiveOperationHandler();
receiveOperationHandler.setUseMemoryOnlySearch(true);
SimplePollingEventListenerContainer pollingEventListenerContainer = 
		new SimplePollingContainerConfigurer(space)
        .template(new Data())
        .receiveOperationHandler(receiveOperationHandler)
        .eventListenerAnnotation(new Object() {
            @SpaceDataEvent
        	public void eventHappened() {
        	}
}).pollingContainer();
{% endhighlight %}
<commit_msg>Some more fixes to LRU page<commit_after>---
layout: post
title:  LRU-Cache Policy
categories: XAP97
parent: memory-management-facilities.html
weight: 200
---

{% summary %}The LRU Cache policy{% endsummary %}

# Overview

When running in LRU cache policy mode, the space evicts the "oldest" objects from its memory. "Oldest" objects are determined by the time they were written, updated or read in the space. In a persistent space mode, evicting a space object means that a space object is  simply be removed from the space memory, but is still be available through the underlying RDBMS. The space reloads this object back into the space memory only if it was requested by a specific read operation.

The space memory manager uses a dedicated thread called **Evictor** - this thread handles the eviction of objects and identifying memory shortage events. In general, eviction can be done using:

- **Max amount of space objects** - evicts objects one by one. Does not use batches. Very moderate mechanism. Turned on by default when running in LRU mode.
- **Available memory** - eviction is done in batches.

Evicting an object from the space requires the space engine to lock the LRU chain during the object removal, and to update the relevant indexes. This means that the eviction based on **available memory** that is done in batches, might impact the space responsiveness to client requests. Still, you might need to use this in case you can't estimate the amount of objects within the space.

# When LRU Cache Policy Should be Used

For applications that use a very large backend database, leveraging the space as a frontend distributed cache using the `LRU` cache policy can speed up read activity while keeping some of the data in memory. In this case - the motivation is to **access the database in the most optimal manner** when performing data access operations against the space to reduce the load on the database as much as you can.

When using `read`, `readById` or `readByIds` operations performing a lookup for a single specific object(s), that cannot be found within the space (this is often refered to as a "cache miss"), the database access by the space is very optimal. Only one row is retrieved from the database per object lookup via the Space Data Source implementation. This will be conducted from the relevant partition only.

But when performing queries using `readMultiple` or `GSIterator` with a template or a `SQLQuery`, the returned result set that may involve relatively large amount of objects. In such case, a space running with the `LRU` cache policy is very likely to retrieve large amount of data from the database:

- When using `readMultiple` having `Integer.MAX_VALUE` as the `max_objects` parameter, every partition will access the database (parallel database access). This may overload the database.
- When using `readMultiple` having `max_objects` < `Integer.MAX_VALUE` the database might be accessed even if there are enough objects matching the query criteria across all the space partitions.
- When loading data from the database, a data eviction process may be triggered. This may impact the performance.
- Database access may involve reading objects that will not be loaded into the space (result set with non-matching routing value that are filtred out by the partition).

In order to minimize the potential database overhead above, the `LRU` cache policy is recommended mostly for applications that use the `read`, `readById` or `readByIds` operations.

Constructing read operations using `SQLQuery` with `Order by`, `Group by` is not recommended either, as it reqires traversing the entire data set in many cases. In such cases, the preffered option is to perform the query directly against the database and select only the IDs, and then retreive the actual data from Space using the `readById` or `readByIds` operations. Then working with a partitioned space, it is recommended to store the routing value in a separate database column, and select it as so that the `readById` or `readByIds` operations will be more efficient.

# The `MEMORY_ONLY_SEARCH` Modifier

The `MEMORY_ONLY_SEARCH` modifier may be used to instruct the space to limit the query to the memory only and not access the database. This will be give flexibility to minimize the database load in cases where this modifier can be applied. 

This is very useful When running with the `LRU` cache policy and deploying a [Polling Container](polling-container.html). By default in such case, all `read`, `take` and `clear` operations will be conducted **both** against the space and the underlying persistent store. By using the `MEMORY_ONLY_SEARCH` modifier, the operations invokved by the polling container will **only access the space** for processing the search, without interacting with the persistent store. 

Here's a polling container example using the `MEMORY_ONLY_SEARCH` modifier:

{% highlight java %}
SingleReadReceiveOperationHandler receiveOperationHandler = new SingleReadReceiveOperationHandler();
receiveOperationHandler.setUseMemoryOnlySearch(true);
SimplePollingEventListenerContainer pollingEventListenerContainer = 
		new SimplePollingContainerConfigurer(space)
        .template(new Data())
        .receiveOperationHandler(receiveOperationHandler)
        .eventListenerAnnotation(new Object() {
            @SpaceDataEvent
        	public void eventHappened() {
                //do something meaningful 
        	}
}).pollingContainer();
{% endhighlight %}

{'fraction_non_alphanumeric': 0.04905280497849364, 'fraction_numerical': 0.0009151642719868216, 'mean_word_length': 4.70057381324987, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3080804', 'n_tokens_mistral': 2594, 'n_tokens_neox': 2473, 'n_words': 1561}