<commit_before># MemcacheObject

MemcacheObject is a Ruby on Rails plugin for easy caching of objects for use
as constants. The plugin grew out of a need at TouchLocal to have ready access
to database table contents that was of reasonable size but changed rarely. The
initial caching method for this was to instantiate a constant that contained
the result of a database query, for example:

    PORTALS = Portal.find_all_portals # Returns a hash keyed on the Portal ID

A portal in the context of TouchLocal is an individual location site, for
example touchlondon.co.uk or touchaberdeen.com. There are >100 of these sites,
and a large proportion of the broad configuration of these Portals is stored
in the Database.

While this was a quick and simple solution, it was also a dirty one, as it
caused each app server instance (each Mongrel or Phusion Passenger runner) to
load its own version of this data. As we added more and more constants into
this simple memory cache, not only did it take longer and longer to spawn a
new server, but the memory footprint of each one grew quite large, limiting
the number that could be deployed per host (essentially, costing more money to
host the site as a result.)

After a time, the logical step of changing this process to use Memcache was
started, but the constraint was that it needed to be a drop-in replacement -
don't break the old code! Thus this suite of tools in this Plugin was born.

## MemcacheObject::Proxy

Basic Usage:

    # First, configure your Memcache server.
    # It should be local to the server, if not localhost.
    # This is best done in a config/initializers file
    LOCAL_CACHE = ActiveSupport::Cache::MemCacheStore.new(
      '127.0.0.1:11211', {:namespace => "local_#{RAILS_ENV}"})
    
    # It's useful to include the RAILS_ENV in the namespace, so that you can 
    # use the same Memcache server in development and test and not have them 
    # conflict.
    
    # Now you can start attaching MemcacheObject::Proxy instances to this 
    # cache instance, as such:
    AUTHORS = MemcacheObject::Proxy.new(LOCAL_CACHE, 'AUTHORS', 1.day) do
      hash = {}
      Author.find(:all).each do |author|
        hash[author.id] = author
      end
      hash
    end

Note that the `MemcacheObject::Proxy.new` call takes a block as a parameter -
this is the block that is called to populate the cache. Standard block syntax
(as in, both {} and do-end) are useful here - the do-end form is used above
simply for clarity. The example here is a bit more complicated to show that
you can do pretty much whatever you like in the block. Plenty of simple cases
are useful too!

`AUTHORS` is now a fully-fledged proxy object. On first run, all that has
happened is that the Proxy is initialised, but no data has been loaded. If you
then access the `AUTHORS` constant, the data is initialized and returned

    >> AUTHORS.size
    [DEBUG] 091201 17:38:03 :: Cache miss: MemcacheObject_Proxy_AUTHORS ({:expires_in=>1 day})
    [DEBUG] 091201 17:38:05 :: SQL (1.0ms)  SET NAMES 'utf8'
    [DEBUG] 091201 17:38:05 :: SQL (1.8ms)  SET SQL_AUTO_IS_NULL=0
    [DEBUG] 091201 17:38:07 :: Portal Load (60.0ms)  SELECT * FROM `authors` 
    [DEBUG] 091201 17:38:07 :: Cache write (will save 120.36ms): MemcacheObject_Proxy_AUTHORS
    => 116

Subsequent calls within the expiration period (in this case, 1 day) skip the
database loading step and just retrieve the data directly from Memcache:

    >> AUTHORS.size
    [DEBUG] 091201 17:41:14 :: Cache hit: MemcacheObject_Proxy_AUTHORS ({:expires_in=>1 day})
    => 116

The Proxy class also implements `method_missing` and `inspect` so that any
calls made on the Proxy are passed through to the stored object in a way that
makes sense. `size`, in the above example, utilized this `method_missing`
implementation as the Proxy itself has no concept of size, but the data does.
In this same way, calls to `inspect` are passed to the target data. This all
goes towards making the Proxy a drop-in replacement for the direct cache
mentioned in the opening section. Additionally, as ActiveRecord objects are
passed through the `MemcacheObject::Mash` object before storage (see below),
there is additional safety in the caching while maintaining application
compatibility.

### Configuration Paramters

The example above defined a Proxy instance as:

    AUTHORS = MemcacheObject::Proxy.new(LOCAL_CACHE, 'AUTHORS', 1.day)

while upon execution the log showed:

    [DEBUG] 091201 17:38:03 :: Cache miss: MemcacheObject_Proxy_AUTHORS ({:expires_in=>1 day})

While the first parameter to the Proxy is the Memcache instance to use, the<commit_msg>Supporting documentation<commit_after># MemcacheObject

MemcacheObject is a Ruby on Rails plugin for easy caching of objects for use
as constants. The plugin grew out of a need at TouchLocal to have ready access
to database table contents that was of reasonable size but changed rarely. The
initial caching method for this was to instantiate a constant that contained
the result of a database query, for example:

    PORTALS = Portal.find_all_portals # Returns a hash keyed on the Portal ID

A portal in the context of TouchLocal is an individual location site, for
example touchlondon.co.uk or touchaberdeen.com. There are >100 of these sites,
and a large proportion of the broad configuration of these Portals is stored
in the Database.

While this was a quick and simple solution, it was also a dirty one, as it
caused each app server instance (each Mongrel or Phusion Passenger runner) to
load its own version of this data. As we added more and more constants into
this simple memory cache, not only did it take longer and longer to spawn a
new server, but the memory footprint of each one grew quite large, limiting
the number that could be deployed per host (essentially, costing more money to
host the site as a result.)

After a time, the logical step of changing this process to use Memcache was
started, but the constraint was that it needed to be a drop-in replacement -
don't break the old code! Thus this suite of tools in this Plugin was born.

## MemcacheObject::Proxy

Basic Usage:

    # First, configure your Memcache server.
    # It should be local to the server, if not localhost.
    # This is best done in a config/initializers file
    LOCAL_CACHE = ActiveSupport::Cache::MemCacheStore.new(
      '127.0.0.1:11211', {:namespace => "local_#{RAILS_ENV}"})
    
    # It's useful to include the RAILS_ENV in the namespace, so that you can 
    # use the same Memcache server in development and test and not have them 
    # conflict.
    
    # Now you can start attaching MemcacheObject::Proxy instances to this 
    # cache instance, as such:
    AUTHORS = MemcacheObject::Proxy.new(LOCAL_CACHE, 'AUTHORS', 1.day) do
      hash = {}
      Author.find(:all).each do |author|
        hash[author.id] = author
      end
      hash
    end

Note that the `MemcacheObject::Proxy.new` call takes a block as a parameter -
this is the block that is called to populate the cache. Standard block syntax
(as in, both {} and do-end) are useful here - the do-end form is used above
simply for clarity. The example here is a bit more complicated to show that
you can do pretty much whatever you like in the block. Plenty of simple cases
are useful too!

`AUTHORS` is now a fully-fledged proxy object. On first run, all that has
happened is that the Proxy is initialised, but no data has been loaded. If you
then access the `AUTHORS` constant, the data is initialized and returned

    >> AUTHORS.size
    [DEBUG] 091201 17:38:03 :: Cache miss: MemcacheObject_Proxy_AUTHORS ({:expires_in=>1 day})
    [DEBUG] 091201 17:38:05 :: SQL (1.0ms)  SET NAMES 'utf8'
    [DEBUG] 091201 17:38:05 :: SQL (1.8ms)  SET SQL_AUTO_IS_NULL=0
    [DEBUG] 091201 17:38:07 :: Portal Load (60.0ms)  SELECT * FROM `authors` 
    [DEBUG] 091201 17:38:07 :: Cache write (will save 120.36ms): MemcacheObject_Proxy_AUTHORS
    => 116

Subsequent calls within the expiration period (in this case, 1 day) skip the
database loading step and just retrieve the data directly from Memcache:

    >> AUTHORS.size
    [DEBUG] 091201 17:41:14 :: Cache hit: MemcacheObject_Proxy_AUTHORS ({:expires_in=>1 day})
    => 116

The Proxy class also implements `method_missing` and `inspect` so that any
calls made on the Proxy are passed through to the stored object in a way that
makes sense. `size`, in the above example, utilized this `method_missing`
implementation as the Proxy itself has no concept of size, but the data does.
In this same way, calls to `inspect` are passed to the target data. This all
goes towards making the Proxy a drop-in replacement for the direct cache
mentioned in the opening section. Additionally, as ActiveRecord objects are
passed through the `MemcacheObject::Mash` object before storage (see below),
there is additional safety in the caching while maintaining application
compatibility.

N.B: If you are using `script/console` and you initialize a
`MemcacheObject::Proxy` object, the IRB console automatically calls `.inspect`
on your returned object. To counteract this behaviour and see the true
behaviour in the console add a `; nil` to the end of your statements

    PORTALS = MemcacheObject::Proxy.new(LOCAL_CACHE, 'PORTALS', 1.day) {Portal.find_all_portals} ; nil

`irb` then returns the `nil` but leaves the PORTALS constant properly
initialized ready for use.

### Configuration Paramters

The example above defined a Proxy instance as:

    AUTHORS = MemcacheObject::Proxy.new(LOCAL_CACHE, 'AUTHORS', 1.day)

while upon execution the log showed:

    [DEBUG] 091201 17:38:03 :: Cache miss: MemcacheObject_Proxy_AUTHORS ({:expires_in=>1 day})

While the first parameter to the Proxy is the Memcache instance to use, the
{'fraction_non_alphanumeric': 0.06213532603132357, 'fraction_numerical': 0.025488791073804893, 'mean_word_length': 4.062176165803109, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 25, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6367461', 'n_tokens_mistral': 2885, 'n_tokens_neox': 2652, 'n_words': 1398}