Ehcache: Cache Replication in Clustered Environment using JGroups

If you are using Ehcache and you want to replicate your cache across all the nodes in a clustered environment, you may find some fruitful information in this post.There are three different ways to replicate your cache across all the nodes in a cluster:

• JGroups Replicated Caching
• RMI Replicated Caching
• JMS Replicated Caching

This post tells about ‘JGroups Replicated Caching’. JGroups is a simple clustered task distribution system. JGroups integration with Ehcache facilitates replicating the cache across the nodes in a cluster.

How to configure?

Cache replication configuration with JGroups is not much complicated .With very simple configuration you can achieve cache replication in your clustered environment. 

You need to configure below files for cache replication:

• ApplicationContext.xml (Spring's application context file)
• Ehcache.xml: (Ehcache configuration file)
• JgroupCache.xml (JGroups configuration file for nodes communication)

ApplicationContext.xml: Configure 'EhCacheManagerFactoryBean' in application context file to initialize cache manager.

<bean id='ehCacheManager' class="org.springframework.cache.ehcache.EhCacheManagerFactoryBean''> <property name="configLocation" value="classpath:Ehcache.xml"/>  <property name="shared" value="true" /> </bean>

Ehcache.xml: To replicate cache in a cluster you need to configure below tags in 'Ehcache.xml' file:

• cacheManagerPeerProviderFactory: This tag is used to create a CacheManagerPeerProvider, which discovers other CacheManagers in the cluster.

• cacheEventListenerFactory: Enables registration of listeners for cache events, such as put, remove, update, and expire.

• bootstrapCacheLoaderFactory: Specifies a BootstrapCacheLoader, which is called by a cache on initialization to prepopulate itself.

Each cache that will be distributed needs to set a cache event listener which replicates messages to the other CacheManager peers. This can be done by adding a 'cacheEventListenerFactory' element of type 'JGroupsCacheReplicatorFactory' to each distributed cache's configuration as per the following example:

<?xml version="1.0" encoding="UTF-8"?> <ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"     xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd"     updateCheck="false">          <cacheManagerPeerProviderFactory         class="net.sf.ehcache.distribution.jgroups.JGroupsCacheManagerPeerProviderFactory"         properties="file=JGroupsCache.xml" />           //This cache is configured to be replicated     <cache name=”mycache" eternal="true" maxElementsInMemory="100"         overflowToDisk="false" diskPersistent="false" timeToIdleSeconds="0"         timeToLiveSeconds="60" memoryStoreEvictionPolicy="LRU">              <cacheEventListenerFactory             class="net.sf.ehcache.distribution.jgroups.JGroupsCacheReplicatorFactory"             properties="replicateAsynchronously=true, replicatePuts=true,             replicateUpdates=true, replicateUpdatesViaCopy=false,             replicateRemovals=true" />                  <bootstrapCacheLoaderFactory             class="net.sf.ehcache.distribution.jgroups.JGroupsBootstrapCacheLoaderFactory"             properties="bootstrapAsynchronously=false" />       </cache> </ehcache>

JgroupCache.xml: In this file you generally need to configure your nodes and there listening ports for cache replication.

<?xml version="1.0" encoding="UTF-8"?> <config>    <TCP bind_addr="host1" bind_port="7831" />    <TCPPING timeout="3000"        initial_hosts=" host1[7831], host2[7832]"  //Two nodes are in the cluster        port_range="1"       num_initial_members="2"/>    <VERIFY_SUSPECT timeout="1500"  />    <pbcast.NAKACK use_mcast_xmit="false" gc_lag="100"       retransmit_timeout="300,600,1200,2400,4800"       discard_delivered_msgs="false"/>    <pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000" max_bytes="400000"/>    <pbcast.GMS print_local_addr="true" join_timeout="5000" shun="false" view_bundling="true"/> </config>

Frequently Asked Questions

1: What if I get below log message If one node tries to send cache notification to others?

'Dropped message from host1-64423 (not in xmit_table)'

Solution: There is a property named ‘discard_delivered_msgs’ should be false in JGroups configuration file.

2: How to keep JGroups configuration file out of web application war file?

Solution: In 'Ehcache.xml', you need not to hard code your 'JGroupsCache.xml'. You can specify this with the help of system property.

Define the JVM argument like -Djsgroup-config-location = C:\jgroups-configuration\JGroupsCache.xml

and then specify this property in 'Ehcache.xml' file.

<cacheManagerPeerProviderFactory

        class="net.sf.ehcache.distribution.jgroups.JGroupsCacheManagerPeerProviderFactory"

        properties="file=${jsgroup-config-location} " />  

3: How to analyze whether nodes are communicating each other?

Solution:  You must see below logs in your server to ensure whether your nodes are registered as per JGroups configuration or not.

Logs:

-------------------------------------------------------------------

GMS: address=IP-ADDRESS-41447, cluster=EH_CACHE, physical address= 2002:19a1:70a:0:0:0:19a1 :70a:58603

-------------------------------------------------------------------

[10/25/13 17:57:38:451 IST] 0000001e JGroupsCacheM I net.sf.ehcache.distribution.jgroups.JGroupsCacheManagerPeerProvider init JGroups Replication started for 'EH_CACHE'. JChannel: local_addr=IP-ADDRESS-41447

cluster_name=EH_CACHE

my_view=[IP-ADDRESS-41447|0] [IP-ADDRESS-41447][IP-ADDRESS-41448]

connected=true

closed=false

incoming queue size=0

receive_blocks=false

receive_local_msgs=false

state_transfer_supported=true

4: How to enable 'Ehcache' logs?

Solution: In your log4j configuration file, add below entries to view the Ehcache related logs.

<category name="net.sf.ehcache"  additivity="false">      <priority value="debug" />      <appender-ref ref="console" />   </category>   <category name="net.sf.ehcache.config"  additivity="false">      <priority value="debug" />      <appender-ref ref="console" />   </category>   <category name="net.sf.ehcache.distribution"  additivity="false">      <priority value="debug" />      <appender-ref ref="console" />   </category>

 Others                                                                                                        

• To get more detail about cache replication methods in Ehcache, you can refer this link.
• To get detailed information about above configuration you can refer this link.
• You can also refer very nicely written post from here.