Ehcache: Cache Replication in Clustered Environment using JGroups

If you are using Ehcache and you want to replicate your cache across all the nodes in a clustered environment, you may find some fruitful information in this post.There are three different ways to replicate your cache across all the nodes in a cluster:

• JGroups Replicated Caching
• RMI Replicated Caching
• JMS Replicated Caching

This post tells about ‘JGroups Replicated Caching’. JGroups is a simple clustered task distribution system. JGroups integration with Ehcache facilitates replicating the cache across the nodes in a cluster.

How to configure?

Cache replication configuration with JGroups is not much complicated .With very simple configuration you can achieve cache replication in your clustered environment. 

You need to configure below files for cache replication:

• ApplicationContext.xml (Spring's application context file)
• Ehcache.xml: (Ehcache configuration file)
• JgroupCache.xml (JGroups configuration file for nodes communication)

ApplicationContext.xml: Configure 'EhCacheManagerFactoryBean' in application context file to initialize cache manager.

<bean id='ehCacheManager' class="org.springframework.cache.ehcache.EhCacheManagerFactoryBean''> <property name="configLocation" value="classpath:Ehcache.xml"/>  <property name="shared" value="true" /> </bean>

Ehcache.xml: To replicate cache in a cluster you need to configure below tags in 'Ehcache.xml' file:

• cacheManagerPeerProviderFactory: This tag is used to create a CacheManagerPeerProvider, which discovers other CacheManagers in the cluster.

• cacheEventListenerFactory: Enables registration of listeners for cache events, such as put, remove, update, and expire.

• bootstrapCacheLoaderFactory: Specifies a BootstrapCacheLoader, which is called by a cache on initialization to prepopulate itself.

Each cache that will be distributed needs to set a cache event listener which replicates messages to the other CacheManager peers. This can be done by adding a 'cacheEventListenerFactory' element of type 'JGroupsCacheReplicatorFactory' to each distributed cache's configuration as per the following example:

<?xml version="1.0" encoding="UTF-8"?> <ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"     xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd"     updateCheck="false">          <cacheManagerPeerProviderFactory         class="net.sf.ehcache.distribution.jgroups.JGroupsCacheManagerPeerProviderFactory"         properties="file=JGroupsCache.xml" />           //This cache is configured to be replicated     <cache name=”mycache" eternal="true" maxElementsInMemory="100"         overflowToDisk="false" diskPersistent="false" timeToIdleSeconds="0"         timeToLiveSeconds="60" memoryStoreEvictionPolicy="LRU">              <cacheEventListenerFactory             class="net.sf.ehcache.distribution.jgroups.JGroupsCacheReplicatorFactory"             properties="replicateAsynchronously=true, replicatePuts=true,             replicateUpdates=true, replicateUpdatesViaCopy=false,             replicateRemovals=true" />                  <bootstrapCacheLoaderFactory             class="net.sf.ehcache.distribution.jgroups.JGroupsBootstrapCacheLoaderFactory"             properties="bootstrapAsynchronously=false" />       </cache> </ehcache>

JgroupCache.xml: In this file you generally need to configure your nodes and there listening ports for cache replication.

<?xml version="1.0" encoding="UTF-8"?> <config>    <TCP bind_addr="host1" bind_port="7831" />    <TCPPING timeout="3000"        initial_hosts=" host1[7831], host2[7832]"  //Two nodes are in the cluster        port_range="1"       num_initial_members="2"/>    <VERIFY_SUSPECT timeout="1500"  />    <pbcast.NAKACK use_mcast_xmit="false" gc_lag="100"       retransmit_timeout="300,600,1200,2400,4800"       discard_delivered_msgs="false"/>    <pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000" max_bytes="400000"/>    <pbcast.GMS print_local_addr="true" join_timeout="5000" shun="false" view_bundling="true"/> </config>

Frequently Asked Questions

1: What if I get below log message If one node tries to send cache notification to others?

'Dropped message from host1-64423 (not in xmit_table)'

Solution: There is a property named ‘discard_delivered_msgs’ should be false in JGroups configuration file.

2: How to keep JGroups configuration file out of web application war file?

Solution: In 'Ehcache.xml', you need not to hard code your 'JGroupsCache.xml'. You can specify this with the help of system property.

Define the JVM argument like -Djsgroup-config-location = C:\jgroups-configuration\JGroupsCache.xml

and then specify this property in 'Ehcache.xml' file.

<cacheManagerPeerProviderFactory

        class="net.sf.ehcache.distribution.jgroups.JGroupsCacheManagerPeerProviderFactory"

        properties="file=${jsgroup-config-location} " />  

3: How to analyze whether nodes are communicating each other?

Solution:  You must see below logs in your server to ensure whether your nodes are registered as per JGroups configuration or not.

Logs:

-------------------------------------------------------------------

GMS: address=IP-ADDRESS-41447, cluster=EH_CACHE, physical address= 2002:19a1:70a:0:0:0:19a1 :70a:58603

-------------------------------------------------------------------

[10/25/13 17:57:38:451 IST] 0000001e JGroupsCacheM I net.sf.ehcache.distribution.jgroups.JGroupsCacheManagerPeerProvider init JGroups Replication started for 'EH_CACHE'. JChannel: local_addr=IP-ADDRESS-41447

cluster_name=EH_CACHE

my_view=[IP-ADDRESS-41447|0] [IP-ADDRESS-41447][IP-ADDRESS-41448]

connected=true

closed=false

incoming queue size=0

receive_blocks=false

receive_local_msgs=false

state_transfer_supported=true

4: How to enable 'Ehcache' logs?

Solution: In your log4j configuration file, add below entries to view the Ehcache related logs.

<category name="net.sf.ehcache"  additivity="false">      <priority value="debug" />      <appender-ref ref="console" />   </category>   <category name="net.sf.ehcache.config"  additivity="false">      <priority value="debug" />      <appender-ref ref="console" />   </category>   <category name="net.sf.ehcache.distribution"  additivity="false">      <priority value="debug" />      <appender-ref ref="console" />   </category>

 Others                                                                                                        

• To get more detail about cache replication methods in Ehcache, you can refer this link.
• To get detailed information about above configuration you can refer this link.
• You can also refer very nicely written post from here. 

Automatic Sequence Diagram Generation

As a developer or architect, you always need to draw some sequence diagrams to demonstrate or document your functionality. And of course, if you do this manually you have to spare much time for this activity. Just think about, what If sequence diagram is generated automatically and you get it free of cost. Your reaction would be ‘wow’ this is great. But the next question will be ‘how’. So, I am publishing this post to describe ‘how’.There are different open source tools which help to generate UML sequence diagrams. ‘Java Call Trace’ is such a tool that can generate UML sequence diagram based on reverse engineering.

Install ‘Java Call Trace’

• Download the java call trace from here.
• Extract its contents at your desired location.

How to Use ‘Java Call Trace’ to Generate Sequence Diagram

     To generate sequence diagram using Java call trace you have to follow below steps:

• Define your filter in <javacalltracer>\Calltracer\filters.txt

This will generate sequence diagram only for the classes which are defined under given package. For example: com.mycom.example 

• Capture call trace (as xml) of your application/program 

For JDK 1.5 and above, add the following JVM options and  execute your application/program.

Syntax:

-agentpath:[full-path-to-folder-of-dll]\calltracer5.dll=traceFile-[trace-file],filterFile-[filter-file],filterList-[filter|filter|...],outputType-[xml or text],usage-[controlled, uncontrolled] -Djava.library.path=[full-path-to-folder-of-dll] -Dcalltracerlib=calltracer5 

Example

-agentpath:C:\calltracer\jvmti\calltracer5.dll=traceFile-C:\calltracer\call.trace,filterFile-C:\calltracer\filters.txt,outputType-xml,usage-uncontrolled -Djava.library.path=C:\calltracer\jvmti -Dcalltracerlib=calltracer5

• Generate sequence diagram using call trace captured in first step

Once you have generated a XML output using the ‘calltracer’ tool you can use the ‘Calltrace2Seq tool’ to generate UML sequence diagram.

<javacalltracer>\Calltrace2Seq\src\Calltrace2Seq.java INPUTXMLFILE-C:\\test.xml OUTPUTFOLDER-C:\\ OUTPUTFILENAME-mysequencediagram

INPUTXMLFILE: This is the generated xml location.

OUTPUTFILENAME: This is the location where generated sequence diagram image is copied.

Note: The output generated by the ‘calltracer’ tool has some header information that will not allow it (the output) to be used directly as an input to the ‘Calltrace2Seq’ tool. You can remove the header from the output xml file and then use this file as input argument for ‘Calltrace2Seq’.

Generated Sample Sequence Diagram

Below diagram is generated using Java Call Trace. There are three classes ManagedBean, Service and DataAccess. Where ManagedBean is having main method which is calling Service's method and Service is calling DataAccess's method.

Java Call Trace Details

If you are interested to get more details about Java Call Trace you can refer below:

<javacalltracer>\readme.txt

<javacalltracer>\calltracer\readme.txt

<javacalltracer>\Calltrace2Seq\readme.txt

Audit Logging using Hibernate Interceptor in JPA 2.0

If you are using JPA 2.0 with Hibernate and you want to do audit logging from middle-ware itself, I believe you landed up on the exact place where you should be. You can try audit logging in your local environment by following this post.

Required JPA/Hibernate Maven Dependencies

<dependency>          <groupId>org.hibernate.java-persistence</groupId>          <artifactId>jpa-api</artifactId>          <version>2.0-cr-1</version> </dependency> <dependency>          <groupId>org.hibernate</groupId>          <artifactId>hibernate-entitymanager</artifactId>          <version>3.6.4.Final</version>    </dependency> <dependency>          <groupId>org.springframework</groupId>          <artifactId>spring-orm</artifactId>          <version>3.0.5.RELEASE</version> </dependency>

JPA Configuration in Spring Application Context File

For JPA/Hibernate, you need to configure entity manager, transaction, data source and JPA vendor in your ‘applicationContext.xml’ file.

<bean id="dataSource"          class="org.springframework.jdbc.datasource.DriverManagerDataSource"          p:driverClassName="oracle.jdbc.driver.OracleDriver"           p:url="jdbc:oracle:thin:@127.0.0.1:1521/mydbservice "          p:username="nvuser" p:password="nvpass" /> <bean id="entityManagerFactory"          class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean"          p:persistenceUnitName="MyPersistentUnit"          p:persistenceXmlLocation="classpath*:persistence.xml"          p:dataSource-ref="dataSource" p:jpaVendorAdapter-ref="jpaAdapter">          <property name="loadTimeWeaver">             <bean class="org.springframework.instrument.classloading.InstrumentationLoadTimeWeaver"/>          </property>               </bean> <bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager"                  p:entityManagerFactory-ref="entityManagerFactory" /> <bean id="jpaAdapter"                  class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter"                  p:database="ORACLE" p:showSql="true" />

Persistence.xml Configuration

To enable the hibernate audit logging you need to configure a property called ‘hibernate.ejb.interceptor’ in persistence.xml file and the value of this property should be the class that extends the Hibernate ‘EmptyInterceptor’.

<?xml version=”1.0” encoding=”UTF-8”?> <persistence version="2.0"          xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"          xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">          <persistence-unit name=" MyPersistentUnit ">                                  //JPA Entity classes are configured here                  <class>com.mycom.entities.User </class>                             <shared-cache-mode>ENABLE_SELECTIVE</shared-cache-mode>                  <properties>                  <property name="hibernate.ejb.interceptor"                           value=" com.xxx.entities.baseentity.AuditLogInterceptor" />                  </properties>          </persistence-unit> </persistence>

Audit Logging Interceptor Implementation

Hibernate provides an interceptor called ‘EmptyInterceptor’ with following methods.

• onSave() – This method is called when an entity is saved, but the entity is not saved into database yet.
• onFlushDirty() – This method is called when an entity is updated, the entity is not update into database yet.
• onDelete () – This method is called when an entity is deleted , the entity is not deleted into database yet.
• preFlush() – This method is called before the saved, updated or deleted entities are not committed to the database.
• postFlush() – This method is called after the saved, updated or deleted entities are committed to database. This is called after preFlush() method.

To perform audit logging follow below steps:

• Create data base table to capture audit data:

          Audit [ Row_ID, Old_Value, New_Value, Column_Changed, Table_Changed,

                    Transaction_Id, User_ID]

• Create JPA entity say ‘AuditTrail’ for above table
• Create class ‘AuditLogInterceptor’ by extending Hibernate ‘EmptyInterceptor’.
• Override interceptor methods in ‘AuditLogInterceptor’
• Capture required information into AuditTrail entity and save entity

public class AuditLogInterceptor extends EmptyInterceptor {         private List<AuditTrail> auditTrailList = new ArrayList<AuditTrail>();       . . . . . .      // First Parameter: The entity which is being updated        // Second Parameter: The primary key of updated row    // Third Parameter:  Current states of updated entity        // Fourth Parameter: Previous states of updated entity     // Fifth Parameter: The variable names of updated entity    // Sixth Parameter: The type of variables of updated entity    @Override    public boolean onFlushDirty(Object entity, Serializable id,             Object[] currentState, Object[] previousState,             String[] propertyNames, Type[] types) {                                 // Get the table name from entity            Class<?> entityClass = entity.getClass();             Table tableAnnotation = entityClass.getAnnotation(Table.class);                                   for (int i = 0; i < currentState.length; i++) {                               // Track changes only for the states which are String or Number types                             if(!(previousState[i] instanceof String                      || previousState[i] instanceof Long                      || previousState[i] instanceof BigDecimal                       || previousState[i] instanceof Integer                      || previousState[i] instanceof Timestamp))                  {                           continue;                  }                                 // Check whether column value is updated                  if(previousState[i] != null && currentState[i] != null                    &&  !(previousState[i].equals(currentState[i]))){                                             AuditTrail auditTrail = new AuditTrail();                                 // Set updated table name                  auditTrail.setTableName(tableAnnotation.name());                  // Set updated column name                  Column col = null;                  try {                                                                           String filedName = propertyNames[i];                         if(filedName.contains("serialVersionUID")){                                   continue;                         }                         Character firstChar =  filedName.charAt(0);                         firstChar = Character.toUpperCase(firstChar);                         String filedNameSubStr = filedName.substring(1);                         String methodName = "get"+ firstChar+filedNameSubStr;                                                Class[] parameterTypes = new Class[]{};                                                    Method ueMethod = entityClass.getDeclaredMethod(methodName, parameterTypes);                         col = ueMethod.getAnnotation(Column.class);                                              } catch (Exception e) {                                 e.printStackTrace();                     }                    if(col != null){                           auditTrail.setColumnIdentifier(col.name());                           }                                // Set old value                 if(previousState[i] != null){                           auditTrail.setOldValue(previousState[i] == null  ? null :                          previousState[i].toString());                     }                                // Set new value                 if(currentState[i] != null){                           auditTrail.setNewValue(currentState[i] == null ? null:                          currentState[i].toString());                 }                                // Set database operation                 auditTrail.setOperation("Update");                                // Set row primary key value                 auditTrail.setRowIdentifier(id.toString());                   auditTrailList.add(auditTrail);                                        }                        }                                 return false;     }             @Override      public void postFlush(Iterator iterator) throws CallbackException {             // Set unique transaction id in all AuditTrail Entities  String transId = UUID.randomUUID().toString();                 for (AuditTrail auditTrail: auditTrailList) {             auditTrail.setTransactionId(transId); }    // Save ‘AuditTrail’ entity list into database using AuditDAO      }   }

Audit logging Sequence Flow

Issue Faced during Audit Logging Implementation

I faced an interesting issue during audit logging implementation in my application. Let me share that issue detail.

I had ‘AbstractDAO’ and 'AuditDAO' class extends this ‘AbstractDAO’. 'AuditDAO' class is configured as ‘@Repository’. ‘AbstractDAO’ has entity manager property which is initialized by Spring Container. I wanted to auto wire ‘AuditDAO’ in ‘AuditLogInterceptor’ to save ‘AuditTrail’ entity. But, this couldn’t happen. The reason was ‘AuditLogInterceptor’ was not defined as spring resource (like @Component or @Resources or @Repository). Hence's its property can’t be auto wired. Since, this interceptor is initialized by hibernate, I couldn't configure this as spring resource. Finally, I had to take the help of StackOverflow and got the idea to solve this problem. You can refer this issue and solution from here.