Large Batch Processing using Hibernate

When using hibernate for the retrieval of large amounts of data, one should remember that memory is limited and that incorrect usage of the hibernate session will cause the first and second level cache to explode in size.

There are two ways to prevent this, either (a) evict the objects once you are finished with them, or (b) use a stateless session. Another point to remember is that you cannot use the list method on the Query because this will retrieve the entire result set.




Statefull Session:

Remember to evict the associated objects from the first level cache once you are finished with them:

Query query = session.getNamedQuery("large");
ScrollableResult result = query
                              .setCacheMode(CacheMode.IGNORE)
                              .scroll(ScrollMode.FORWARD_ONLY);

while (result.next()) {
      Object[] v = result.get();
      Object a = v[0];
      Object b = v[1];

      … batch process …
      Session.evict(a);
      Session.evict(b);
}

Stateless Session:

There is no need to evict objects from the first level cache however all lazy relationships will be unavailable.

Query query = session.getSessionFactory()
                  .openStatelessSession()
                  .getNamedQuery("large");

ScrollableResult result = query.scroll(ScrollMode.FORWARD_ONLY);
while (result.next()) {
      Object v = result.get(0);
      … batch process …       
}

Another useful feature is to utilise the setProperties method on the query object set appropriate query parameters instead of setting each parameter individually.

Bind using parameters:

Query query = session.getNamedQuery("mydosageQuery");
query.setString("dosageForm", "SLS");
display(query.list());

Bind using setProperties:

query = session.getNamedQuery("mydosageQuery");
query.setProperties(new MyDosageQuery(“SLS”));
display(query.list());

(This is more of a note to self really...)