0

Hibernate Lazy Initialization exception solved by Long Session

asked 2009-03-18 20:48:01 +0800

edudant gravatar image edudant
219 1 1 1
zk.datalite.cz

updated 2009-03-18 21:10:25 +0800

We use, MVC architecture with Spring + JPA and databinding (very similar solution as in http://www.zkoss.org/smalltalks/mvc4/). In the beginning we used only JPA API functions and Toplink Essentials as JPA provider. After some time, we needed some advanced configuration and tried to switch to Hibernate JPA.

However to my surprise lot of (previously unknown) troubles with lazy loading occurred. I tried OpenSessionInView design pattern, but it helped only a little (master list and detail form on selected row - list is loaded in first request, detail in update – detail contains lazy loaded collection). After some study I found, that Toplink essentials load data from lazy collection even on detached objects (it is for me some magic, because I didn’t found any documentation on it. Probably L2 cache is first updated and results from cache are used).

I studied “Hibernate in action” for the solution. It says “your first choice should be
to fetch the complete required graph in the first place“ – but often it is not feasible to decide what to fetch in advance. Usually in update request, I can decide what to reload from database, but it is very inconvenient for the programmer. By using entity collections - I can leverage databinding for detail form from my example without coding anything (and it worked with Toplink Essentials)!

In Hibernate OpenSessionInView tutorial at http://www.hibernate.org/43.html I found chapter regarding this issue: Why can't Hibernate just load objects on demand?. It describes drawbacks of loading objects on deamend as Toplink does. Frankly, I still thing that toplinks solution is better.

Based on this article and chapter 8.2.4 (Using a long session) of Hibernate in Action I tried another solution. It talks about “application transaction.“ It sounds to me like reasonable term with RIA application. Indeed what I do is something like app transaction. On web form I navigate on multiple UI items with master/detail data. And what this transaction usually demarcate is ZK's desktop. I tried the solution proposed in this chapter with desktop as holder of session and it works!

I attach entity manager to desktop and while desktop is alive, it holds only one entity manager all the time. Because DB connection is expensive resource, in cleanup procedure I remove DB connection from EM and in init procedure I return fresh connection from the pool. Because all changes are made on service layer within transaction, no commit or rollback is needed in here. I register following class in zk.xml as a zk listener.

public class HibernateLongSession implements ExecutionInit, ExecutionCleanup {
    public void init(Execution exec, Execution parent) {
        if (parent == null) { //the root execution of a servlet regest
            // actual desktop
            Desktop desktop = exec.getDesktop();
            
            // entity manager factory
            EntityManagerFactory emf = (EntityManagerFactory) SpringUtil.getBean("entityManagerFactory");

            // if entity manager is not yet set
            if (!TransactionSynchronizationManager.hasResource(emf))
            {
                // get entity manager from session
                EntityManager entityManager = (EntityManager) desktop.getAttribute("HibernateEntityManager");

                if (entityManager == null)
                {
                    // not found, create new one (probably first call on this desktop)
                    entityManager = emf.createEntityManager();
                }
                else
                {
                    // found, I shoud reconnect database  connection
                    Session hibernateSession = (Session) entityManager.getDelegate();
                    hibernateSession.reconnect();

                }

                // setup entity manager for spring (it uses it from collection
                TransactionSynchronizationManager.bindResource(emf, new EntityManagerHolder(entityManager));
            }
        }
    }

    //-- ExecutionCleanup --//
    public void cleanup(Execution exec, Execution parent, List errs) {
        if (parent == null) { //the root execution of a servlet request
            Desktop desktop = exec.getDesktop();
            EntityManagerFactory emf = (EntityManagerFactory) SpringUtil.getBean("entityManagerFactory");

            EntityManagerHolder emHolder = (EntityManagerHolder) TransactionSynchronizationManager.getResource(emf);
            TransactionSynchronizationManager.unbindResource(emf);

            if (emHolder != null )
            {
                EntityManager entityManager = emHolder.getEntityManager();

                // instead of closing entity manager just close database connection
                Session hibernateSession = (Session) entityManager.getDelegate();
                if (hibernateSession.isConnected() && hibernateSession.isOpen())
                        hibernateSession.disconnect();

                desktop.setAttribute("HibernateEntityManager", entityManager);
            }
     }
}

I still have some concerns with this solution and I will very appreciate any comments from you. Main concern is about memory usage, while first level cache in entity manager itself will be open on the server between requests. However, ZK probably holds these objects in its model anyway. Another example is search form, where lot of different object is used within one desktop (possible solution is reconnect hibernate session from code in appropriate place (before search ) as proposed in Hibernate book).

What do you thing about this idea? Do you see any major drawbacks?

Thanks.

Jiri Bubnik

delete flag offensive retag edit

26 Replies

Sort by » oldest newest

answered 2009-03-19 12:36:44 +0800

robertpic71 gravatar image robertpic71
1275 1

updated 2009-03-19 12:37:31 +0800

Hi

I've done only small projects with zk+spring+hibnernate(JPA), but this could be an option. The drawback could be the growing 1stLevel Cache and holding objects for update - longer as required. Maybe you need a manual flush after the long transaction. It depends on the application.

I've read the book "Java Persitence with Hibernate" (the follow-up from hibernate in action) and there is a description for "conversations" aka long Transactions aka "Unit of Work".

The most difference is this line:

@PersistenceContext(type = PersistenceContextType.EXTENDED)
public void setEntityManager(EntityManager entityManager) {
     this.entityManager = entityManager;	
}

PersistenceContextType.EXTENDED should do similar things like your code. In fact, the Context will not closed.
Instead of manuel flush, i annotate the last transaction.

Here is a composer/controller.

@Repository("testController") 
@Scope("prototype")
@Transactional(readOnly=true,propagation = Propagation.NOT_SUPPORTED)
public class PlansTest extends GenericForwardComposer {
	
	private AnnotateDataBinder binder;
	
	Planung selectedPlanung;
	private EntityManager entityManager;

	@PersistenceContext(type = PersistenceContextType.EXTENDED)
	public void setEntityManager(EntityManager entityManager) {
		this.entityManager = entityManager;
	}
	@Override
	public void doAfterCompose(Component _win) throws Exception {
		super.doAfterCompose(_win);
		_win.setVariable("model", this, false);  // make models visible for the databinder

	}
	public void onCreate$oldplanswin(ForwardEvent event) {
	    binder = (AnnotateDataBinder) event.getTarget().getVariable("binder", false);
	}
	// Test lazyLoad in the Controller 
    public void onClick$test() {
    	 listModel = (ListModelList) plansList.getModel();
		Planung planung = (Planung) listModel.get(1);
    	System.out.println("Kunde " + planung.getNummer());
    	System.out.println("Kunde " + planung.getKunde().getName()); // kunde = lazyLoad
    }

    public Collection getPlanungen() {
	Query query = entityManager.createQuery("from " + Planung.class.getName());
	List<Planung> planungen = query.getResultList();
	return (List<Planung>) planungen;
    }
    
    public Planung getSelectedPlanung() {
    	return selectedPlanung;
    }
    public void setSelectedPlanung(Planung planung) {
    	selectedPlanung = planung;
    }
    @Transactional(readOnly=false,propagation = Propagation.REQUIRED)
    public void save() {
	// this sould flush all action
    }
}

However, i do not use this until now. I've worked with "Session per Request" and detach and merge objects.
In my tests i've troubles with my ServiceLayer. The servicelayer creates his own entityManager. AND: In my real applications, i don't want the entityManager outside my DAO's and Servicelayer.

When using the extended EntityManager, i have no troubles with lazyLoad in the composer and the databinder.
Actually, i've no time to check the details. Maybe in some later projects.

/Robert

link publish delete flag offensive edit

answered 2009-03-25 07:00:46 +0800

adrianlim83 gravatar image adrianlim83
69 1 2

Hi edudant, i still received the same exception. The following is my partially code.

zk.xml
...
<listener>
<description>Hibernate Long Session</description>
<listener-class>org.zkoss.util.HibernateLongSession</listener-class>
</listener>
...

jpa.xml

...
<bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="dataSource" ref="dataSource"/>
<property name="loadTimeWeaver">
<bean class="org.springframework.instrument.classloading.InstrumentationLoadTimeWeaver"/>
</property>
</bean>

<bean id="txManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>

<tx:advice id="txAdvice1" transaction-manager="txManager">
<tx:attributes>
<tx:method name="*" propagation="REQUIRED"/>
</tx:attributes>
</tx:advice>

<aop:config>
<aop:advisor advice-ref="txAdvice1" pointcut="execution(* org.zkoss.util.HibernateLongSession.*(..))" />
</aop:config>
...

HibernateLongSession.java:

copy and paste from your existing code


May i know is there any part i am wrong? Thanks alot for your help and idea sharing.

link publish delete flag offensive edit

answered 2009-03-25 08:25:11 +0800

adrianlim83 gravatar image adrianlim83
69 1 2

Hi edudant, I do some tracing to know what is happening in the code. It seem like exactly how we configure this in zk.xml:

...
<listener>
<description>Spring TransactionSynchronizationManager handler</description>
<listener-class>org.zkoss.zkplus.spring.SpringTransactionSynchronizationListener</listener-class>
</listener>


<listener>
<description>ThreadLocal Variables Synchronizer</description>
<listener-class>org.zkoss.zkplus.util.ThreadLocalListener</listener-class>
</listener>

<preference>
<name>ThreadLocal</name>
<value>
org.springframework.transaction.support.TransactionSynchronizationManager=
resources,synchronizations,currentTransactionName,currentTransactionReadOnly,actualTransactionActive;
org.springframework.orm.hibernate3.SessionFactoryUtils=deferredCloseHolder;
org.springframework.transaction.interceptor.TransactionAspectSupport=transactionInfoHolder;
</value>
</preference>
...

Alright, let me clarify the problem. If we have less than 20 records in the listitem (with lazy load called), it work fine, however, it happen when the records is more than approximately 22 which the lazy load initialization exception will be thrown. I am not sure how the TransactionSynchronizationManager work, but I assume that could be some bugs which couldn't able to handle more than 22 records.

link publish delete flag offensive edit

answered 2009-03-27 02:57:17 +0800

adrianlim83 gravatar image adrianlim83
69 1 2

Hi edudant, i found out the reason why. My current architecture is using spring to manage the hibernate session factory by HibernateDaoSupport instead of toplink essential. I have develop quite a number of utilities which should not be a good practice if i change to toplink essential (as i may required to change all the flow). Thus, i guess this is not applicable to my situation.

link publish delete flag offensive edit

answered 2009-03-27 12:33:04 +0800

edudant gravatar image edudant
219 1 1 1
zk.datalite.cz

updated 2009-03-27 12:37:52 +0800

Hi,

the example is not complete solution yet, unfortunetlly I don't have much time right now to clean the code and find all bugs.

With Toplink Essentials you don't need this solution at all, because there is no such thing as LazyLoadExcaption in it (they open new session internally for you- hibernate folks consider this somewhat obscure and have lot of reasons why it is wrong). So discussed solution is for hibernate only.

Anyway:
This solution is something like OpenSessionInView pattern (the part with TransactionSynchronizationManager I actually copied from spring OpenSessionInView class). You added some transaction (as to your spring configuration), but I think it is not needed. The other part of your configuration was about ThreadLocal variable synchronization, which I don't use at all - I switched of event thread parameter (thread variables might be problem, because the hibernate session is set in the listener).

Only missing part of my example is zk.xml listener configuration:

     <listener>
        <description>Hibernate Open Session In conversation life-cycle</description>
        <listener-class>cz.datalite.zkspring.config.HibernateLongSession</listener-class>
     </listener>

put breakpoint to init() and cleanup() methot to see, if everything works correctly.

As I did some more tests with this solution, I found some bugs:
- hibernate session is not closed properly - it is memory leak and out of memory exception comes very fast. Probably I should add new listener to desktop destroy event...
- I didn't figure out when and how many times the event listener is actually called, sometimes the event thread variable is set to some hibernate session before init() listener
- hibernate now does some tricks with jdbc connection and the solution with disconnect() and reconnect() doesn't work as expected. Probably disconnect() is not needed anymore, because Hibernate will attach jdbc connection only while in transaction, but it needs some more investigation.

... but still - this solution works and these problems looks solvable. I will continue with my investigation. If anyone wants to try it out, I can send my source code with JMX monitors and spring configuration (send me an email: jiri dot bubnik at datalite dot cz).

... finally - I don't consider myself an expert in this field, I'm still looking for good solution. So this might be all wrong for some reason. But I don't buy the argument, that I should prepare all data in service layer or reatach entity in each request. It is just too complicated and brings much more code.

I think that everyone who uses Hibernate + ZK must fight lazy initialization problems, just because ZK is so rich RIA :-) - almost like desktop application. I'm looking for something like Seam-managed persistence context.

Am I missing some obvious solution?

link publish delete flag offensive edit

answered 2009-03-27 12:44:08 +0800

edudant gravatar image edudant
219 1 1 1
zk.datalite.cz

updated 2009-03-27 12:45:06 +0800

> To robertpic71:

Many thanks for your ideas. I have read some more about persistence contexts in "Java Persitence with Hibernate", but this doesn't look like solution for me.

PersistenceContextType.EXTENDED is usable only for statefull components and I have to manage transactions myself. But I like my transactions on service layer (with stateless beans) and standard way of transaction propagation. The only thing I need is lazy loading on presentation layer.

My solutions is equivalent to OpenSessionInView (which is actually recommended by many hibernate folks), while your is truly long transaction with commit after some amount of request (wizard screen).

Or is there a way how to uses extended persistence context for my purpose?

Regards,

Jiri Bubnik

link publish delete flag offensive edit

answered 2009-03-27 14:03:37 +0800

robertpic71 gravatar image robertpic71
1275 1

>> PersistenceContextType.EXTENDED is usable only for statefull components and I have to manage transactions myself. But I like my >> transactions on service layer (with stateless beans) and standard way of transaction propagation. The only thing I need is lazy loading on presentation layer.

Yes, that's a little bit dirty. The next EJB/JPA should remove this handicap. Meanwhile i use Session per Request and detached Objects.

The book do not recommend this way until the next JPA-Update. Another way is to set the (Hibenate) Flushmode to Manual. Of course is also a little bit dirty, because you have to override Hibernatesettings outside the servicelayer.

However, i was a little bit suprised that the Open-Session-in-View do not solve the problem. Any book/articel say: Use OSIV and forget the lazyloading-problems. It seems that OSIV-Pattern from ZK do not spawn all loadings (only zul - no XML-Requests).

And here a complete other resolution from an german forum:

remote-lazy-loadingl

This solution uses an aspect to catch the LazyInitializationException in the domainbean. It's like toplink - it produces overhead (extra SQL) but it works.

/Robert

link publish delete flag offensive edit

answered 2009-03-28 12:53:59 +0800

edudant gravatar image edudant
219 1 1 1
zk.datalite.cz

Open-Session-In-View should be named Open-Session-In-Http-Request, because that what is does. It keeps all entities in one request managed. I suppose it was suficcient solution with JPS pages, when you always get new entities from service layer in each request. But with ZK I hold entities between requests and they became detached (thus lazy loading not available). What I need is someting like Open-Session-In-Conversation.

Catching LazyInitializationException with AOP seems a bit tricky :-), however it might work. It is problably usefull for workaround with remote client (with ZK we are in the same JVM, only out of transaction). It opens new entity manager, new transcation, reatach entity (means extra query) and then loads data.

I will give more time to Long Session solution, but if it fails, I will give remote lazy loading a shot and inform you in this thread.

Thank you,

Jiri

link publish delete flag offensive edit

answered 2009-03-30 02:02:05 +0800

adrianlim83 gravatar image adrianlim83
69 1 2

Hi edudant, when is your solutions release? Anyhow, I would highly appreciate your solutions and the way you simplify the it to the easier way.

link publish delete flag offensive edit

answered 2009-03-30 02:07:22 +0800

adrianlim83 gravatar image adrianlim83
69 1 2

@edudant, between, if you need help because of time issue I could properly help on some part. However, I may need some understanding of the algorithms and architecture design

link publish delete flag offensive edit
Your reply
Please start posting your answer anonymously - your answer will be saved within the current session and published after you log in or create a new account. Please try to give a substantial answer, for discussions, please use comments and please do remember to vote (after you log in)!

[hide preview]

Question tools

Follow

RSS

Stats

Asked: 2009-03-18 20:48:01 +0800

Seen: 12,685 times

Last updated: Oct 09 '09

Support Options
  • Email Support
  • Training
  • Consulting
  • Outsourcing
Learn More