Tags
People
Badges
Help-
CORE FRAMEWORK
FEATURED COMPONENTS
-
RESOURCES
-
LEARN ZK
RESOURCES
Hi,
We are getting a lot of messages of hung Threads in many applications developed with ZK framework. The first time we saw the error "... may be hung" in the SystemOut.log file we thought was the event thread model, so we made a change in the zk.xml and added the following configuration
(WebContainer : 4428" (000011c3) has been active for 66757 milliseconds and may be hung)
<system-config>
<disable-event-thread>false</disable-event-thread>
</system-config>
Once we made the change we put againt the application in the production enviroment but we got the same errors in more recurrent than before.
The ZK version in our applications is ZK 5.0.5 and this applications has between 500 and 600 users.
The applications are running under Websphere Application Server 6.1x and under SunOS 5.10, we analyzed the thread dump hoping to find some errors in our classes but we didn't find anything.
All the stacktrace in the thread dump are like this:
Stacktrace 1
at java.lang.Object.wait(Native Method) - waiting on [0xfffffffe8267b180] (a java.lang.Object) at org.zkoss.zk.ui.impl.UiEngineImpl.doActivate(UiEngineImpl.java:1687) - locked [0xfffffffe8267b180] (a java.lang.Object) at org.zkoss.zk.ui.impl.UiEngineImpl.activate(UiEngineImpl.java:1636) at org.zkoss.zk.ui.impl.UiEngineImpl.desktopDestroyed(UiEngineImpl.java:192) at org.zkoss.zk.ui.impl.SimpleDesktopCache.desktopDestroyed(SimpleDesktopCache.java:121) at org.zkoss.zk.ui.impl.SimpleDesktopCache.stop(SimpleDesktopCache.java:178) - locked [0xfffffffe8225d488] (a org.zkoss.zk.ui.impl.SimpleDesktopCache$Cache) at org.zkoss.zk.ui.impl.SessionDesktopCacheProvider.sessionDestroyed(SessionDesktopCacheProvider.java:59) at org.zkoss.zk.ui.impl.AbstractWebApp.sessionDestroyed(AbstractWebApp.java:321) at org.zkoss.zk.ui.http.WebManager.sessionDestroyed(WebManager.java:386) at org.zkoss.zk.ui.http.HttpSessionListener23.sessionDestroyed(HttpSessionListener23.java:45) at com.ibm.ws.webcontainer.httpsession.SessionContext.sessionDestroyedEvent(SessionContext.java:3559) at com.ibm.ws.webcontainer.httpsession.SessionData.sessionUnbound(SessionData.java:1018) at com.ibm.ws.webcontainer.httpsession.SessionContext.notifySessionInvalidated(SessionContext.java:1094) at com.ibm.ws.webcontainer.httpsession.SessionData.invalidate(SessionData.java:621) - locked [0xfffffffe81e4dff8] (a com.ibm.ws.webcontainer.httpsession.MemorySessionData) at com.ibm.ws.webcontainer.httpsession.HttpSessionFacade.invalidate(HttpSessionFacade.java:181) at org.zkoss.zk.ui.http.SimpleSession.invalidateNow(SimpleSession.java:346) at org.zkoss.zk.ui.impl.UiEngineImpl.doDeactivate(UiEngineImpl.java:1776) at org.zkoss.zk.ui.impl.UiEngineImpl.execUpdate(UiEngineImpl.java:1170) at org.zkoss.zk.au.http.DHtmlUpdateServlet.process(DHtmlUpdateServlet.java:561) at org.zkoss.zk.au.http.DHtmlUpdateServlet.doGet(DHtmlUpdateServlet.java:456) at org.zkoss.zk.au.http.DHtmlUpdateServlet.doPost(DHtmlUpdateServlet.java:464) at javax.servlet.http.HttpServlet.service(HttpServlet.java:763) at javax.servlet.http.HttpServlet.service(HttpServlet.java:856) at com.ibm.ws.cache.servlet.ServletWrapper.serviceProxied(ServletWrapper.java:276) at com.ibm.ws.cache.servlet.CacheHook.handleFragment(CacheHook.java:463) at com.ibm.ws.cache.servlet.CacheHook.handleServlet(CacheHook.java:281) at com.ibm.ws.cache.servlet.ServletWrapper.service(ServletWrapper.java:256) at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:989) at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:501) at com.ibm.ws.wswebcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:464) at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:90) at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:744) at com.ibm.ws.wswebcontainer.WebContainer.handleRequest(WebContainer.java:1455) at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:113) at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:454) at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewInformation(HttpInboundLink.java:383) at com.ibm.ws.http.channel.inbound.impl.HttpICLReadCallback.complete(HttpICLReadCallback.java:102) at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165) at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217) at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161) at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:136) at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:195) at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:743) at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:873) at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1469)
Stacktrace 2
at com.ibm.ws.webcontainer.httpsession.SessionData.getValueGuts(SessionData.java:836) - waiting to lock [0xffffffff57d24690] (a com.ibm.ws.webcontainer.httpsession.MemorySessionData) at com.ibm.ws.webcontainer.httpsession.SessionData.getValue(SessionData.java:1243) at com.ibm.ws.webcontainer.httpsession.SessionData.getAttribute(SessionData.java:1129) at org.zkoss.zk.ui.http.SimpleSession.getAttribute(SimpleSession.java:218) at org.zkoss.zk.ui.http.SimpleSession.hasAttribute(SimpleSession.java:226) at org.zkoss.zk.ui.http.SimpleSession.hasAttribute(SimpleSession.java:256) at org.zkoss.zk.ui.impl.DesktopImpl.hasAttribute(DesktopImpl.java:587) at org.zkoss.zk.ui.impl.PageImpl.hasAttributeOrFellow(PageImpl.java:389) at org.zkoss.zk.ui.AbstractComponent.hasAttributeOrFellow(AbstractComponent.java:918) at org.zkoss.zk.ui.Components$Wire.containsVariable(Components.java:1068) at org.zkoss.zk.ui.Components$Wire.wireOthers(Components.java:1034) at org.zkoss.zk.ui.Components$Wire.myWireVariables(Components.java:975) at org.zkoss.zk.ui.Components$Wire.wireVariables(Components.java:971) at org.zkoss.zk.ui.Components$Wire.access$500(Components.java:880) at org.zkoss.zk.ui.Components.wireVariables(Components.java:525) at org.zkoss.zk.ui.util.GenericAutowireComposer$BeforeCreateWireListener.onEvent(GenericAutowireComposer.java:232) at org.zkoss.zk.ui.impl.EventProcessor.process0(EventProcessor.java:177) at org.zkoss.zk.ui.impl.EventProcessor.process(EventProcessor.java:140) at org.zkoss.zk.ui.impl.UiEngineImpl.processEvent(UiEngineImpl.java:1602) at org.zkoss.zk.ui.impl.UiEngineImpl.process(UiEngineImpl.java:1386) at org.zkoss.zk.ui.impl.UiEngineImpl.execNewPage0(UiEngineImpl.java:446) at org.zkoss.zk.ui.impl.UiEngineImpl.execNewPage(UiEngineImpl.java:305) at org.zkoss.zk.ui.http.DHtmlLayoutServlet.process(DHtmlLayoutServlet.java:225) at org.zkoss.zk.ui.http.DHtmlLayoutServlet.doGet(DHtmlLayoutServlet.java:146) at javax.servlet.http.HttpServlet.service(HttpServlet.java:743) at javax.servlet.http.HttpServlet.service(HttpServlet.java:856) at com.ibm.ws.cache.servlet.ServletWrapper.serviceProxied(ServletWrapper.java:276) at com.ibm.ws.cache.servlet.CacheHook.handleFragment(CacheHook.java:463) at com.ibm.ws.cache.servlet.CacheHook.handleServlet(CacheHook.java:281) at com.ibm.ws.cache.servlet.ServletWrapper.service(ServletWrapper.java:256) at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:989) at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:501) at com.ibm.ws.wswebcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:464) at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:90) at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:744) at com.ibm.ws.wswebcontainer.WebContainer.handleRequest(WebContainer.java:1455) at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:113) at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:454) at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewInformation(HttpInboundLink.java:383) at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:263) at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214) at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113) at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165) at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217) at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161) at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:136) at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:195) at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:743) at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:873) at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1469)
Thanks in advance for your help
Greetings,
Hi
1.
Could you try to disable thread ? using ZK's default. Please refer to Event Threads
It mentioned
"By default, ZK processes an event in the same Servlet thread that receives the HTTP request. It is the suggested approach because the performance is better and it is easy to integrate with other framework"
Which means
<system-config> <disable-event-thread>true</disable-event-thread> </system-config>
2. Do you use server push ? It may also cause the issue. If you use server push, how do you use it ?
Hi samchuang, thanks for your replay.
1. Actually im using the default configuration of ZK 5, i understood zk 5 has the thread event disabled. And for example when a call a modal window or show a message box the thread is not suspended, for that reasson i had to implement code like described in the link http://books.zkoss.org/wiki/ZK_Developer%27s_Reference/UI_Patterns/Event_Threads/Messagebox.
I made a mistake writing <disable-event-thread>false</disable-event-thread>, my zk.xml looks like this
<zk>
<device-config>
<device-type>ajax</device-type>
<timeout-uri>/index.zul</timeout-uri><!-- An empty URL can cause the browser to reload the same URL -->
</device-config>
<session-config>
<session-timeout>1800</session-timeout>
<max-desktops-per-session>10</max-desktops-per-session>
<max-requests-per-session>5</max-requests-per-session>
</session-config>
<!-- Para las pruebas de rendimiento con selenium
<system-config>
<id-generator-class>com.consystec.selenium.idgenerator.ConsystecIdGenerator</id-generator-class>
</system-config> -->
</zk>
2. I don't use server push
I also noticed (Tivoli Performance - WAS) when the application has a low demand works fine (lower than 50 servlet sessions), when the number of sessions grows, begins the hanging threads immediately. Whith the event thread enable it occurs too but occasionally, so I decided to disable the event thread to minimize the incidence but the opposite happened. May be its because the amount of the application users and sessions, but i do not know what else to do.
The application its very simple, has 4 pages and in the last page redirects to others sites (http://www.youtube.com/watch?v=BEJ9HQT1vt8)
Thank you again in advance.
Anyone has the same problem with too many users ??
Hi
is't possible to provide a War file that can reproduce the issue ?
by the way, just for curious, do you have any reason for these ?
<max-desktops-per-session>10</max-desktops-per-session> <max-requests-per-session>5</max-requests-per-session>
Something I saw, when the application has the event thread enable the application works fine for a week,
then starts the hung threads. When i changed to event thread disable, the thread pool of websphere was
full in less than 30 min.
This week i'm going to deploy a new version of this application with ZK 5.0.7 EE with event thread enabled
and see what happends in production enviroment. I do not want to ask to the server administrator any load balancing, because it involves
purchasing a license from IBM and maybe serialize everything in my application.
bfontana, what application server are you using ?? Could be websphere application server or could be ZK framework, i'm
really worry about.
samchuang, the reason for these:
<max-desktops-per-session>10</max-desktops-per-session>
<max-requests-per-session>5</max-requests-per-session>
was just testing, my zk.xml just have the default tags and the disable-event-thread tag.
And in the other hand samchuang, i ask to my superior if we can provide the source code but he doesn't let me.
We are using:
Tomcat 5.5.25
Java 1.6 64 bits Sun's JVM (I don't remember update but I could find it if it helps).
RedHat Linux Enteprise Server
"physical" server is a virtualized VMWare VM with 4 Xeon cores assigned and 6 GB RAM.
So I'm afraid is not a WebSphere/ZK issue.
Hi,
Good news, problem solved. I updated the version of zk to 5.0.7, and in the xml just disabled the event threads, taking into consideration all the changes discribed in this seccion and subsection http://books.zkoss.org/wiki/ZK%20Developer's%20Reference/UI%20Patterns/Event%20Threads
Also increase the maximum threads in webcontainer to improve the performance as seen in the Tivoli Performance Viewer.
Asked: 2011-03-31 20:44:38 +0800
Seen: 838 times
Last updated: Oct 11 '11
