0

Filedownload - Memory Leak

asked 2017-03-29 14:47:48 +0800

sensdb gravatar image sensdb
21 2

Hi there,

I suspect a memory-leak in the Filedownload.save-Method.

Executing the following command a couple of hundred times for the same > 1MB-File without any further actions and the Java GC is not able to fully release the Memory

Filedownload.save(data, contentType, filename);

I tried implementing a DownloadServlet, executed via

Session currentSession = Sessions.getCurrent();
currentSession.setAttribute(DownloadServlet.DATA, data);
currentSession.setAttribute(DownloadServlet.FILE_NAME, filename);
currentSession.setAttribute(DownloadServlet.MYME_TYPE, contentType);

Execution currentExec = Executions.getCurrent();
currentExec.sendRedirect("DownloadServlet");

unfortunatelly after executing this command once no further ZK-Events will be sent/received!

delete flag offensive retag edit

10 Replies

Sort by ยป oldest newest

answered 2017-03-29 14:54:22 +0800

Darksu gravatar image Darksu
1991 1 4

Hello sensdb,

It all depends on when the GC will release the memory allocated.

A trick is to use the following:

byte[] byte1=new byte[10000000];
byte1=null;
System.gc();

Best Regards,

Darksu

link publish delete flag offensive edit

answered 2017-03-29 15:26:46 +0800

sensdb gravatar image sensdb
21 2

Hi,

thanks for the fast reply.

Unfortunately it didn't do the trick: downloading a 6.75MB-File 54 times and I received a heap space error!

Any more ideas? I already tried duplicating the Filedownload-code but with releasing the memory explicitly but it also didn't work.

Thanks in advance...

link publish delete flag offensive edit

answered 2017-03-29 20:46:57 +0800

Darksu gravatar image Darksu
1991 1 4

Hello,

I guess increasing the heap space will only solve your problem temporarily but it is worth the shot.

Also it would be interesting to see how you initialize and release your variables/collections.

Best Regards,

Darksu

link publish delete flag offensive edit

answered 2017-03-30 08:20:45 +0800

sensdb gravatar image sensdb
21 2

Hi,

here is a snippet of my code:

@Command
public void downloadEntityMedia(@BindingParam(VAL)
final MyEntity myEntity) {
    MyEntityContent ec = deviceLI.getMyEntityContent(getSubject(), myEntity);
    download(ec.getFileName(), null, ec.getContent());
}

where getMyEntityContent is:

public MyEntityContent getMyEntityContent(Subject subject, MyEntity myEntity) throws SensorDBException {
    Query q = em.createNamedQuery(MyEntityContent.QUERY_GET_BY_ID);
    q.setParameter(MyEntityContent.PARAM_ID, myEntity.getEntityId());
    List<MyEntityContent> resultList = q.getResultList();
    if (resultList.size() == 1) {
        return resultList.get(0);
    }
    return null;
}

and download is:

public void download(String filename, String contentType, byte[] data) {
    // OLD -> leads to disabled eventing
    // Session currentSession = Sessions.getCurrent();
    // currentSession.setAttribute(DownloadServlet.DATA, data);
    // currentSession.setAttribute(DownloadServlet.FILE_NAME, filename);
    // currentSession.setAttribute(DownloadServlet.MYME_TYPE, contentType);
    // Execution currentExec = Executions.getCurrent();
    // currentExec.sendRedirect("DownloadServlet");
    // via FILEDOWNLOAD
    Filedownload.save(data, contentType, filename);
    data = null;
    System.gc();
}

As you already mentioned, increasing the heap space is only a temporary solution and I would really prefer a stable solution :-/

Tanks again

link publish delete flag offensive edit

answered 2017-03-30 08:57:58 +0800

cor3000 gravatar image cor3000
4143 1 7
ZK Team

updated 2017-03-30 08:58:41 +0800

when suspecting a memory leak: one way to find the source of the problem is to take a memory dump. This can be done easily with $JAVA_HOME/bin/jvisualvm. Then look into the dump to see which objects hold references to the largest byte-arrays. Usually that leads to the root of the problem.

Btw: your original idea ...

Execution currentExec = Executions.getCurrent();
currentExec.sendRedirect("DownloadServlet");

... will stop the client engine. If you define a target frame (either an iframe or a new browser tab) it will continue to work:

Execution currentExec = Executions.getCurrent();
currentExec.sendRedirect("DownloadServlet", "_blank"); 
//or use the id of a hidden download frame (which is what ZK does when using FileDownload.save() to avoid the unload event)

Then the client engine will not stop due to an onBeforeUnload event triggered by the browser when starting a download in the same frame

link publish delete flag offensive edit

answered 2017-03-30 09:18:30 +0800

cor3000 gravatar image cor3000
4143 1 7
ZK Team

I just checked the implementation: ZK will cache the download medias (containing the byte-arrays) in the current DesktopImpl object

https://github.com/zkoss/zk/blob/v8.0.2.2/zk/src/org/zkoss/zk/ui/impl/DesktopImpl.java#L176

as you can see here, ZK will create a new cache on demand which holds up to 500 downloadable media objects - for Up to 15 minutes. Then it will start cleaning the cache whenever housekeep() is called - which happens at various occasions.

https://github.com/zkoss/zk/blob/v8.0.2.2/zk/src/org/zkoss/zk/ui/impl/DesktopImpl.java#L449-L459

So you won't have a real memory leak since the data is kept for 15 minutes and then cleaned up. It will also be cleaned up earlier if you close the browser tab or refresh the current URL. Which will call Desktop.destroy()

https://github.com/zkoss/zk/blob/v8.0.2.2/zk/src/org/zkoss/zk/ui/impl/DesktopImpl.java#L969

link publish delete flag offensive edit

answered 2017-03-30 10:22:14 +0800

sensdb gravatar image sensdb
21 2

Hi,

after some "code-sniffing" I also found the DesktopImpl (lets call it) issue: so basically the memory will be released after Session-timeout or 15 minutes by CachedMap-lifetime, so this calms me as I see no real use-case where a user downloads big files again and again and again ;-)

Thanks a lot I see now that there is not a real problem...

link publish delete flag offensive edit

answered 2017-03-30 10:39:05 +0800

cor3000 gravatar image cor3000
4143 1 7
ZK Team

yes the session timeout will be the latest point in time when the cache is cleared. In many cases that will happen earlier either after the 15 minutes or when a browser tab is closed / reloaded (even if the session continues in different browser tab).

link publish delete flag offensive edit

answered 2017-03-30 15:09:18 +0800

sensdb gravatar image sensdb
21 2

Hi cor3000,

I'm kind of intrigued by

In many cases that will happen earlier either after the 15 minutes or when a browser tab is closed / reloaded (even if the session continues in different browser tab).

Do I have to configure/implement something special for this feature or am I looking at the wrong place? Simply reloading or closing a tab doesn't call the Desktop's destroy-Method (tested with IE11 and Opera neon)...

link publish delete flag offensive edit

answered 2017-03-31 01:55:43 +0800

cor3000 gravatar image cor3000
4143 1 7
ZK Team

updated 2017-03-31 01:56:14 +0800

yes there is a configuration that affects this (and the usual browser related security features)

  1. if using Desktop Recycling then destroy won't be called, as the desktop instance is cached for later reuse.

  2. sometimes the desktop cleanup mechanism is interrupted by browsers (could be the case if an older ZK version is used).

In general it works like that (just to give you a few points where to start debugging): Once the browser closes a tab/reloads a page/navigates to another URL it triggers an onBeforeUnload event. ZK listens to this in order to send a final request to the server to remove the desktop (rmDesktop) which looks like this:

http://server:[port]/appname/zkau?dtid=zab23&cmd0=rmDesktop&opt_0=i

You can verify those events in your browser's network tab (F12 in Chrome and don't forget to enable 'Preserve log') or using other network monitoring tools such as fiddler2 or wireshark.

Without this event (e.g. the browser crashed, or security settings/constraints prevent AJAX calls in onBeforeUnload) ZK doesn't know the client is gone, and keeps the Desktop in the cache until the Session times out, or the max number of desktops per session is reached, or the desktop timeout was exceeded (however a desktop timeout doesn't clean the desktop immediately it merely makes it eligible for GC)

link publish delete flag offensive edit
Your reply
Please start posting your answer anonymously - your answer will be saved within the current session and published after you log in or create a new account. Please try to give a substantial answer, for discussions, please use comments and please do remember to vote (after you log in)!

[hide preview]

Question tools

Follow
1 follower

RSS

Stats

Asked: 2017-03-29 14:47:48 +0800

Seen: 48 times

Last updated: Mar 31 '17

Support Options
  • Email Support
  • Training
  • Consulting
  • Outsourcing
Learn More