0

Building zk front end to handle multiple simultaneous long running data queries?

asked 2011-10-31 18:42:05 +0800

davout gravatar image davout
1429 3 18

I have design problem that I'm looking to solve with ZK.

There is a requirement to build a web app to support a portal style multi window display where each window is displays two panes: a query statement in the top pane, and the query results in a grid in the second pane. The queries are being sent to an underlying custom analytics engine. Each query window need to be modeless - think MDI. The data queries to the underlying custom analytics engine can be very long running - 2 minutes plus sometimes to return results.

To address this I want to provide the user with a MDI style interface where they can:
* Open a new query window
* Type in the query string and select 'go'. Under the hood the code will fire off the query to the remote custom analytics engine
* Open a second query window
* In the second query window type in a new query and select 'go'
* Repeat this for a third window and so on.
* Once a query window has received its results, the user is free to change the query statement and fire off a second different query.

Behind the scenes something has to receive a notification that a query has completed and that the related window grid needs to be updated with the results. There is no indication of when a query will be completed by the underlying custome analytics engine. If my app fires off queries from windows A, B C and D; the results could return in any order.

Any suggestions for how to deliver this in ZK?

delete flag offensive retag edit

28 Replies

Sort by ยป oldest newest

answered 2011-10-31 21:22:29 +0800

gekkio gravatar image gekkio flag of Finland
899 1
http://gekkio.fi/blog

updated 2011-10-31 21:22:42 +0800

Let me make sure I understood correctly: the user must be able to use the other query windows while a window is executing a query?

If that's true, here's some quick thoughts:

1. You will need server push
Polling server push is not very good, so you probably want to look at ZK EE comet server push.

2. You will (probably) need a thread pool for background processing
I'm guessing your custom analytics engine calls are not asynchronous, so you will absolutely need a thread pool that will do the blocking calls.
If you create Thread objects on the fly, your application will be crushed in high load situations.

3. Try to encapsulate the responsibilities of the windows to separate composers
The naive approach would be to have two Composer classes:
- MainComposer which manages the windows (= creates windows/closes them/etc)
- WindowComposer which manages a single window (you would have one WindowComposer for each window)

Do NOT create a huge composer which has a gigantic list of Window components that you try to manage in a single place.

4. Prefer asynchronous server push instead of synchronous
This basically means that do NOT call Executions.activate/deactivate, but use Executions.schedule instead.

I can provide more detailed explanations later, but now I need some sleep :)

link publish delete flag offensive edit

answered 2011-10-31 22:34:41 +0800

davout gravatar image davout
1429 3 18

Thanks for the initial input... more insights would be very useful.

If I can get a basic prototype working I'd like to share the source code with the community, since I suspect its a common business problem.

link publish delete flag offensive edit

answered 2011-11-02 21:11:03 +0800

davout gravatar image davout
1429 3 18

Hi... any further suggestions?

link publish delete flag offensive edit

answered 2011-11-03 18:20:41 +0800

gekkio gravatar image gekkio flag of Finland
899 1
http://gekkio.fi/blog

updated 2011-11-03 18:21:31 +0800

Here's some code for the ideas I presented earlier. This is all from the top of my head so it might not even compile, but these should show the basic building blocks you will need to implement your application.

AnalyticsThreadPool.java:
This class manages the thread pool. In my example the pool size is 20, so that means that no more than 20 active requests are supported. Requests that cannot be started right away are placed into a queue so they will be eventually processed.
Don't use a too large thread pool (more than 100 is definitely overkill). You need to register this class as a listener in zk.xml!

public class AnalyticsThreadPool implements WebAppInit, WebAppCleanup {
  private static ExecutorService EXECUTOR_SERVICE;
  
  public void init(WebApp webApp) {
    if (EXECUTOR_SERVICE == null) {
      EXECUTOR_SERVICE = Executors.newFixedThreadPool(20);
    }
  }
  public void cleanup(WebApp webApp) {
    EXECUTOR_SERVICE.shutdownNow();
    EXECUTOR_SERVICE = null;
  }
  public static void submit(Runnable task) {
    EXECUTOR_SERVICE.submit(task);
  }
}

AnalyticsTask.java:
This class represents a single query and stores the desktop so that the query results can be asynchronously sent back to ZK.

public class AnalyticsTask implements Runnable {
  private final Desktop desktop;
  private final EventListener eventListener;
  private final AnalyticsParams queryParams;

  public AnalyticsTask(Desktop desktop, EventListener eventListener, AnalyticsParams queryParams) {
    this.desktop = desktop;
    this.eventListener = eventListener;
    this.queryParams = queryParams;
  }

  public void run() {
    AnalyticsResult result = ... // run the query
    if (desktop.isAlive()) {
      Executions.schedule(desktop, eventListener, new AnalyticsEvent(result));
    }
  }
}

AnalyticsEvent.java:
This class is a custom event that contains results from a single query.

public class AnalyticsEvent extends Event {
  public static final String NAME = "onAnalyticsResult";

  public final AnalyticsResult result;

  public AnalyticsEvent(AnalyticsResult result) {
    super(NAME, null, null);

    this.result = result;
  }
}

WindowComposer.java:
This class shows the basic stuff you need in your window composer.

public class WindowComposer extends ... {

  public void doAfterCompose ... {
    ...
    if (!desktop.isServerPushEnabled()) {
      desktop.enableServerPush(true);
    }
  }

  public void onClick$submit() { // Just an example. You can of course submit AnalyticsTasks from any method, not just onClick
    AnalyticsParams params = ... // Read all parameters from somewhere
    AnalyticsTask task = new AnalyticsTask(desktop, this, params);
    AnalyticsThreadPool.submit(task);
  }

  // This will be called when the results are ready
  public void onAnalyticsResult(AnalyticsEvent event) {
    AnalyticsResult result = event.result;
    // Do stuff with result
  }

}

AnalyticsParams.java:
This is something you have to write yourself. Make a class that contains all the parameters you need to make a query. You must NOT include any ZK components or ZK stuff here.

public class AnalyticsParams {
  // ?
}

AnalyticsResult.java:
This is also something you have to write. Make a class that contains all the results of a query.

public class AnalyticsResult {
  // ?
}

link publish delete flag offensive edit

answered 2011-11-04 08:46:38 +0800

terrytornado gravatar image terrytornado flag of Germany
9393 3 7 16
http://www.oxitec.de/

Bookmarked. Thanks gekkio.

link publish delete flag offensive edit

answered 2011-11-04 10:16:24 +0800

benbai gravatar image benbai
2228 6
http://www.zkoss.org

I guess the UI will like this one:

ZKFiddle-Link

query_page.zul
<zk>
<window title="show busy" border="normal">
<attribute name="onLater">
// do some query then clear busy
try{
int i = 0;
boolean queryExit = false;
// query logic here
while (!queryExit) {
i ++;
if (i == 10)
queryExit = true;
else
Thread.sleep(1000);
}
} catch (Exception e) {
alert("oops !!");
}
query_output.setValue(" this is the query result for the quer - " + query_input.getValue());
Clients.clearBusy(self);
</attribute>
<vbox>
<textbox id="query_input" rows="2" value="input query string" />
<textbox id="query_output" rows="2" value="the query result will output here"/>
<button label="do query">
<attribute name="onClick">
// show busy message here
Window win = self.getParent().getParent();
Clients.showBusy(win, "Information is being queried, please wait...");
Events.echoEvent( "onLater", win, null ) ;
</attribute>
</button>
</vbox>
</window>
</zk>

index.zul
<zk>
<div id="query_div" />
<button label="open query window">
<attribute name="onClick">
Iframe iframe = new Iframe();
iframe.setSrc("query_page.zul");
iframe.setParent(query_div);
</attribute>
</button>
</zk>

link publish delete flag offensive edit

answered 2011-11-04 11:26:21 +0800

gekkio gravatar image gekkio flag of Finland
899 1
http://gekkio.fi/blog

No iframes are needed. The whole thing is based on server push instead of iframes.

link publish delete flag offensive edit

answered 2011-11-04 15:26:18 +0800

davout gravatar image davout
1429 3 18

>> You need to register this class as a listener in zk.xml! <<

How?

link publish delete flag offensive edit

answered 2011-11-04 16:52:43 +0800

gekkio gravatar image gekkio flag of Finland
899 1
http://gekkio.fi/blog

Check this out.

Basically you just add this:

<listener>
  <listener-class>yourpackage.AnalyticsThreadPool</listener-class>
</listener>

link publish delete flag offensive edit

answered 2011-11-05 04:01:32 +0800

RichardL gravatar image RichardL
768 4

Is there any reasin why you can't use iframes? Server push is heavy on the server and it's worth checking out lighter-weight options first.

Straight iframes would be the first choice, but if you want your windows to be able to communicate with each other and/or the containing component, maybe you could check out EventQueue. If you have ZK EE you can use group scope and this will allow you to post events to all desktops of the browser windows belonging the same group of browser windows.

This doesn't use server push so if you want to want the containing window to be able to recieve events from the queue, you'd need to have a timer on the containing window to periodically subscribe to the queue (null events if nothing is happening).

If you're interested in this approach, check out this link http://blog.zkoss.org/index.php/2010/07/13/event-queue-communication-among-inline-frames-without-server-push/.

link publish delete flag offensive edit
Your reply
Please start posting your answer anonymously - your answer will be saved within the current session and published after you log in or create a new account. Please try to give a substantial answer, for discussions, please use comments and please do remember to vote (after you log in)!

[hide preview]

Question tools

Follow

RSS

Stats

Asked: 2011-10-31 18:42:05 +0800

Seen: 1,257 times

Last updated: Jan 28 '12

Support Options
  • Email Support
  • Training
  • Consulting
  • Outsourcing
Learn More