Million Record Challenge

answered 2008-10-31 18:56:53 +0800

hkn
246 ● 3

Hi everybody!

Even I think this competition isn't really fair I found it interesting enough to spend some with it. I just make use of a paging listbox with four columns. The implementation is simple and stupid as I did no special optimization. To be honest: 1.000.000 seem impossible with this approach. I just tested up to 50.000 successfully. May an ItemRender will speed up loading? I have no idea... Maybe one of the ZK gurus can show how to speed it up! ;-)
And by the way: The Flex example is really impressive!

/ Horst

Maybe someone is interested to improve my quickshot:

The ZUL file:
<?page id="mio" title="1 Million Records" cacheable="false"
language="xul/html" zscriptLanguage="Java" contentType="text/html;charset=UTF-8"?>
<?init class="org.zkoss.zkplus.databind.AnnotateDataBinderInit"?>
<zk xmlns="http://www.zkoss.org/2005/zul"
xmlns:h="http://www.w3.org/1999/xhtml"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.zkoss.org/2005/zul http://www.zkoss.org/2005/zul/zul.xsd">
<window title="My First Window" border="normal" width="600px"
height="95%" id="win" use="Crtl">
<hbox>
<combobox id="cb" readonly="true" >
<comboitem id="cb1" value="1" label="10.000" />
<comboitem id="cb2" value="2" label="20.000" />
<comboitem id="cb3" value="3" label="30.000" />
</combobox>
</hbox>
<listbox id="lb" width="560px" height="550px" mold="paging" pageSize="100" >
<listhead>
<listheader label="col 1" sort="auto" />
<listheader label="col 2" sort="auto" />
<listheader label="col 3" sort="auto" />
<listheader label="col 4" sort="auto" />
</listhead>
<listitem self="@{each=itm}">
<listcell label="@{itm.a}" />
<listcell label="@{itm.b}" />
<listcell label="@{itm.c}" />
<listcell label="@{itm.d}" />
</listitem>

</listbox>
</window>
</zk>

And here is my backend java class: Crtl.java
//------------------------------------------ START Crtl.java -----------
import java.util.Random;
import org.zkoss.zk.ui.Components;
import org.zkoss.zk.ui.event.Event;
import org.zkoss.zk.ui.ext.AfterCompose;
import org.zkoss.zkplus.databind.AnnotateDataBinder;
import org.zkoss.zul.Combobox;
import org.zkoss.zul.ListModelList;
import org.zkoss.zul.Listbox;
import org.zkoss.zul.Window;

public class Crtl extends Window implements AfterCompose {

protected AnnotateDataBinder binder;
protected Combobox cb;
protected Window win;
protected Listbox lb;

final static int anz1 = 10000;
final static int anz2 = 20000;
final static int anz3 = 30000;
final static ListModelList data1 = new ListModelList(anz1);
final static ListModelList data2 = new ListModelList(anz2);
final static ListModelList data3 = new ListModelList(anz3);
Random r;

public void onCreate$win(Event e) throws Exception {

binder = new AnnotateDataBinder(win);
// CfgConfigsDAO dao = new CfgConfigsDAO();
// cfg.setAttribute("dao",dao);
binder.loadAll();
}

// @Override
public void afterCompose() {
// wire variables
Components.wireVariables(this, this);
// NO need to register onXxx event listeners auto forward
Components.addForwards(this, this);

}

public class ListData implements Comparable<ListData>{
String a;
String b;
String c;
String d;

public ListData(String a, String b, String c, String d) {
this.a = a;
this.b = b;
this.c = c;
this.d = d;
}

public String getA() {
return a;
}

public void setA(String a) {
this.a = a;
}

public String getB() {
return b;
}

public void setB(String b) {
this.b = b;
}

public String getC() {
return c;
}

public void setC(String c) {
this.c = c;
}

public String getD() {
return d;
}

public void setD(String d) {
this.d = d;
}

@Override
public int compareTo(ListData o) {
return this.a.compareTo(o.getA());
}

}

private void create(ListModelList data, int anz) {
int j = 0;
for (int i = 1; i <= anz; i++) {
j++;
if (j == 1000) {
System.out.println("i=" + i);
j = 0;
}
long l = r.nextLong();
String a = String.format("" + i + " - %1$x", l);
l = r.nextLong();
String b = String.format("%1$x", l);
l = r.nextLong();
String c = String.format("%1$x", l);
l = r.nextLong();
String d = String.format("%1$x", l);
ListData o = new ListData(a, b, c, d);
data.add(o);
}
}

public Crtl() {
super();
r = new Random(5343336);
if (data1.isEmpty()) {
System.out.println("Start");
create(data1, anz1);
System.out.println("Done1");
create(data2, anz2);
System.out.println("Done2");
create(data3, anz3);
System.out.println("Done3");
}
}

public void onSelect$cb(Event ev) throws InterruptedException {
System.out.println("SELECT " + cb.getValue());
lb.setModel(getData());
}

public ListModelList getData() {

if (cb.getSelectedItem() == null)
return new ListModelList();
String iv = (String) cb.getSelectedItem().getValue();
System.out.println("get DATA " + iv);
if (iv.equalsIgnoreCase("1")) {
System.out.println("anzahl1=" + data1.getSize());
return data1;
} else if (iv.equalsIgnoreCase("2")) {
System.out.println("anzahl2=" + data2.getSize());
return data2;
} else if (iv.equalsIgnoreCase("3")) {
System.out.println("anzahl3=" + data3.getSize());
return data3;
}
return new ListModelList();
}
}
//------------------------------------------ END Crtl.java -----------

answered 2008-11-01 03:44:13 +0800

henrichen
3869 ● 2

Please see this smalltalk.

http://www.zkoss.org/smalltalks/loadondemand/

answered 2008-11-01 11:30:35 +0800

hkn
246 ● 3

Hello Henri,

yes thanks for that hint. My naive expectation was that when I use
a listbox with mold="paging" and a model object of ListModel* or whatever
(here ListModelList) that zk does this automaticaly.
When working with a sql database it makes sense to control the paging
by program as this can be linked to SELECT phrase "limit row_Start to row_end"
but when using a list of POJOs it would be great if zk can do this for us.

kind regards

Horst

answered 2008-11-02 09:16:29 +0800

hkn
246 ● 3

Hello zkfan!

Yes and no, zk has a server centric approach. So solutions need to be different, and for zk you have to implement sorting etc and some kind of paging. When I try to load a really large number of records while I try to keep my records in memory I get some out of memory exceptions even if I assign the jvm plenty of mem (about 2GB). Further, my FF collects memory like hell. The Flex speed is really impressive. Nevertheless zk has advantages when the data is read from databases and with databinding it is really great.

So I don't really thing that the approaches are comparable, but
First: paging as described in the mentioned smalltalk needs to much coding if the data is stored in a list of pojos. That should work automatically.
Second: Why do I need the paging mold. The scrollbar events are sufficient to calculate the visible part in a list box. Sorting of a pojo list requires almost nothing if the getters return values are of type string, int, float etc. So the browser just needs to hold the visible part and the Listbox would delive a server centric solution which is comparable to the Flex solution.
May I find some time to implement this - christmass is coming soon :-)

Horst