Tags
People
Badges
Help-
CORE FRAMEWORK
FEATURED COMPONENTS
-
RESOURCES
-
LEARN ZK
RESOURCES
My application contains a view based on a list box with a list model that implements size() and getElementAt() using a buffer that is filled on demand from the database.
Everything works fine. But when the database gets big (say in the hundreds of thousands entries) I noticed OutOfMemoryExceptions.
The culprits seems to be ListboxDataLoader.syncModel0(int, int) that has
for (; --newcnt >= 0; ++min) {
if (renderer == null)
renderer = (ListitemRenderer) getRealRenderer();
_listbox.insertBefore(newUnloadedItem(renderer, min), next);
}
and before that
final ListModel _model = _listbox.getModel(); final int newsz = _model.getSize(); final int oldsz = _listbox.getItemCount(); // ... int newcnt = newsz - oldsz;
so that initially the listbox will try to add _model.getSize() empty cells (I believe).
Now, I noticed that use mold="paging", i.e. using the "clicky" paging works better, but still, when sending ListDataEvent.CONTENTS_CHANGED because some filter changed, somehow ListboxDataLoader (and likewise GridDataLoader) start to massively create content listitems.
So my possibly naive question is: Is there actually a built-in way to safely page over a potentially large database model without actually loading more data or actually creating more ZK objects that needed for display (i.e. not correlating with the database size)?
This the posts paging-and-sorting-large-ammount-of-data and HandlinghugedatausingZK seem to indicate that I need to (essentially) build my own paging mechanism. Both posts are somewhat dated, so maybe that is not true anymore.
Any suggestion on how to do this the "ZK" way would be great.
Thanks, Henning
Hey there,
Bumped karma for links ;)
You are correct. Listbox (as well as grid) as a component will maintain a placeholder state for not-yet-rendered items. If you are looking for an out-of-the-box "everything rendered on demand" component, then I'd suggest checking the EE component biglistbox, which is designed for serving large datasets by only drawing objects when they are requested, and not keeping placeholders otherwise. (you can test it with the eval versions in mavel or direct download). It requires a matrix model implementation, but you've essentially already done most of that with your current custom ListModel.
Creating a listbox with a custom paging control lets you pull model objects on demand and only create objects for the current page, which is definitely more optimizable than the default Listbox and Listbox dataloading, which serves the general case.
The first question I'd ask is, do you use features of the Listbox component? If the goal is to display tabular data, Grid might be a better first step, since grid doesn't care about maintaining model information such as selected items, etc. This makes implementing a load-only-the-displayed range easier to implement for grid, since you don't need to maintain the additional states / features provided by Listbox.
Then from there, the other options that you mentioned are still valid. By creating the children of the listbox manually on paging events, you can control how much is rendered, rather than have the component generate placeholders.
Hi MDuchemin,
Thanks for looking into this. I would consider it not improbable that we will sign up for PE or EE - need to get some more experience with ZK however. We are having one product in the pipeline that uses ZK and one other for which ZK is a contender for a UI framework replacement.
For now I am considering limiting the data to show to a reasonable number of rows and expect users to either filter some more or actually download. Nobody is really looking at more that a 1000 rows of data in the end anyway.
Thanks, Henning
Hey Henning,
I see, limiting the amount of model data through filtering is a good option for the default case.
This type of structure always requires a bit of customization depending on the goals for the big dataset, and how to display it to the end user.
If you do need to go forward with CE options to display these large dataset, here's a possible structure:
You could build up on the regular Listbox component by having a displayModel (which contains only 1 page of data at a time) and a dataLayerModel, use a paging component and listen to paging events to pull data from the dataLayerModel into the displayModel. This way, the Listbox would only keep 1 page of data worth of listitem in memory regardless of the true model size.
If you go down this kind of route, there are some Listbox features that may need reviewing if you are using them in your page. For example, Listbox will use the ListModelList (when provided) as the selection control. If your display model no longer contains a given item (because you have cleared it an moved to a different page), then that item is no longer selected, and will not be if you move back to the previous page. So you would also need to pass-through the selection to the actual model when a selection event happens, and pass the selection state back from the big model (in case you care about selecting stuff in that big model).
I did take a quick look into ListSubModel for that use case, but I don't think it would help much, since your main issue is that the Listbox creates one placeholder for each non-loaded object, and ListSubModel will still cause it to create the same amounts of placeholders.
Hi MDuchemin,
I saw that post https://www.zkoss.org/wiki/SmallTalks/2009/July/Handlinghugedatausing_ZK. It's the same idea, right? But as you say, it seems to "work against" the idea of list box. I might look into that nevertheless. For now however, I need to push forward a bit urgently unfortunately.
Thanks, Henning
Hey Henning,
It's basically the same concept yes. Have the Listbox or Grid only aware of the displayed page, have a custom paging (not the default Listbox or Grid), with the correct number of pages, then do the bridge between the actual data and the display component in Composer.
I wouldn't say it works against Listbox, to be honest, but it is certainly not the out-of-the-box behavior.
Everything is a trade off, in this case between specialization and "most common use case". These components are designed to provide a 2 lines of code tabular data display from a java collection (plus template / renderer).
ListModel model = newListModelList(myList);
grid.setModel(model);
That doesn't mean that your use case is wrong or cannot be achieved, just that it's more specialized than the default state of these components and thus require custom code.
BigListbox is the component meant to support that type of "only pull rendered data in a big set" out-of-the-box service, so that's why I pointed to that one as the "native ZK answer" to that use case.
There are uses cases where the custom listbox/grid + paging + handling data in composer structure is more appropriate. The most important thing to me is intentionality, and deciding on the solution that makes the most sense for each given use case based on the specific requirement, the context around it (the existing state of the application, the type of data, the type of operations expected on that data, etc.).
Asked: 2023-09-18 04:45:12 +0800
Seen: 13 times
Last updated: Sep 25