Problems with Table being very slow

[MaSta]

Good day! In my project I'm programming a script processor. It reads Excel files containing SCPI or ModBus messages, processes them row by row and sends the command(s) to a device. This is all working fine meanwhile.

What is left is a bad performance of table display which is a log. My code writes into that table every row, into 6 columns. I also do some color marking. Tables cannot be written sequentially. You have to provide an array that grows by every row and it written as is. It seems to be expected that the bigger the array, the slower the table will get.

I've put timer VIs into my code to measure the single code parts. The entire read/write from and to the device with all the processing itself takes 0-4 ms, sometimes a few more. The table refresh per row takes about whopping 45 ms, so that in average one row ends up being about 50 ms.

In another thread someone suggested to use a property to defer panel update, which I use now. Without that, one row takes even much longer, about 100 ms. This is only coming from the table being updated!

Is this normal, is this really expected? Did you have this yourself and could solve or circumvent and if so, how? Are there faster alternatives?

Thanks in advance.

[pincpanter]

Yes, table refresh is often really slow, but usually with Defer panel updates the performance becomes acceptable.

Make sure to set this property at the very beginning and at the very end of you updating code.

[LLindenbauer]

A while back I did some benchmarking on the table. In addition to deferring panel updates, you might want to try to disable the automatic column height. Also, the table works way faster when all columns have the same height (globally set).

My hypothesis is that the vertical table scroll bar wants to be exact within the overall pixel length of the filled table, which triggers updates in all cells on a change of data. I think I saw something that looked like O(n) or O(n log n) when even one row had a custom height, but O(1) if row height was set globally.

[MaSta]

In fact I do that, but for a different reason. Having automatic row height generally enabled doesn't make the table adjust the row height automatically whenever you fill it. I learned that you must trigger it by setting the property to off first and then to on again. So I combined that with the panel defer, but yes, it would additionally slow down table refresh. However, I need automatic row height. 

[LLindenbauer]

How full is the table when you start seeing the slowdown and how much of it is visible? Your data update rate (4 ms) sounds like you are generating large amounts of data. Maybe you can solve this issue by separating the data from the display - implementing your own MVC, so to speak. You would keep the table, but take from the data store only the part that is needed to fill the visible area of the table.

[MaSta]

The interval of the loop is set to, let's say 100 ms, but I also would like be able to set it to 50 ms or less. Not much writing/reading per row, not much data. 20 ms is at least 50 rows per seconds, I don't consider that as much data. So from the 100 ms per row it uses approx. the half, the rest is waiting time. But if approx. 45 ms of the 50 ms is just for fillin a table I think something's quite odd.

The table is supposed to fill row by row to monitor what's going on. Doing it pagewise is no solution. I mean, I can live with a 100 ms being a safe minimum, but I expect the table becoming slower and slower.

It starts with the first row already. The headline is permanently filled. The array I feed is, of course, two-dimensional, but at that point only one row and 6 columns. As soon as the panel defer is turned off again, the table is refreshed, using up about 45 ms. With defer still being on, the code only needs less than 5 ms (when there is some reading from the device).

"You would keep the table, but take from the data store only the part that is needed to fill the visible area of the table." - Good idea, but I also want to be able to scroll through the results when I pause or stop. I could, of course, use a scroll event, if that's available.

[LLindenbauer]

---

@MaSta wrote:

"You would keep the table, but take from the data store only the part that is needed to fill the visible area of the table." - Good idea, but I also want to be able to scroll through the results when I pause or stop. I could, of course, use a scroll event, if that's available.

---

That would be the task of the Controller part. You'd make your own scroll bar out of, for example, a slider control between 0 and 1. On a value change, use that to determine what part of the data to display, then only fill that part of the data into the table. This gives you a O(1) scaling with your 50 ms update rate - not great, but not terrible.

[MaSta]

What I wonder: what do people do who have to create a speedy software that can really scroll very quickly, listing data sort of in real time?

The table class used in LV is similar to the standard table class in C#. We have such a list box in our C# based software and it also cannot scroll/refresh as quick as you would imagine.  

[stevem181]

> What I wonder: what do people do who have to create a speedy software that can really scroll very quickly, listing data sort of in real time?

Many of my systems require large tables. To vastly increase their performance, I use “Brute Force” methods.

The basic strategy is: create a grid of indicators, make them look like a table, then update them using the “Set Control Value by Index” function. “Brute Force” can give you speed but this speed comes at a cost. Specifically, more dev work and reduced functionality. Depending on the application, the functionality may be sufficient.

Attached is a demo project (LabVIEW 2018) that utilizes this technique.

Operation:

You can ADD single rows or N number (settable) of rows to the table
When you ADD N rows, the data rate is 2ms (can be changed in a subvi)
Hover over a “cell” to show data in a larger string indicator
The CLEAR button clears the table
Scroll buttons can be used to navigate the “table” (Uses Radio Button control)

Changing “cell” colors is not in this version. This can be done by adding another grid of Picture Rings behind the string indicators (and the code to support them).

[Hooovahh]

---

@stevem181 wrote:

Many of my systems require large tables. To vastly increase their performance, I use “Brute Force” methods.

---

I kudo'd your post because it is awesome to share.  But I personally wouldn't implement it this way.  My UIs generally scale, and having each control have to resize, would be a pain.  That and I like the smooth scrolling that scrollbars give, among other table and listbox features, like cell color and font, glyphs, and column resizing.

I'd rather go with an implementation similar to this over on LAVA.  It is called a Virtual MCLB.  It basically just loads the data that is being shown into the table, and nothing else.  This allows for doing things like setting cell color and font without having to do it for the whole table all at once.

https://lavag.org/topic/15289-virtual-multicolumn-listbox/

There is room for improvements of course, but this could be turned into a QControl pretty easily so that it is easier to use.