What is the best way to implement a rolling array?

[simonides]

I need to implement an array that takes a new value and stores it at the front and stays under a maximum size. I have implemented it in different ways but I am not sure which one is the best. Because this part is slowing the application down dramatically I need it to be the best perfomant way possible. I terms of reallocations I think I should go with the first one but is it the fastest?

I would also like to put that into a subVI and use it with different data types. How would I do that? use a variant?

[GerdW]

Use rotate with replacearraysubset...

[simonides]

Thank you for your answer. And how would you put that into a sub VI without createing different subVIs for every type I want to use it with?

[Bob_Schor]

Why are you doing this?  One reason I can think to do this would be to implement some form of moving average.  A trick I've used that lends itself to speed and simplicity (particularly when the Messy Details are hidden inside a sub-VI) is to use a fixed-length Queue, using Lossy Enqueue to add the latest element (and have the oldest "fall off the end") and use Get Status to return all of the Queue elements (without removing them from the Queue).

As for having one routine handle multiple types of data, you already know that an Array has to have all elements of the same Type.  If you make this a Variant, then you will (I suspect) drastically slow things down for little tangible benefit (as you'll still need to know the element type in order to re-create the original data type for any meaningful use).  If, on the other hand, you develop a sub-VI that takes, say, I32s in and returns an Array of I32 (or the average, or whatever it is you want to do), you can create another almost-identical sub-VI for Dbls, for Sgls, for U16, etc.  Now you can "wrap" all of these into a Polymorphic VI -- when you put this down on your Block Diagram, it doesn't "decide its type" until you wire the appropriate Input, then Presto, you instatiate the appropriate instance.

To learn about this, go to Help and type in Polymorphic in the Index tab.

Bob Schor

[crossrulz]

Have a good look at the Signal Processing->PtByPt palette.  There are a lot of goodies in there.  The one in there I find myself using a lot is the Mean PtByPt.vi.  There is also the Data Queue PtByPt.vi in there that you might find interesting.

[simonides]

Thanks for your answers.
I want to pass this data to the UI (charts that display this data) and need to make sure that it doesn't use up all the memory so i have to limit the maximum size of the Arrays. Are there any plans by NI to implement some Kind of "Generics" to avoid writing VIs that the compiler could easily deduce? How does NI implement things like queues? They can also handle arbitrary types. I am thinking of clusters with different datasets for example.

With these moving averages tricks. Is the order maintained too ? Are your solutions just more convenient or also faster? I mean it's not that bad to place these two blocks if it's faster or better.

[altenbach]

---

@simonides wrote:
I want to pass this data to the UI (charts that display this data) and need to make sure that it doesn't use up all the memory so i have to limit the maximum size of the Arrays. 

---

Charts already have a built-in buffer where you can define the history size. No code needed.

Can you be more specific on what you are actually trying to do? The information seems very vague so far. You can reverse the x-axis if needed.

How often is data added to the buffer and how often do you need to update the indicators?

Are you having speed or memory issues? How many history elements do you need?

What is the current speed and what speed do you need? How do you measure speed?

What else does the application do?

Can you explain your question about subVI use?

[Yamaeda]

Gerds solution is probably the best performance wise (but you need to preallocate or handle the array growing).

Another solution is Build array (add to front) and Get array subset. If the size requested is bigger than the current size you'll simply get what you have, else it'll be trimmed to size.

/Y

[tst]

Any solution you use would have some performance/memory implications, because there's a limit to the amount of the control you have. At the very least, you would have one copy for the buffer and a second one for the chart.

The first thing to consider is how much you actually need to optimize. Your example shows 12 elements, but I'm assuming your needs are for something else. Even so, up to a point there's no need to worry about optimization.

If you just want a chart on the UI, just set the chart to have the right length and pass the values into it using a queue. It will manage its own buffer. Otherwise, the lossy enqueue method tends to be the simplest. The rotation method can be most efficient memory-wise, but it does come with a performance penalty on moving all the elements. You can keep track of where you're updating to avoid that, but then you will get a copy when you read the array in the correct order.

---

@simonides wrote:
Are there any plans by NI to implement some Kind of "Generics" to avoid writing VIs that the compiler could easily deduce?

---

Yes, several over the years. The feature actually exists in LV in several forms (XNodes, generics, VI macros, to name some of them), but they each have their issues and so they were never finalized and made official. They can be used, but might not work under all circumstances, might crash or disappear in future versions, etc.

The type adapting primitives that NI has are mostly implemented in C.