New Array Collector Primitive

[smmarlow]

Getting the best performance while accumulating array values for display can take up a bit of space.  I propose a new primitive, either an array primitive or a node structure (like the feedback node) that accumulates an array of a specific size.  It should be smart and size itself automatically if dropped into a loop, the way indexing output tunnels do.  It should also keep track of how many times it has been called and return only the points collected, or the last N points if the size input is wired.  It is functionally equivalent to the "Collector" express VI, but has the ability to automatically size itself, accept a wired size input, and is truly polymorphic.  It would also put the best of memory management to work by NI's clever engineers to maximize performance. Here is my best shot at its implementation as an array primitive:

[smmarlow]

PS.  I should say that if you wanted to get fancy, you could add an optional conditional input terminal to control whether or not the element is added to the array, mimicking the function of a conditional autoindexing node.  While I have implemented it here as an array primitive, it is probably better to choose a stucture node, since it will "tie" itself to a loop, and has its own memory space, like a feedback node.  The reason I show a primitive is because what you see is a VI that I wrote that has all the features I've described except auto sizing and polymorphism. 

[altenbach]

The idea looks useful, but it probably would require some fine tuning and more features.

• There needs to be a "reset" boolean input which would delete all old content.
• Should there be an option for fixed buffer size with an optional padding value (e.g. NaN, zero, etc)
• it should accept scalars or arrays (similar to a chart or "built array in concatenate mode").
• The autosizing looks somewhat troublesome.
• It should not need a surrounding loop (e.g. for use in an action engine style subVI)
• There could be a lot of memory thrashing if a variable is wired to the size. Many beginners might wire it to the the iteration terminal, for example.
• For 2D arrays we need to know which dimension to grow.

My kudos are conditional until all discussions are in... will watch this discussion for a while.... 😉

[dthor]

Sounds a lot like stuff that can be accomplished with Queues. Can you tell me how it's not similar?

[smmarlow]

All useful comments to be sure.  

@altenbach.  

*  Yes it needs much fine tuning

* Yes there could be memory thrashing in the hands of a novice.   But such is the case also for the initialize array primitive. 

* You are correct there probably needs to be a reset boolean.  

* There would be a fixed buffer size.  The function would return a subset of the buffer if it is not yet full. There could be a padding     value, but it is not necessary with the subset return.  The buffer could be initialzed with the default type.  Presumably, this function would borrow from the indexing output tunnel.

* It does not need a loop, sorry if I was not clear or confused on that issue.  If it is not in a loop, the # elements would have to be wired.

@dthor

It's true you can accomplish the same thing with queues, but not with a single node.  Queues also do not preallocate memory space and required several functions to create, destroy, enqueue, dequeue, etc.  This primitive is a single node.

[dthor]

Ah, I didn't remember that queues don't preallocate memory. Also, yes, it takes 3 nodes (5 and an extra loop with preallocation) to do it with queues, vs 1 with your suggestion.

[smmarlow]

@altenbach  I just had a thought that if the primitive only resizes the buffer when the reset is true (or on first call), memory thrashing should not be a problem.  The reset boolean input could be made an optional input.  If left unwired, the function would size the buffer on first call only, even if a variable such as the iteration terminal were wired to the # elements.  The reset booelan would be false by default, so the only way to get memory thrashing would be to wire a value that is always true to the reset input.

[GregSands]

Another solution could be to move the auto-indexing terminal inside the loop (but still showing its link to the loop):

This doesn't have the FIFO ability, but to my mind that's a separate issue.

[donkdonk]

What comes first to my mind: looks like one of the point-by-point vi's. It's just that they only accept scalars.

Could a Data Queue PtByPt.vi do the trick when it would accept clusters?

(and why not other types)

(Or is memory allocation a problem here? Or something else I don't see?)

[Darin.K]

The Feedback Node with a delay could be tweaked to get some of the functionality.

The delay must be set during edit mode and is fixed, this is similar to the Collector Express VI.  I am not sure how vital runtime control is, along as there is an option to set it to 'Infinity'.  If necessary and it does not create more problems than it solves, this could be added.  Memory management may not be absolutely optimal, but it should be similar to the shift register which is pretty darn good.

There is already an enable terminal which would serve as conditional concatenation.

There is already an initializer terminal.

What would need to be added is an output which is the current buffer.  This could be moved to outer loops much like initializer terminal.  This would help with my biggest concern which is the potential to autoindex this output when passing it outside a For Loop.

[smmarlow]

Thanks for commenting, everybody.  It appears this idea has stalled, but all of the suggestions are well taken.  I have created a reentrant VI that has the features described in my proposed primitive. It won't be able to auto-size itself in a loop, so will always have to have a value wired to the # elements input.  I have modified several versions and will assemble them into a polymorphic VI.  Any of these might be easily modified for use with custom data types.