Memory usage of array of clusters

[siadajpan]

Hi

I'd like to build an array of cluster. Each cluster containing 2 1-d arrays of points (X and Y coordinates)

I know, that amount of clusters in the array will be 1024 but I don't know how many points will be in each cluster (can be 500-4000).

Is initializing 1024 empty cluster array gonna help with memory usage, when my clusters are being filled with data? Or when I'll start filling each cluster, the whole array will have to be copied by LabVIEW, when it will grow bigger and bigger?

In other words, is LabVIEW saving array of clusters as an array of pointers, each cluster in different location, or as an array of data and keeps everything together?

What about array of DVRs each being a pointer to a cluster? Has anyone used this before?

[Mike_King]

I think if your clusters have variable sizes, as you fill them, labview's memory manager will have to reallocate space and move data around, since they are not fixed size.

you could create a fixed 2d array, large enough for all array sizes, then crop the columns as needed by knowing the lengths, but still requires some memory adjustments to split the arrays.  

You could also put arrays into a single array, just indexing out the individual arrays by keeping indexes and length.  DVR is the same if you are adjusting the memory space required, but may help reduce copies for accessing it, that's all. Hope that helps, I'm sure there are many other ideas others have as well.

[Yamaeda]

I'd guess the array in a struct is actually an array pointer, so the cluster is a fixed size, but i might be wrong there. If it isn't, it's easy enough to replace it with a DVR and handle the array elsewhere.

If you know it'll be 1024 elements you can preallocate it, but generally i wouldn't bother. 🙂

/Y

[Ben · Accepted solution]

I have used queues of clusters where I specified the max size of the queue when created THEN I fill the queue with max-sized elements and THEN purge the queue without destroying same. After that I use the queue to store the clusters. Using queues have the advantage that the elements do not have to have a single block of memory and can be scattered in memory space. 

That method pre-allocates the memory required and minimizes the jitter at start-up and while running.

Ben

[karolek]

Wow, that's pretty interesting concept. I'll try that for sure. Thanks!

[siadajpan]

hi

I don't quite understand how do you put the data inside the queue that has already data inside. Are you using lossy enqueue?

Like that:

I guess this is good for storing data when high-speed streaming, but is it possible to then select n-th element and display it? Or after data are collected you store it somehow else to operate on them more easily?

[Ben]

---

@siadajpan wrote:

hi

I don't quite understand how do you put the data inside the queue that has already data inside. Are you using lossy enqueue?

Like that:

I guess this is good for storing data when high-speed streaming, but is it possible to then select n-th element and display it? Or after data are collected you store it somehow else to operate on them more easily?

---

Add a flush queue after the initial filling.

Not intended for random access of the entries in the queue but if forced maybe use a "Preview queue" but I would suspect that requiring new memory to access the required array of queue entries.

Ben 

[Yamaeda]

If you need random access i s'pose it's better to create 1000 named 1-element queues instead, or DVRs.

/Y

[crossrulz]

---

@Ben wrote:

Add a flush queue after the initial filling.

---

No, that will actually release all of the memory you just tried to pre-allocate.  The proper way to empty a queue is to use a loop to dequeue all of the elements (While Loop with a Dequeue that has a 0 timeout, stop the loop when you get a timeout OR another FOR loop).

[Ben]

---

@crossrulz wrote:

---

@Ben wrote:

Add a flush queue after the initial filling.

---

No, that will actually release all of the memory you just tried to pre-allocate.  The proper way to empty a queue is to use a loop to dequeue all of the elements (While Loop with a Dequeue that has a 0 timeout, stop the loop when you get a timeout OR another FOR loop).

---

Thank for correcting me!

That is what I believe I did  ... (if memory which is old serves me).

Ben