sort cluster efficiency

[Gabi1]

Hei guys,

i have a big cluster, composed of several elements of different format.

i sort this cluster by the first element (second transformed into first...).

the problem is that my array is about a few tens Mb, which takes about 500ms to compute and prepare before sorting. the sorting takes 3 seconds!!

is there a way to make it faster? should i take care of sorting only the first element without "tie-breaker" elements after?

[GerdW]

Hi Gabi,

what do you expect when sorting 'a few tens Mb'???

Next point: do you have a cluster of arrays or an array of cluster or what? Can you provide a (simplified) example?

[Gabi1]

I tried Altenbach solution from a thread about a year ago,

here a pic of how it looks like.

but the sorting takes actually exactly the same time.

btw i have checked that it is the sorting itself and no other element that takes most of the time.

Message Edited by Gabi1 on 04-30-2007 07:02 AM

[shoneill]

Are these 4 elements all that is contained within the cluster? DBL, Bool, I32, I32?

If so, then to get to tens of MBs, you must have millions of elements.

Is this the case?  If so, then your sorting is simply a huge task.  I thought originally you were sorting complicated structures, but the sort itself is simple (DBL and I32).  Are you sure the re-ordering of the array isn't taking the time?

Shane.

Message Edited by shoneill on 04-30-2007 02:17 PM

[GerdW]

Hi Gabi,

some comments:
1) Please read the development styleguide! So many hidden/non-straight wires...
2) Altenbach most often gives you an optimal solution

It's just a question of how big your cluster-array is - the bigger the array the longer it takes to sort it!

[Gabi1]

Hello Shane,

yes i have huge nb of elements, it gets very fast to the 100MB. i prepare before this subvi this array of cluster, where i cut ramps of about 10-20 sec into 10us elements...

i have tried to make it much simpler, but i need everypiece of info in those clusters. the time taking element is actually the sorting itself. the way i do it on this diagram was supposed to be a little bit faster than simply "sort array", but it turned out to be the same.

at this point the only solution i see is not to sort at all, and be carefull to prepare my array sorted. this is quite a task ,as i interleave elements all the time with different timing.

will update on devellopements

Gabriel

[Gabi1]

Hei GerdW,

i know this is dirty programing. i am still in developement.

yes my cluster array is enormous. thinking of some sol right now

[smercurio_fc]

3 seconds is not that unreasonable, considering the amount of data. You said you have about 20 seconds of data, at a 10 usec interval. That works out to about to 2 million elements. With each element being a cluster of a DBL, BOOL, I32, and I32, that works out to about 32MB of data (this doesn't take into account the overhead of the cluster structure). On my P4, 3GHz, sorting an array of 2 million clusters of that structure took 2.7 seconds, using the standard "Sort 1D Array" function. I do not find that unreasonable.

[smercurio_fc]

Note: this seems to be a continuation of this thread. It is always best to keep the discussion in the original thread, as it makes it easier for others to follow what's going on since there's some history.

[xseadog]

Hi

I'll be interest:ed to heard what actual hardware you are using

If you are slicing at 10usec what are you sampling at?(1MHz)  How many channels?


maybe I misunderstand the  expalnation

xseadog