calling a library function node much faster than labview code?

[dakeddie]

Hi,  I wrote a labview routine to perform a multiple tau autocorrelation on a large array of integers.  A multi tau autocorrelation is a way to reduce the computation time of the correlation but at the expense of resolution.  You can taylor the multitau correlation to give you good resolution where you need it.  For instance, I require good resolution near the middle (the peak) of the correlation, so I do a linear autocorrelation for the first 64 channels from the peak, then I skip every second channel for the next 32, then skip every 4th channel for 32 more, then skip every 8th for 32 channels... etc.

Originally, I wrote my own multitau calculation, but it took several hours to perform for just 1024 channels of the correlation of around 2million points of data.  I need to actually do the the correlation on probably 2 billion or more points of data, which would take days.  So then I tried using labview's AutoCorrelation.vi which calls a library function.  It could do a linear autocorrelation with 4 million points in less than a minute.  I figured that writing my code in C and calling it using a call library function node would be faster, but that much faster?

Finally, I wrote some code that extracts the correlation data points that I would've got from my multitau code from the linear correlation function that I get from the AutoCorrelation.vi.  Clearly this is not optimal, since I spend time calculating all those channels of the correlation function just to throw them away in the end, but I need to do this because the final step of my procedure is to fit the correlation function to a theoretical one.  With say 2million points, the fit would take too long.  The interesting thing here is that simply extracting the 1024 point from the linear autocorrelation function takes a significant amount of time.  Is labview really that slow?

So, my questions are...  if I rewrite my multitau autocorrelation function in C and call it using a call library function node, will it run that much faster?  Can I achieve the same efficiency if I use a formula node structure?  Why does it take so long just to extract 1024 points from an array?

I've tried hiding indicators and this speeds things up a little bit, but not very much.

I'll attach my code if you're interested in taking a look.  There is a switch on the front panel called 'MultiTau'... if in the off position, the code performs the linear autocorrelation with the AutoCorrelation.vi, if in the on position, it performs a multitau autocorrelation using the code I wrote.  Thanks for any help.

[Wiebe@CARYA]

Hi,


The C routine that AutoCorrelation.vi is using is probably a higly optimised routine. If you write a routine in LabVIEW, it should be less then 15% slower. But you'd have to know all ins and outs of LabVIEW. How data is handled, when memory is allocated, etc. Also note that the AutoCorrelation.vi has years of engineering behind it, and probably multiple programmers.


It might even be possible that the c code uses an algorithmic improvement, like the Fast Fourier Transform improves speed on the Fourier Transform. I think the autocorrelation can be done using FFT, but that isn't my thing, so I'm not sure.


For a fair comparation, posting the code in this forum was a good idea. I'm sure together we can get it to 115% or less of the C variant. (15/115 is just a guess, btw)


I'm still using LV7.1 for client compatibility, so I'll look at the code later.


Regards,


Wiebe.


"dakeddie" <x@no.email> wrote in message news:1147990209902-367537@exchange.ni.com...
Hi,&nbsp; I wrote a labview routine to perform a multiple tau autocorrelation on a large array of integers.&nbsp; A multi tau autocorrelation is a way to reduce the computation time of the correlation but at the expense of resolution.&nbsp; You can taylor the multitau correlation to give you good resolution where you need it.&nbsp; For instance, I require good resolution near the middle (the peak) of the correlation, so I do a linear autocorrelation for the first 64 channels from the peak, then I skip every second channel for the next 32, then skip every 4th channel for 32 more, then skip every 8th for 32 channels... etc. Originally, I wrote my own multitau calculation, but it took several hours to perform for just 1024 channels of the correlation of around 2million points of data.&nbsp; I need to actually do the the correlation on probably 2 billion or more points of data, which would take days.&nbsp; So then I tried using labview's AutoCorrelation.vi which calls a library function.&nbsp; It could do a linear autocorrelation with 4 million points in less than a minute.&nbsp; I figured that writing my code in C and calling it using a call library function node would be faster, but that much faster?Finally, I wrote some code that extracts the correlation data points that I would've got from my multitau code from the linear correlation function that I get from the AutoCorrelation.vi.&nbsp; Clearly this is not optimal, since I spend time calculating all those channels of the correlation function just to throw them away in the end, but I need to do this because the final step of my procedure is to fit the correlation function to a theoretical one.&nbsp; With say 2million points, the fit would take too long.&nbsp; The interesting thing here is that simply extracting the 1024 point from the linear autocorrelation function takes a significant amount of time.&nbsp; Is labview really that slow?So, my questions are...&nbsp; if I rewrite my multitau autocorrelation function in C and call it using a call library function node, will it run that much faster?&nbsp; Can I achieve the same efficiency if I use a formula node structure?&nbsp; Why does it take so long just to extract 1024 points from an array?I've tried hiding indicators and this speeds things up a little bit, but not very much.I'll attach my code if you're interested in taking a look.&nbsp; There is a switch on the front panel called 'MultiTau'... if in the off position, the code performs the linear autocorrelation with the AutoCorrelation.vi, if in the on position, it performs a multitau autocorrelation using the code I wrote.&nbsp; Thanks for any help.



MultiTauAutocorrelate.vi:
http://forums.ni.com/attachments/ni/170/185730/1/MultiTauAutocorrelate.vi

[GerdW]

Hello dakeddie,

after having a first look into your vi I saw a massive appearance of local variables! Ugglyyy 🙂

I just changed the multitau=false-case to use less locals. This way you don't need so many frames (or even sequences at all). Please clean up the other case using similar techniques!
This way your code will run faster. Use wires instead of locals...

[dakeddie]

GerdW,

Wow, that's a great tip.  I never knew that locals were to be avoided.

Thanks.

[Jarrod_S.]

Here's an oldie but a goodie. You can tell the age of this document by some of the screenshots, but the information is still valid and generally very useful. This is a good one to bookmark for later. Enjoy!

[nzed]

hi dakeddie,

i am also trying to correlate the signal of a single photon counting timetrace, therefore i am very interested in your implementation of the multi tau correlation procedure. unfortunately, i am working with LabView 7.1 and so i can not open your code example.

to visualize the time trace of 4 channels simultaniously with 1 ms binning time, i wrote a labview program using the old DAQ-components. feel free to use and improve it.

i would be very glad, if you could inform me about your progress in implementing the mutli tau correlation algorithm.

Greez,
nzed

[dakeddie]

Hi nzed.

Are you also doing dynamic light scattering?  If not, I'm still curious about your application.

I wasn't able to run your code because I'm missing a few .vi's, but this is ok.  I'm using the DAQmx for counting and binning and it's working very well.  I'd recommend it.  The code I used was just a minor modification of an example that was supplied with labview.  I'm able to bin at 100ns intervals, but unfortunately I run into memory problems when I try to autocorrelate this.

When I bin at a more reasonable 10us then I get quite nice results.

As for the code, I rewrote the rest of my multitau autocorrelation code according to the suggestions above and it did run extremely faster than before, but it's still much slower than calling the built-in AutoCorrelate.vi and then picking out 1024 data points from that.

[Atacama]

Hello dakeddie,

Any chance you can post your C code to compare ?

Thank you

---

@dakeddie wrote:
Hi,  I wrote a labview routine to perform a multiple tau autocorrelation on a large array of integers.  A multi tau autocorrelation is a way to reduce the computation time of the correlation but at the expense of resolution.  You can taylor the multitau correlation to give you good resolution where you need it.  For instance, I require good resolution near the middle (the peak) of the correlation, so I do a linear autocorrelation for the first 64 channels from the peak, then I skip every second channel for the next 32, then skip every 4th channel for 32 more, then skip every 8th for 32 channels... etc.

Originally, I wrote my own multitau calculation, but it took several hours to perform for just 1024 channels of the correlation of around 2million points of data.  I need to actually do the the correlation on probably 2 billion or more points of data, which would take days.  So then I tried using labview's AutoCorrelation.vi which calls a library function.  It could do a linear autocorrelation with 4 million points in less than a minute.  I figured that writing my code in C and calling it using a call library function node would be faster, but that much faster?

Finally, I wrote some code that extracts the correlation data points that I would've got from my multitau code from the linear correlation function that I get from the AutoCorrelation.vi.  Clearly this is not optimal, since I spend time calculating all those channels of the correlation function just to throw them away in the end, but I need to do this because the final step of my procedure is to fit the correlation function to a theoretical one.  With say 2million points, the fit would take too long.  The interesting thing here is that simply extracting the 1024 point from the linear autocorrelation function takes a significant amount of time.  Is labview really that slow?

So, my questions are...  if I rewrite my multitau autocorrelation function in C and call it using a call library function node, will it run that much faster?  Can I achieve the same efficiency if I use a formula node structure?  Why does it take so long just to extract 1024 points from an array?

I've tried hiding indicators and this speeds things up a little bit, but not very much.

I'll attach my code if you're interested in taking a look.  There is a switch on the front panel called 'MultiTau'... if in the off position, the code performs the linear autocorrelation with the AutoCorrelation.vi, if in the on position, it performs a multitau autocorrelation using the code I wrote.  Thanks for any help.

---

[Atacama]

All,

Any chance somebody can post the MultiTauAutocorrelate.vi for Labview version 7.1, so I can have a look?
Many thanks.

[EngrStudent]

The code is attached to the first post in the thread.