05-22-2017 10:34 AM
Hello,
I'm using a producer-consumer loop to log and perform some calculations on accelerometer data. With my current code I'm noticing the consumer loop takes an excessive amount of time to complete, roughly one iteration occurs per second. My samples to read per channel is set to 200 with a sampling frequency of 50000 Hz. I'm performing impact testing hence the high sampling frequency.
I've attached my code and would like some assistance with reducing the execution time of my consumer loop. Are there particular processes in my loop causing the lag i.e the loops within my consumer loop, integration blocks, etc?? If so, what might I do to reduce the execution time?
Also am I able to time different stages within the loop to determine what may be causing the lag?
I understand the code is cumbersome as i'm new to labview, at this stage the run time is the greatest cause of my concern. Any help would be greatly appreciated!
Thanks
05-22-2017 11:23 AM
As the very first step, I'd recommend to cleanup the code - separate code into subVIs, nicely organize block diagram. Then you can start with time benchmarking.
The easiest time benchmarking - is to have FGV with "Set Time", "Calculate Elapsed Time" states, where you'll set initial time, and after that calculate, how much time has elapsed. Then you put this FGV into the code, and check timing.
Also, you have loop iterations - try to parallelize calculation, if possible. Check this link, it should be helpfull.
Then, instead of "Insert Into Array", use "Replace Array Subset". Now you init array for the second inner loop with 0 elements, and then do Insert. Instead, you need to init array with the expected number of elements, and then replace those elements inside of the loop. It'll be also faster.
But it's really difficult to check the code, b/c it's large and messy...
Let me wish you luck with it 😃
Sincerely, kosist90
05-22-2017 11:59 AM
There are at least two things that could take a long time in the consumer loop.
Graphing:
Plotting more data less often will reduce the work the plots require.
File writing:
Writing to text file is slow... write to a raw binary file while testing. After the test you can convert to text format.
You should also open the file before the test, write during the test and then close it once after you are done. Using the write to spread-sheet file as you have it is opening, writing, closing... every iteration.
Use two consumer loops:
That way you can split the file and graphing work across multiple cores.
Another approach could use DAQmx Write to TDMS to do the file writing automatically.
Ben
05-22-2017 12:55 PM
@Ben wrote:
There are at least two things that could take a long time in the consumer loop.
Graphing:
Plotting more data less often will reduce the work the plots require.
File writing:
Writing to text file is slow... write to a raw binary file while testing. After the test you can convert to text format.
You should also open the file before the test, write during the test and then close it once after you are done. Using the write to spread-sheet file as you have it is opening, writing, closing... every iteration.
Use two consumer loops:
That way you can split the file and graphing work across multiple cores.
Another approach could use DAQmx Write to TDMS to do the file writing automatically.
Ben
Hi, Ben:
Just to clarify the bold text above, do you actually mean that writing numbers as a string makes it extremely inefficient - e.g., writing the decimal unsigned integer value "255" as a string takes 3 bytes to do while writing it as a binary uses only one byte - and therefore takes a lot longer to write?
I might also add that if you are writing floating point values to string, you also lose precision. (Savings in space will vary, as a double precision floating point already uses 8 bytes.)
05-22-2017 01:25 PM
@billko wrote:
@Ben wrote:
There are at least two things that could take a long time in the consumer loop.
Graphing:
Plotting more data less often will reduce the work the plots require.
File writing:
Writing to text file is slow... write to a raw binary file while testing. After the test you can convert to text format.
You should also open the file before the test, write during the test and then close it once after you are done. Using the write to spread-sheet file as you have it is opening, writing, closing... every iteration.
Use two consumer loops:
That way you can split the file and graphing work across multiple cores.
Another approach could use DAQmx Write to TDMS to do the file writing automatically.
Ben
Hi, Ben:
Just to clarify the bold text above, do you actually mean that writing numbers as a string makes it extremely inefficient - e.g., writing the decimal unsigned integer value "255" as a string takes 3 bytes to do while writing it as a binary uses only one byte - and therefore takes a lot longer to write?
I might also add that if you are writing floating point values to string, you also lose precision. (Savings in space will vary, as a double precision floating point already uses 8 bytes.)
Yes.
One of the spec for a disk drive speaks how fast you can write to the drive and is specified as "bytes/second". reduce the number of bytes and you reduce the amount time required to record the values. Also keep in mind the delimiter adds a byte between each value and the decimal point requires a bytes.
Your "lose precision is also valid.
Ben
05-22-2017 01:34 PM - edited 05-22-2017 01:37 PM
@Ben wrote:
@billko wrote:
@Ben wrote:
There are at least two things that could take a long time in the consumer loop.
Graphing:
Plotting more data less often will reduce the work the plots require.
File writing:
Writing to text file is slow... write to a raw binary file while testing. After the test you can convert to text format.
You should also open the file before the test, write during the test and then close it once after you are done. Using the write to spread-sheet file as you have it is opening, writing, closing... every iteration.
Use two consumer loops:
That way you can split the file and graphing work across multiple cores.
Another approach could use DAQmx Write to TDMS to do the file writing automatically.
Ben
Hi, Ben:
Just to clarify the bold text above, do you actually mean that writing numbers as a string makes it extremely inefficient - e.g., writing the decimal unsigned integer value "255" as a string takes 3 bytes to do while writing it as a binary uses only one byte - and therefore takes a lot longer to write?
I might also add that if you are writing floating point values to string, you also lose precision. (Savings in space will vary, as a double precision floating point already uses 8 bytes.)
Yes.
One of the spec for a disk drive speaks how fast you can write to the drive and is specified as "bytes/second". reduce the number of bytes and you reduce the amount time required to record the values. Also keep in mind the delimiter adds a byte between each value and the decimal point requires a bytes.
Your "lose precision is also valid.
Ben
Thank you, Ben. I wanted to make sure I understood what you meant. When a statement such as yours is made by an LV guru as yourself, I have to make sure I'm not missing a revelation. 🙂
05-22-2017 01:35 PM
@billko wrote:
I might also add that if you are writing floating point values to string, you also lose precision. (Savings in space will vary, as a double precision floating point already uses 8 bytes.)
If I really need to save space, I'll save them as single precision binary. Four bytes is half the space as a double floating point. For any reasonable numbers in the engineering world, there is still enough precision giving you about 6 or 7 significant figures.