Improving memory usage of Consumer/Producer loop

[HuguesThales]

Good afternoon,

I’m currently setting up the generation and acquisition of SPI signals using a HSDIO NI-6541 module.

My main VI “Acquisition and Generation fetch” is divided in 3 sections:

• Generation of SPI signals
• Acquisition of SPI signals
• Decoding of SPI signals

The acquisition and decoding section are in a Producer/Consumer loop. Unfortunately, the decoding speed is lower than the acquisition speed and my memory eventually overflows.

I would like to increase my acquisition time as much as possible by increasing the decoding speed. I attempted to do that though the « express mode » of “SPI_decode.vi”. However, the increase in decoding speed hasn’t resulted in a sufficient increase in acquisition time … I’m starting to think the problem might be elsewhere!

Here is some data:

F_HTR  (Hz)	Express decode	T_loop  (s)	SPI frames per record	Records done before overflow	Time before overflow (s)	Number of decode_SPI before overflow
7200	False	1	2	248400	69	9
7200	True	1	2	298800	83	30
7200	False	2	2	230400	64	2
7200	True	2	2	288000	80	14
7200	False	2.5	2	234000	65	1
7200	True	2.5	2	306000	85	12

Let’s consider the last 2 rows:

• At F_HTR=7200Hz,T_loop=2.5s , F_sampling=20MHz and SPI frames per record=2, the acquisition section fetches F_HTR/(SPI frames per record)=3600Hz records of F_sampling×(1/F_HTR +6.5×10^(-6))≈2907 samples. Each sample being a 8bit integer. This result in the acquisition of 3600×2907≈10.4652Mbytes
• Each of those records is decoded into two 16 bits integers and two 32 bits integers. Hence 98bits=12bytes. Therefore, each execution of “decode_SPI” should result in an additional T_loop seconds of acquisition.
• Activating “express decode” results in an increase of the number of decode_SPI before overflow by 11 which should result in an increase of acquisition time of 27.5s. The observed increase in acquisition time is 20s, 27% less than expected.
• The same observation can be made of the other rows.

Now, I have 2 questions:

• Would there be any way of improving the memory usage of my producer/consumer loop?
  • Please note that I cannot decrease the sampling frequency my SPI clock is at 10MHz
  • I cannot decrease the number of sample per record. If I did I would have to increase the number of records acquired per second which would lead to my fetch rate to be lower than my acquisition rate, hence my HSDIO acquisition memory to overflow. This is because “the current implementation of the HSDIO driver uses a separate DMA transfer for each record that gets transferred. For large number of samples per record and a relatively small number of records, this is not an issue and the performance is much better. There is some overhead in making a DMA transfer that can slow down your overall transfer speed, especially with a large number of small records.” (see: https://forums.ni.com/t5/PXI/Why-is-my-fetch-speed-slow-on-my-6542-DIO/td-p/685080)
• Any idea as to how to optimize “decode_spi.vi”? I tried to use Boolean operations whenever possible but I might be missing something. I could provide a more detailed explanation of how I’m decoding my data if required.

Thanks a lot in advance for your help,

Hugues

[wiebe@CARYA]

Could you zip the code or post decode_spi.vi separately?

[HuguesThales]

Hi wiebe,

Please find attached "decode_SPI.vi".

Please note that I have to decode my SPI signal because it is sampled by the On Board Clock (20MHz) instead of the SPI_CLK (10MHz). I'm currently working on sampling my MISO signal with SPI_CLK with NI technical support. (It’s not trivial because SPI_CLK is not free running and I don’t know how I could generate a free running 10MHz CLK while generating trigged SPI signals)

SPI_CLK is generated by the HSDIO module and is therefore synchronized with the On Board Clock. One solution to decrease decode SPI execution time would be to not look for SPI_CLK's rising edges but to just take every 2 sample as a rising edge.

I was hoping you could point out bad coding practice, such as dynamically filling arrays. I've started LabVIEW a month ago and am still struggling with the basics.

Thanks a lot for your help,

Hugues

[wiebe@CARYA]

There's probably some speed to gain here. Some style improvements as well, but nothing too shocking.

For starters, use auto indexing. There's no point in using shift registers and a build array. LabVIEW will do the same, and almost always more efficiently. It will clear up the code as well.

The two subVIs are missing. There are significant. As there result is always used to get all the indices, the VIs can probably be modified to return the indices. That will at worse be a cleanup\simplification, but might increase performance just a little.

Can you post the other two VIs? And also some realistic inputs, preferably with desired outputs? That will help us a lot.

[HuguesThales]

Here are the 2 other VI.

I am not too sure of how auto indexing works,  I will do some research and get back to you if I don't understand.

Thanks 🙂

Hugues

[wiebe@CARYA · Accepted solution]

I think this does the same thing. It's easy to make a mistake though without being able to run the VI.

Spot the differences .

It's starting to fit the screen. I don't expect major speed increments, but it shouldn't run slower.

I'm sure there are some optimizations possible, but usually algorithmic improvements beat semantic improvements by an order of magnitude (~10X).

(EDIT: That crossed your post of the subVIs)

[HuguesThales]

Well, it's a lot faster! I've been acquiring data for the past 30min without overflow, amazing 🙂

Thanks a lot!

[wiebe@CARYA]

Great! Probably just the autoindexing that did the trick.