LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Why are shift registers are so slow

I am dealing with large amounts of data, arrays in excess of 65000x15.
I need to build the array in a loop. When I use the shift register
approach it takes forever. The fastest way I have found so far is to
write the rows out to a spreadsheet file and read them back in. This
is fairly quick, but seems rediculous to have to do. Memory should be
much faster then a HD
0 Kudos
Message 1 of 17
(5,801 Views)
The slowness is due to memory relocation everytime the array size is
changed. Try to initilize a fixed sized array (65000x15)and wire it
to the shift register, then use "Replace Array Element" to replace a
single element, a row or a column depends on what you need.

Joe
Message 2 of 17
(5,779 Views)
Thank you. I had tried this originally, but must have not had it quite right. Got it working now and it is super fast!
0 Kudos
Message 3 of 17
(5,779 Views)
Hi,

I have a very similar problem. I have 3 arrays connected to shift registers, and plotted in 2 XY graphs (2 data arrays and 1 time array). Initially, I was using a Build Array block to generate the arrays, but noticed that as time (hours) went by and the arrays got bigger, each loop iteration was taking longer and longer. I need the loop to run at a particular rate, but it was slowing down over time.

I followed the advice of this post (determine the size of the array beforehand, initialize it, and then use Replace Array Element instead), but I had the same problem. My 1-D arrays can grow up to 200,000 rows, and I have 3 of them. Perhaps graphing all this data in an XY graph is the problem? Is there another way around it?

Thanks,
Sima
0 Kudos
Message 4 of 17
(5,752 Views)
sima wrote:

> Hi,<br><br>I have a very similar problem. I have 3 arrays connected
> to shift registers, and plotted in 2 XY graphs (2 data arrays and 1
> time array). Initially, I was using a Build Array block to generate
> the arrays, but noticed that as time (hours) went by and the arrays
> got bigger, each loop iteration was taking longer and longer. I need
> the loop to run at a particular rate, but it was slowing down over
> time.
>
> I followed the advice of this post (determine the size of the array
> beforehand, initialize it, and then use Replace Array Element instead),
> but I had the same problem. My 1-D arrays can grow up to 200,000 rows,
> and I have 3 of them. Perhaps graphing all this data in an XY graph is
> the problem? Is there another way around it?

Shift register add very little overhead to your VI, in fact except in a
few rare cases in earlier versions of LabVIEW it is almost always the
fastest way if you need handling large data repetitevely.
As you already suspected the plotting of such large arrays is the real
problem. What can you do?

Do you really need to plot all that data? Maybe a shorter history would
be enough? If that is not an option then put another loop inside that
loop (with also a shift register) which acquires several data elements
(10 or more depending on your input speed) repetitively before leaving
that loop and plotting the new resulting array on the front panel. This
will reduce overhead greatly.

Rolf Kalbermatter
Rolf Kalbermatter  My Blog
DEMO, Electronic and Mechanical Support department, room 36.LB00.390
0 Kudos
Message 5 of 17
(5,723 Views)

There is an excellent presentation of the possible ways of dealing with large data sets, including display issues, here

Chilly Charly    (aka CC)
0 Kudos
Message 6 of 17
(5,718 Views)
Hello again,
The culprit in my situation was indeed plotting all the information in the XY graph. I noticed how smoothly and quickly the program ran without those displays, even for hours. Since I still need to graph my data, I chose to plot a shorter history using a moving window.
Thanks for all the suggestions,
Sima
0 Kudos
Message 7 of 17
(5,639 Views)

Hi, I have the same issue. I need to acquire a 108000x8 data matrix. I initialize an array that size, enter a while loop with a shift register and replace one row at each iteration with the data that I am gathering online. 

I need an update rate of 1000 Hz so loop with a wait 1 ms)

Everything works fine with smaller array (i.e. 20000), but loop slows down when I request a larger file.

Any idea?

Thanks in advance

Antonella

0 Kudos
Message 8 of 17
(3,956 Views)

HI Antonella,

 

please attach your VI when you have problems with the VI…

Best regards,
GerdW


using LV2016/2019/2021 on Win10/11+cRIO, TestStand2016/2019
0 Kudos
Message 9 of 17
(3,948 Views)

@antonella86 wrote:

Hi, I have the same issue. I need to acquire a 108000x8 data matrix. I initialize an array that size, enter a while loop with a shift register and replace one row at each iteration with the data that I am gathering online. 

I need an update rate of 1000 Hz so loop with a wait 1 ms)

Everything works fine with smaller array (i.e. 20000), but loop slows down when I request a larger file.

Any idea?


Yes, we need to see the VI. You talk about "online" and "request file", so what does each loop iteration actually do? Is the data coming from a network location? How exactly is it read?

 

Also be careful with terminology. The term "matrix" is a special datatype. I assume you have a plain 2D array.

0 Kudos
Message 10 of 17
(3,924 Views)