I'm trying to optimize a software that collect, organize and display data obtained from a PLC.
In the beginning the software runs very well, but after 10 min the CPU Usage goes to 20-30% and the User Interface becomes very slow (almost not responsive)
PC config = i7-9700 3.00GHz with 8GB DDR4
I'm focusing in two consuming things: array and xy graph. In this post I'm trying to find the best way to limit the array size from an 2D Array
Details of my application
1. The data comes from a PLC in a 1D Array with variable array size on each iteration
2. LabVIEW organize the data into a 2D Array with 5 Columns (Time, Sensor1, Sensor2, etc.)
3. The number of Rows is the target to limit. More Rows = More time displayed in the XY Graph
In my software the current max number of rows is 100k (that can display around 15 min of data)
Current I'm using a routine like this to limit the 2D Array Size:
I don't have LabVIEW 2012, and when I try to "Save for Previous Version, it fails (because I use the 2019 "Time Delay" function, which can be "done by hand" but requires a Frame Sequence to tie it to the Error Line, though, now that I think about it, isn't really necessary for this Demo. I just wanted to "slow this down" so that I'd have reasonable millisecond values for the "t" parameter).
When this finishes, it will generate "Buffer Size" (default 1000) points with t increasing by 0 to 20 ms per sample. The default Overflow is 100, so it runs for 1100 loops, taking about 11 seconds, and shows you 1000 points starting at a t value near 1000, ending near 11000.
Some hint's, your PC is shifting data all the time, you need to avoid the add to array.
Init your array once at the start with max size, use it as a ring buffer with pointers .. EDIT : the lossy queue Bob mentioned is the way to go
If it's only to show the data: the chart can do that for you
BUT your screen has 2k-4k pixel for 15 min data (if you want to show all) no need to hold 100k values. Ok it's 2023 , memory is cheap , but do need to store them as DBL?
Diagram update rate, reduce it as much as acceptable (if you handle 500k values)
I have a set of Array tools I posted which has a VIM which is made to force a 1D array size to be a size. This can be a minimum, or a maximum size and will then trim or add from the beginning or end.
I now realize having a 2D array version of this might be useful too. Under the hood it just does array size checking and isn't very fancy.
Sorry @Bob_Schor and @Henrik_Volkers, I think my example VI caused you to focus in the subVI.
The subVI was only an example that I made to have some data to visualize.
The input data of my application is very different from that subVI, there's no Elapsed Time calculation, the Data Type are defined by the PLC (PLC data comes in float array)
I apologize for my mess
I've made another example to replace the first one.
I've removed the subVI, the While Loop and other things that could cause misunderstanding.
Think in this VI as a subVI of a bigger application.
Input data = PLC Data, comes in a 1D Array with data type defined by the PLC and have fixed size (defined by the PLC too). Let's say that this input array have 1000 elements on each iteration.
I only need a few of these elements, that's why I add the "For Loop", to get the elements that I want. Each element represents one sensor or variable of my process.
In my example the "Time" is the element 21,"Sensor1" is the element 30 and etc.
With "Time" + "Sensor1" data I can plot one xy_graph
With "Time" + "Sensor2" data I can plot another xy_graph
With "Time" + "Sensor1" + "sensor3" + "Sensor4" I can plot another xy_graph
With "Sensor4" + "Sensor5" I can plot another xy_graph... and so on
The output of the "For Loop" is an 1D array with only the elements that I want to store and plot
My doubt starts after the "For Loop":
What I am currently doing is to make a 2D-Array to store the data. The column 0 of the array stores "Time", column 1 stores "Sensor1" data, column 2 stores "Sensor2" data and so on
If I leave the 2D array without size limit, the LabVIEW closes by memory RAM or high CPU usage
What I'm doing for limit the array size is the "In Place Element" structure.
• In the VI of the image "Array (limit size v2).vi", the IPE structure is discarding the oldest value of the 2D-Array
• Is there a better way to discard the oldest value before adding the new one. With this I can keep the array with the right limit_size
Note: This is only one part of my doubt, I am breaking my doubts in more parts, to stay simple
In the future I'm gonna make an example that is more similar to my application.
Sorry, @mthheitor, I don't think you paid attention to Henrik's and my attempts to demonstrate some methods to limit an array size. Here's one -- create an array of the pre-determined Maximum Array Size and embue it with the following characteristics:
There are at least two ways to do this. One is to implement what is called "a Circular Buffer", which is an Array with two "pointers", one to the "first element", the other to the "last element". I'll let the name of this "circular buffer" suggest how it is implemented and used. The other (which probably is implemented as a Circular Buffer) is a Lossy Queue. We've shown you both techniques, but you failed to understand how to use them in your application. Either that, or you have so confused us that we cannot figure out why our answers aren't the solution you require. Oh, well, we tried ...
As mentioned by Henrik_Volkers, if you only need to display data, you can use Chart. The chart not only displays data but can also act as a fixed size FIFO. The size of Chart history can be predefined in it properties.
If the number of channels is fixed, you can convert the PLC DBL array to a cluster of DBLs and connect it to your Chart.
Don`t forget to define number of plots in array to cluster conversion utility.
Moreover, if you also need the history of the data, you can get it from the chart's "History" property node.
(PLC data comes in float array) - if your PLC return floats (32 bit) - use SGL type instead of DBL
@Bob_Schor I think that I'm starting to understanding the concept of Lossy Queue. Don't noticed that there was a function called "Lossy Enqueue Element"
That's the vi to store 100k of the PLC Data. I think that's the basic solution, right?
@diman27, don't know if I can use Waveform Chart or Graph, because the Time (dt) isn't constant.
Searching for the difference of the three (chart, graph, xy graph):
A chart is typically used for when you acquire a single point for 1 or more channels at a regular interval. A graph is used when you acquire multiple points for 1 or more channels at a regular interval. An XY graph is used when you sample at irregular intervals
Displays data typically acquired at a constant rate.
• Graph: displays one or more plots of evenly sampled measurements (does not store historical data)
• Chart: maintains a history of data, or buffer, from previous updates (Chart History Length)
• XU Graph: Displays data acquired at a non-constant rate and data for multivalued functions
The instruction OPC-UA Read sometimes gives 100 ms, 200 ms or 500 ms. It depends of what PLC is doing.
The time variable is obtained by one variable of the PLC (in my example it's 21 of the PLC Data Array)
I'm using this time variable to plot a XY Graph
And there's some graphs that the X-Axis isn't time, but the sensor measurement.
Like Sensor1 (X-Axis) vs Sensor2 (Y-Axis)
On the topic of buffers, there are several tools people have made online for handling this. I experimented with XNodes over on LAVA here. I then turned this into VIMs and posted it here. Dr. Powell also posted a VIM version here.