Start/Stop large data processing at will?

[binpersonl]

Dear all,

What is the best way to do start/stop large data processing at will? Such as when the user click the "start" button, the button will change itself to "stop" button, meanwhile the data processing starts and diplays results. This processing will stop either when it finishes or the user click the "stop" button. If it finishes naturally, the button will become "start" again.

Is "producer/consumer" structure can do that? how the 'procducer' stops the long processing in'consumer'?

Thank you very much!

[altenbach]

Define what you mean by "large" and "long".

Attach some simple code.

What kind of "processing"?

[binpersonl]

---

Time-consuming signal processing, e.g. half hour to several hours.  I am doing the design and plan and do not have code yet.

@altenbach wrote:

Define what you mean by "large" and "long".

Attach some simple code.

What kind of "processing"?

---

[aputman]

Don't use the DAQ assistant. 

[GerdW]

Hi binpersonl,

I am doing the design and plan and do not have code yet.

Generic answer on your rather vague question:

Once you split your "large and long" processing into smaller sub-steps it becomes quite easy to pause/stop the task in between. A state machine approach will also help…

[binpersonl]

---

Thanks, I am not sure how to split, could you give more detals on how to split?  or any simple template/examples I can learn to split?

or can we just use some while loops to control the processing etc?  or notifier to notify the processing to start/stop?

@GerdW wrote:

Hi binpersonl,

I am doing the design and plan and do not have code yet.

Generic answer on your rather vague question:

Once you split your "large and long" processing into smaller sub-steps it becomes quite easy to pause/stop the task in between. A state machine approach will also help…

---

[GerdW]

Hi binpersonl,

how can I say which way to split your processing when you don't tell anything about it?

Usually when a modern computer has to calculate for hours the workload can be split into several parts, either by the algorithm to apply or by the amount of data used for a "worker" routine.

Example:

Calculating a fractal at high res/deep detail can take quite some time. But you can split the work by calling several parallel worker routines, each one processing just a part if the image. This way you can faciliate modern multi-core CPUs as well as "break" the workers faster by sending a stop command to each of them…

Another example:

When analyzing video frames the work usually is done frame-by-frame (or on a specific amount of frames). You can easily stop the analyzing process after each frame, no need to wait till the last frame is reached…

[binpersonl]

---

Thanks, maybe a bit complicated, so no any simple vi program to illustrate the concept /dataflow?  For example,

there is a for/while loop representing processing in costumer side, the user can control start/stop of the loop (processing) from the producer side. Is it possible?

@GerdW wrote:

Hi binpersonl,

how can I say which way to split your processing when you don't tell anything about it?

Usually when a modern computer has to calculate for hours the workload can be split into several parts, either by the algorithm to apply or by the amount of data used for a "worker" routine.

Example:

Calculating a fractal at high res/deep detail can take quite some time. But you can split the work by calling several parallel worker routines, each one processing just a part if the image. This way you can faciliate modern multi-core CPUs as well as "break" the workers faster by sending a stop command to each of them…

Another example:

When analyzing video frames the work usually is done frame-by-frame (or on a specific amount of frames). You can easily stop the analyzing process after each frame, no need to wait till the last frame is reached…

---

[altenbach]

All you need to do is do the processing in a parallel loop from the UI code and have it check for start or stop signals occasionally during processing (e.g. 1-10x per second). How you do that really depends on the problem. Some processes cannot be unterrupted, for example if you do a huge 2D FFT, it needs to complete before it can check the notifications. Of course you could split it up by doing 1D FFTs in a stack of loops and check at each iteration but this will slow the code down somewhat, so you need to find the right balance.

Again, what kind of processing are you doing. Is it memory intensive or CPU intensive? Maybe you are doing it wrong and it could complete in seconds using a better algorithm. See also the videos in my signature.

And yes there are plenty of mechanisms for communcating between loops. (queue, notifier, local variable, channel wires (LabVIEW 2016+ only),  etc.)

For example, nonlinear fitting can be slow so I typically read an action engine inside the model subVI that creates an error 43 (operation canceled by user), which will communicate to the fitting code that it is time to stop.

[binpersonl]

---

Can I ask how do you genenrate the error and read the action engine? and how do you communicate with the fitting code?

altenbach wrote:

For example, nonlinear fitting can be slow so I typically read an action engine inside the model subVI that creates an error 43 (operation canceled by user), which will communicate to the fitting code that it is time to stop.

---