Indeterminate number of concurrent processes

[Porras]

Hi, all.

The question seems to be really easy to solve, but I can't find a way to do it. Well, imagine you have an array of objects. For each one of them, I want to run a process. These processes have to run concurrently. But the problem is that the number of objects in the array can be different in every execution. So, I need to run an indeterminate number of concurrent processes. Is this possible?

I thought of using a FOR structure, but it doesn't work, since this kind of structure is sequential.

Can you please help me?

Thank you very much,
Francisco.

[altenbach]

What kind of "processes" (simple calculations? DAQ? interactive?)?

How many such processes typically run at the same time (A few? tens?, thousands?, millions?)

---

Porras wrote:
I thought of using a FOR structure, but it doesn't work, since this kind of structure is sequential.

---

Are you simply trying to keep all CPU cores busy or do the processes require to be run perfectly in parallel for extended amounts of time?

If you have LabVIEW 2009, you can parallelize the iterations of a FOR loop if iterations don't depend on each other (More information). Maybe that's all you need.

Unless you have as many CPU codes as you have processes, things cannot be fully parallel. Same with shared resources such as disk access.

Message Edited by altenbach on 11-22-2009 02:32 PM

[Porras]

Hi, altenbach, thanks for your fast reply.

These processes consist of timed processes for each object. For example, for object with index 0 in the array:  action 1 - wait 10 secs - action 2 - wait 25 secs... ; for object with index 1 in the array: action 1 - wait 5 secs - action 2 - wait 15 seconds... and so on; these processes have to run simultaneously. The "actions" are performed via Call Library Function Nodes with C++ implemented functions.

They aren't so many processes, can be between 3 and 15, aprox.

Maybe I'll have to upgrade to LabView 2009 and use the parallel FOR loop, because these processes are independent each other. I saw in your link that you have to specify the maximum number of parallel loops, so, I think it'll be possible to use an indeterminate number of loops below that maximum, isn't it?

By the way, will be a problem to use these Call Library Function Nodes executing parallelly?

Thank you so much,
Francisco.

[altenbach]

---

Porras wrote:
By the way, will be a problem to use these Call Library Function Nodes executing parallelly?.

---

Yes, that can be a problem. Who wrote the dll?

Do they actions have to operate in parallel or are the action calls short compare to the wait times? 

---

Porras wrote:
Maybe I'll have to upgrade to LabView 2009 and use the parallel FOR loop, because these processes are independent each other. I saw in your link that you have to specify the maximum number of parallel loops, so, I think it'll be possible to use an indeterminate number of loops below that maximum, isn't it?

---

I thought it will only generate as many parallel processes as you have processing cores, so thing might not work for you as you expect.

I still don't quite get the entire scenario. If it is just a matter of operating various short tasks at irregular intervals, maybe there is a much easier solution. Can you give a few more details.

[Porras]

Hi, and thanks again, altenbach.

- Every dll call is a short call compared with the wait times. If not, I suppose the control execution would pass to the dll function and LabView would wait until it had finished, isn't it?

I attach a chart where you can understand better the overall process. Tell me please if you need more specific details.

Thanks,
Francisco.
 

[Jarrod_S.]

You might check out this blog post for an alternative way to instantiate N concurrent processes, where N is the number of elements in an array. It uses new features (not the parallel for loop) available in LV8.6 and LV2009.

Message Edited by Jarrod S. on 11-22-2009 09:56 PM

[Ben]

This Nugget shows how to run multiple background processes.

If you are going to be calling the same dll from multiple places makes sure it is "thread-safe" and configure the call to NOT run in the UI thread.

Ben

[Jarrod_S.]

Good post, Ben. I had done a quick search to look for something referencing the standard way of spawning processes in LabVIEW to balance out the experimental version I posted, but didn't find your nugget.

[altenbach]

---

Porras wrote:
- Every dll call is a short call compared with the wait times. If not, I suppose the control execution would pass to the dll function and LabView would wait until it had finished, isn't it?

I attach a chart where you can understand better the overall process. Tell me please if you need more specific details.

---

You received some fancy advice above that I was trying to avoid until we figure out if you really need it. 🙂

If the dll calls are short compared to the wait times, we need to know what "concurrent" means to you. As I said, unless you have N CPU cores, "parallel execution" does not exist on a level of CPU cycles. How many cores do you actually have?

So: how much jittter is allowed. e.g. if the dll call (and I assume now that all nodes call the same dll) takes 2ms, would it be sufficient to execute it at t=10seconds for element 1 and t=10.002seconds for element 2, for example if so scheduled. Is that still "concurrent"?

Where do the loops get the list of time delays? Is this e.g. a 2D array?

What is the ouput of each of the parallel  loops? Is this hardware IO, a simple indicator change, or does each produce e.g. an output array?

Does each loop has it's own front panel?

Does each loop have the same number of steps?

What should happen if one code thread encouters an error (close only that loop? stop all?)

Should there be a way to stop everything by the operator if needed?

In my opinon, it would be more reasonable to do it all in one simple master loop. First calculate a list of tasks (array element, time, etc.) and then you simply would call the dll for the various tasks (array elements) as their times arrives, using one master clock.

Message Edited by altenbach on 11-23-2009 08:23 AM

[Porras]

Thank you all.

Altenbach:

So: how much jittter is allowed. e.g. if the dll call (and I assume now that all nodes call the same dll) takes 2ms, would it be sufficient to execute it at t=10seconds for element 1 and t=10.002seconds for element 2, for example if so scheduled. Is that still "concurrent"?

Yes, it's enough.

Where do the loops get the list of time delays? Is this e.g. a 2D array?

Every array element is an object belonging to a class. One of its properties is a 1-D array with this information.

What is the ouput of each of the parallel  loops? Is this hardware IO, a simple indicator change, or does each produce e.g. an output array?

 Each parallel loop doesn't produce anything like a whole, but every calling library node calls to a function that deals with hardware adquisition and returns a value. So, every parallel loop will get (in every iteration) as many values as calling library nodes are there in i.

Does each loop has it's own front panel?

No.

Does each loop have the same number of steps?

No. Maybe this is one of the principal problems.

What should happen if one code thread encouters an error (close only that loop? stop all?)

In that case, only the implicated loop would have to stop.

Should there be a way to stop everything by the operator if needed?

Yes, it has to be.

In my opinon, it would be more reasonable to do it all in one simple master loop. First calculate a list of tasks (array element, time, etc.) and then you simply would call the dll for the various tasks (array elements) as their times arrives, using one master clock.

Maybe I see this more complicated compared with trying to solve it by the "parallel way". Because every loop has a different number of steps, a different set of time delays, every loop execution is independent from the rest... I found it difficult to merge all loops into one master loop and calculate the moments in which every dll node has to be called.

So, I'm gonna try first understanding what Jarrod pointed out, try to put it into practice for my problem, and, if it can't still be used, at least, I'll have learned something new 😄

Thank you so much,
Francisco.