Programmatic way to parallel clone vi for large array?

[86Ohms]

Hello,

I've been attempting to articulate my need to create clone vi's to solve a problem but I appear to have a lack of forum/google search-fu.

I have a cross-correlation sub-vi and need to sweep ~500 frequencies against data collected with a SDR. I'd like to create a large amount of clone vi's programmatically and have them run in parallel to correlate a list of frequencies from an array without having to manually add each clone for parallel execution.

My thought process is getting close to how a FPGA or multi-channel receiver may operate to efficiently acquire signals in parallel from sampled data.

Attached is a simplistic version of the code. The sub-vi is setup for preallocated clone reenterant execution:

Thanks!

[Terry_ALE]

I do not know of a way to do this other than scripting.  This would make a very complex VI.

I see that you would have an FFT inside each of these VIs.  How big will the FFT inside be?

[86Ohms]

@Terry_ALE,

The FFT is actually smaller than the data acquired -- around ~500 points compared to the data samples @ 12500 samples (12.5MS/s @ 1ms rate).

There are actually ~20 extra frequency bins to search within the sub-vi to account for doppler/offset frequency which is another large reason for needing parallel clones in the code.

I'm post-processing the data, and throughput isn't currently an issue. I would like to understand a more efficient process if it can be done strictly in software without having to add a FPGA-like processing module or transfer code to Python and use CUDA drivers with a GPU.

[Terry_ALE]

Sorry I misunderstood.  I thought you were modeling this for porting to an FPGA.

Parallelization on an FPGA vs Software can be quite different.

For software I would use something like this https://zone.ni.com/reference/en-XX/help/372614J-01/lvhowto/acbr_call_clones/

This should allow you to call many instances of the processing VI in software.

[Mark_Yedinak]

Wouldn't "Start Asynchronous Call" work or am I missing something?

[altenbach]

You could just parallelize the FOR loop and the rest will be automatic. 🙂

Unless you have a CPU with 500 cores, creating 500 parallel clones just increases the memory footprint and process contention and will be significantly more expensive than only using a reasonable number of clones based on the number of cores. Typically you don't need more clones than CPU cores.

As a first step you might want to inspect you innermost algorithm in detail for efficiently and inplaceness. Maybe you can gain an order of magnitude there. Create at least five different implementations of your algorithm and do extensive benchmarking to see what's best. Then search the internet for radically different implementations for getting the same result and test those too. Feel free to share your code.

[Ajskrim]

You can use parallel FOR loop as altenbach mentions, just be careful, cause the data order may be mixed (as 10th instance of the loop can finish earlier than 5th). 

[Kevin_Price]

Do the clones retain any state information in feedback nodes or USR's?  I ask because you mentioned intentionally choosing "preallocated clones"...

If so, things might get a little more involved.   Here's an old thread where I was looking to do something similar with subvi's that needed to retain state information.  In that case, I needed a consistent correlation of a unique clone instance to a specific For Loop iteration #.   If you *don't* need that, then never mind.

-Kevin P

[altenbach]

---

@Ajskrim wrote:

You can use parallel FOR loop as altenbach mentions, just be careful, cause the data order may be mixed (as 10th instance of the loop can finish earlier than 5th). 

---

Since he is autoindexing at the output boundary, the order of the result will be correct, and independent of the execution order of the clones.

The parallel FOR loop also has an optional "parallel instance ID" which could be used to tell the subVI what instance it is.

From looking at the original example in the first posts, execution order is not an issue, but we haven't seen any inner code to be 100% sure. (The quality of our advice typically cannot exceed the quality of information given to define the problem. 🙂 )

[Ajskrim]

altenbach, You're right, sorry for the confusion for the author @86Ohms.