Large data transfer between modules

[FlatCat]

I am new to DQMH, but was thinking of using DQMH for a project in which I collect and store video. I'm not using IMAQdx. Each frame is 300k bytes.

Would it be acceptable to use DQMH request events to transfer frames between a module that collects the images and another that stores them to disk and another that analyzes them in real time? Or would that be too inefficient? 

Thanks, 

David

[Taggart · Accepted solution]

If you were worried, you could just send a DVR.

[FlatCat]

Of course! Thanks.

[TiTou]

IMAQ image is a special beast in LabVIEW, it's a by ref data type so I don't think you need to use DVR.

Forking an image wire doesn't duplicate the image data.

If you have time, take a look at this thread on Lava.org : https://lavag.org/topic/21212-imaqimage-references/Imaq where this is being discussed.

Also, do you really need to transfer your image from one DQMH module to another one?

You might, if one is doing the acquisition and another one is doing processing but if it's only acquire and save as video, one module with 1 or 2 helper loops might be enough.

Also, one general piece of advice I would give about buffer management for IMAQ images is to pre-allocate 2 separate sets of buffers for acquisition and for images to be saved. And a 3rd one for processing if so needed.

[drjdpowell]

Do you know how long it takes to copy your 300kb images (which aren't imaq, I understand)?  Only if this is too large then go with a DVR.  By-reference is more complicated to do right.  By-value is simpler, if you can afford the copies.  300kb sounds quite small to me; I would expect saving those images to be your rate-limiting step, not copying.

[FlatCat]

Not IMAQdx. I read from the camera directly through its DLL. I can collect, analyze and store 120 images per second currently on 4 cameras.  

I have not tried to time how long it takes to copy an image from one place to another. Probably much less than 1 ms. But at the frame rates I am going, every bit helps. 

One vi with two loops is how I have been doing it for a long time. One to collect/analyze/display, another to store. But the architecture is very ad hoc. I thought with something more formal like DQMH, I would make the code more modular and maintainable. Maybe that's overkill. 

[drjdpowell]

"Every bit helps" is also known as "premature optimization".  

[FlatCat]

Normally, I'd agree. There are hundreds of systems out there working nicely. If it ain't broke... But in this case, I am working on adding some significantly more complex analyses, and increasing the size of the images. So while I am reorganizing, I thought to take the opportunity to improve, update, modernize, streamline some pretty old code that was written when I knew less about LabVIEW. 

[FabiolaDelaCueva]

Hi FlatCat,

Keep us posted with your progress and if you run into any issues. At Delacor we are working with a customer that is acquiring images at a very high rate and NOT USING IMAQ. We are using DQMH successfully. We are using DVRs to pass the large data arrays between the acquisition module and the display module.

We are acquiring at 2 Gigsamples per second! 

One thing that we realized is that you want to separate your acquisition from the display, this ensures all your acquisition and analysis never goes to the UI thread. In this particular case, we are using intensity graphs to display the images, so we created a DQMH module for the Intensity Graph that has a request to Display Data and the Acq/Analysis module calls that request. 

Eventually, the acquisition and analysis will be done in LabVIEW FPGA, at that point, we can keep the display module as is and we will just replace the current DQMH ACQ module with the DQMH FPGA module.

Good luck!

Fab

[FlatCat]

Wow. I wish I could sneak a peak at your code. 🙂

As I said above, right now I have a big Main that has separate loops for acquisition/analysis/display, and then user interface. I will separate them.