Speed up IMAQ ? Getting only 12 fps

[jbeale1]

Setup: Labview 2015 SP1 on Win7. CPU: i5-4310M @ 2.7 GHz, 8GB RAM.

Camera: www.ximea.com/en/products/usb3-vision-cameras-xiq-line/mq042cg-cm

I made a simple Peak-Hold .vi to stack up image frames, like a long-exposure camera. Enclosed jpeg image shows output as I wave a flashlight around. The Ximea USB3 camera generates 4Mpixel frames at 90 fps as reported by the Vision Acquisition Express block, but the computer can only run my processing and display loop at about 12 fps.  If I remove the image output window, it does not change the loop speed, so just displaying the image isn't limiting the FPS. If I remove the 2D ColorValue shift register and all the IMAQ 2D array conversion blocks and only display the raw image, I actually do see a reported 89 fps loop rate. When running, Windows says my total CPU is at only 30% so this is apparently single-threaded, and I also notice none of my 4 cores is above 50% utilized.

In this vi, I convert the camera image to a 2D array, then a ColorValue array, take the pixel-wise maximum of the current and the previous image, then convert the 2D color value array back to an image and display it.  The camera resolution is fixed at 2k x 2k pixels and unfortunately does not have on-chip 2x2 binning available. For now I am stuck with this particular camera model. I am using the camera in 8-bit mode. It also has a 10-bit mode, but everything is much slower in that mode (I presume labview handles it as 2 bytes per color channel per pixel).

The problem is I want a higher frame rate. I am willing to downsample if that helps. Is there an IMAQ Downsample or Rescale or equivalent?  The enclosed PNG image shows the real VI. The enclosed VI file is the same, except without the static 2D ColorValue array of all zeros initializing the shift register, which being 4 Mpixels takes a lot of memory. Thanks for any suggestions! 

[Bob_Schor]

You found your problem already -- you are trying to do much too much in a single loop.  Because LabVIEW uses Data Flow, you can't acquire images faster than Image Acquisition Time (11 msec if 90 FPS) + Image Processing Time (which seems to be around 72 msec).

I'm not certain I know why your Image Processing takes so long.  How big is it?  What really are you trying to do?  Even if you got all of the Processing out of the Acquisition loop and into a parallel loop, you'd quickly run out of memory if you are really generating 4Mpixel frames.  [Hmm -- is this a color image?  4M pixels * 4bytes/pixel (for RGB color) = 16MB memory -- that a chunk!  Is your image really 2000 x 2000 in size?

Also, please, please, please do not submit code as a PNG (unless it is a LabVIEW Snippet).  They are frustrating to look at (I basically refuse to do so) because we can't "inspect" closely.

Bob Schor

[jbeale1]

Thank you for your reply.  I put the actual Labview .VI code in my post so you could inspect closely, with the exception as noted that I did not include the 4 MB initialization as your server would probably complain, and I also included the PNG image of the code so you could see where I had that initialization block connected. 

Yes, my images really are 2048 x 2048 pixels in color (RGB) so 1 frame is 16 MBytes as you noted.  Processing 12 FPS at this resolution means 192 MB/sec. That alone isn't such a big deal on a modern PC with 10+ GB/s memory bandwidth, but I suspect there are many copies of each frame made as the image is transformed from "image" to "2D array of 32 bit INT" to "2D array of (R,G,B,A) cluster" and back again, and that maybe is the problem.  Would prefer to do transformations "in place" if that would speed the process.  Also, pipeline the process to take advantage of multicore CPU.

This 2k x 2k pixel map is the only resolution this camera provides, and there is no option provided by the interface to downsample it in-camera. I would like to downsample it in LabView if that would reduce my loop time. Is that possible?  Is there a "IMAQ Rescale" or "IMAQ Downsample" or the like? The code I included is a simple case of what I am really trying to do.  

I am no expert, but I suspect I should be looking at something that provides vectorized CPU or GPU operations on images. Does LV already optimize my code in that way, or is there is a dedicated image processing LV library to do this?

[Yamaeda]

You could try to separate it into 4 loops, where the latter 3 is queues. You can do it in steps to see if/how much it helps.

Top loop only captures and send image to a queue(1).

2nd loop pops queue(1) and do the 1st transformation and sends the ColoValue to queue(2)

3rd loop pops queue(2) and do the maximum check, and sends the result to queue(3).

4th loop pops queue(3) and rebuilds an image and show it.

If the transformation is too slow you might even have 2 loops popping the same queue. (This is not a 100% solution depending on speed and how the DLL works, but sometimes it works well)

How'd that turn out?

/Y

[Bob_Schor · Accepted solution]

---

@jbeale1 wrote:

Is there a "IMAQ Rescale" or "IMAQ Downsample" or the like?

---

How about IMAQ Resample?  I haven't used this, but from its Help description, it sounds like it can turn your 2000 x 2000 pixel image into a 1000 x 1000 (or 500 x 500) pretty easily (and in one step).

Bob Schor

[jbeale1]

Thank you both for the suggestions, indeed "IMAQ Resample" does work. It didn't show up with the <ctrl-space> Quick Drop menu, but I found it on the Vision and Motion->Vision Utilities->Image Manipulation palette.  Using a 2x downsample to 1024x1024 pixels, plus a queue with two separate loops, I can get 30 fps which is good enough.

There is one strange thing: the output image is shown at 1024 x 1024 as expected but occasionally, just for 1 frame it jumps up to 2048x2048, then instantly back to 1024x1024 again. I don't see how this is possible in my code, unless the IMAQ Resample simply fails to resample sometimes (!?)  I include the code below, minus a large queue reference initialization frame that would add several MB to the size of the file.

[jbeale1]

Ah. The queue is passing pointers to memory, not the memory itself, and one problem is that I didn't have the "Img Dest" input to IMAQ Resample connected, so the image was transformed in-place and meanwhile the other loop sometimes accessed the image memory before the transformation finished.  Looks like I need to construct my own double-buffering scheme to avoid reading the image before it's converted.