Real-Time Saving AVI Subraction Image Frame Rate?

[mhall]

Hi,

My VI is meant to do the following on command (and usually in this order):

- RUNNING VI: Display real time continuous video on running VI (works fine). Let's call this "RT image".

- SNAP: Snap and Save reference image on pressing 'snap' (works fine). Call "Ref".

- SUBTRACT: display real time difference video between "RT image" and "Ref" (works fine). Call this "DIFF".

- SAVE: Save "DIFF" as an .avi video file

- STOP: Closes everything nicely.

As soon as I hit save (in mhall_original_nobuffer), the frame rate drops a lot, which is expected, as the acquisition rate is much higher than the saving rate.

I've tried queueing it in file "mhall_altered_buffer", but it seems to just save the "RT image" and not the "Diff" (but the frame rate is okay). It also gets stuck in the consumer loop when you hit STOP.

Could I please get some help on how to save a subtraction image in real time? I'm a bit lost with the 'producer/consumer' loop.

FYI I'm using a GigE Dalsa Genie Nano, 30fps (controlled in MAX) @ 1000x1000 pixels, 8-bit grayscale. I dont think the data transfer from the GigE is the problem, it's the writing to file!

Many Thanks,

Matt

[Bob_Schor · Accepted solution]

• Why are you casting the Images to the same Grayscale (u8) format used to acquire them?  
• At 30 fps, saving a 1 MB video requires streaming at 30 MB/sec, certainly do-able, but a bit of a strain.  I don't know how much processing the AVI system tries to do (and don't know what Codec you are using and how much this adds to the load).
• A great way to speed up time is to cut down on the displays.  Consider displaying every 10th image, which should give you a 10-fold increase in speed from those loops, at least.
• Do you realize you are pulling the same data from the camera multiple times?  This means multiple megabyte Transfers from camera memory to PC memory over some sort of bus.  Better is to get the data into memory (an IMAQ Buffer) once, and use it as often as you need to. 
• I'm a little confused  about your processing.  It would help if you could simplify and organize things so wires flow mostly left-to-right and stay as straight as possible.  I also see a Feedback whose purpose I don't understand.  
• I suspect you are not doing what you say you want to do, namely to "snap" one Image and save the difference between the Snap and the "raw" data from the Camera.  I was going to say you should save all the Images and do the subtraction later, but I forgot you were making AVIs (of unknown providence/Codec) of huge sizes and might not be able to recover the original single images in their 1000 x 1000 Pixel glory after the AVI compression takes place.
• It looks like (but it can't be, can it?) you subtract the the Snapped Image from the Acquired (Grabbed) Image, saving it in the Cast_Sub Buffer, which you then use as the Buffer to hold a second Grabbed Image (overwriting the careful subtraction) which you then save to an AVI.  [I might have gotten lost in the kinks and twists of your Image Wires, see the above plea to keep them horizontal and straight, and simpler, I should have added].
• With all this stuff going on, I was going to recommend a better Producer/Consumer construction (based on passing Images, not camera buffer pointers requiring another movement of data from camera to memory) than Queues, but there's more basic work to be done, first.

Bob Schor

[mhall]

Hi Bob,

Thanks very much for your comments. I'll address them and tidy everything up before I come back with any more queries!

For now though: The AVI saving is just because this is what previous users in our group have used (who have since left so I cant ask for their help).

• It looks like (but it can't be, can it?) you subtract the the Snapped Image from the Acquired (Grabbed) Image, saving it in the Cast_Sub Buffer, which you then use as the Buffer to hold a second Grabbed Image (overwriting the careful subtraction) which you then save to an AVI.  [I might have gotten lost in the kinks and twists of your Image Wires, see the above plea to keep them horizontal and straight, and simpler, I should have added]. - This may well be why the second VI (with the saving) is saving the original image instead of the subtracted image. Thanks for this!
• Do you realize you are pulling the same data from the camera multiple times?  This means multiple megabyte Transfers from camera memory to PC memory over some sort of bus.  Better is to get the data into memory (an IMAQ Buffer) once, and use it as often as you need to - Will fix this. I think I got confused myself when trying to implement the saving aspect of it.
• I suspect you are not doing what you say you want to do, namely to "snap" one Image and save the difference between the Snap and the "raw" data from the Camera.  I was going to say you should save all the Images and do the subtraction later, but I forgot you were making AVIs (of unknown providence/Codec) of huge sizes and might not be able to recover the original single images in their 1000 x 1000 Pixel glory after the AVI compression takes place. - For now, the images are purely for qualitative purposes of looking at fringe patterns (the difference image), so I'm happy to save to AVI for this reason. Would it potentially be better to save the images in a different format?

Cheers,

Matt

[Bob_Schor]

Difficult to say a priori which method of saving video data will be faster.  There are codecs, compression algorithms, dependence on data, etc.  Best advice is to "do it the slow way", save individual images using some lossless method (so not .jpeg), then use these images to simulate a camera presenting the data at 30 fps and attempt to save them as AVIs and see which method "wins".  Note that it may well be data-dependent, so test it with "real data".

Bob Schor

[mhall]

Hi,

Thanks for your earlier help – I’ve tried to implement everything you’ve said and hugely simplified (and tidied!) the code. Apologies for the previous mess! See attached.

I’m happy now as far as the acquisition goes. The program grabs a real time image (on running), copies a single frame of the grab (SNAP button), and displays the absolute difference between the two (SUBTRACT button).

I’m also happy that the program now saves the original grabbed image at the proper frame rate using the buffer-queue system. For the Motion JPEG codec, this runs instantaneously with the program. For the Y800 Uncompressed, it takes some time to stop after I hit ‘STOP’ (which makes sense), and doesn’t quite seem to work. But I’m happy with just a compressed video for the time being (the pixel values seem to be sensible when opening still images in MATLAB).

• Is there a way of saving the ‘difference image’ instead of the grabbed image? I assume this has something to do with the ‘passing images’ method you mentioned?

• Ideally I also want to add a lookup table to the ‘difference image’ before saving it to enhance the contrast – I’ve done this before for acquisition and display, but don’t know how this affects saving?

Cheers!

Matt

[Bob_Schor · Accepted solution]

---

@mhall wrote:

• Is there a way of saving the ‘difference image’ instead of the grabbed image? I assume this has something to do with the ‘passing images’ method you mentioned?

• Ideally I also want to add a lookup table to the ‘difference image’ before saving it to enhance the contrast – I’ve done this before for acquisition and display, but don’t know how this affects saving?

---

Saving any set of Images as an AVI follows the same procedure whether the Images come from a Camera or from a "computation".  First steps -- open the AVI and create a Buffer to hold the Images.  Next, in a Loop, load the Buffer with an Image (from a Camera or from a Computation) and write to AVI.  When the Loop exits, close the AVI and Dispose the Image.

If you want to do any processing of the Image before writing it to a file, you certainly can do that.  The only "trick" is how much time such processing takes.  Having a Producer/Consumer pair between acquiring the Image (from Camera or Subtraction) and Processing/Saving can give you a little "elasticity" in the form of a Queue, but you may want to monitor your Queue size and be sure you are not getting compute-bound.  My "guess" is you'll be OK.

Bob Schor

P.S. -- congratulations on your progress.  Now get those horizontal wires a bit straighter and remove as much White Space as aesthetically possible.  The Goal is "Fit entire Block Diagram on a Laptop Screen".  A very helpful trick for doing this is to say "This code does X, I don't need to "see the details", so I'll make it a sub-VI, design an Icon for it (maybe a square box that says "Subtract Ref Image" so I'll know what it is on the Block Diagram I that I use it), and replace an acre of loops, structures, wires, function with a 32 x 32 pixel Iconicized sub-VI.  As an example, here is a VI I wrote to save an Image as a PNG, also writing a "Ticker" (clock timing) information file to help during development and debugging:

Save Image

[billko]

---

@Bob_Schor wrote:

---

@mhall wrote:

• Is there a way of saving the ‘difference image’ instead of the grabbed image? I assume this has something to do with the ‘passing images’ method you mentioned?

• Ideally I also want to add a lookup table to the ‘difference image’ before saving it to enhance the contrast – I’ve done this before for acquisition and display, but don’t know how this affects saving?

---

Saving any set of Images as an AVI follows the same procedure whether the Images come from a Camera or from a "computation".  First steps -- open the AVI and create a Buffer to hold the Images.  Next, in a Loop, load the Buffer with an Image (from a Camera or from a Computation) and write to AVI.  When the Loop exits, close the AVI and Dispose the Image.

If you want to do any processing of the Image before writing it to a file, you certainly can do that.  The only "trick" is how much time such processing takes.  Having a Producer/Consumer pair between acquiring the Image (from Camera or Subtraction) and Processing/Saving can give you a little "elasticity" in the form of a Queue, but you may want to monitor your Queue size and be sure you are not getting compute-bound.  My "guess" is you'll be OK.

Bob Schor

P.S. -- congratulations on your progress.  Now get those horizontal wires a bit straighter and remove as much White Space as aesthetically possible.  The Goal is "Fit entire Block Diagram on a Laptop Screen".  A very helpful trick for doing this is to say "This code does X, I don't need to "see the details", so I'll make it a sub-VI, design an Icon for it (maybe a square box that says "Subtract Ref Image" so I'll know what it is on the Block Diagram I that I use it), and replace an acre of loops, structures, wires, function with a 32 x 32 pixel Iconicized sub-VI.  As an example, here is a VI I wrote to save an Image as a PNG, also writing a "Ticker" (clock timing) information file to help during development and debugging:

Save Image

---

Just to expand a bit on why an organized block diagram is critical (and it's not that data flows faster through straight wires).  A neat, organized block diagram is analogous to text code with consistent, nested indentation.  The compiler doesn't care, but anyone developing the code would strangle you if you used random indents - and I would probably just leave the room if that happened.

So just remember, neatness counts!  The frustrated developer you save just might be yourself.  😄

[mhall]

Thanks, both of you, but especially to Bob!

I've got it working. It really was as simple as Bob said with passing the image instead of the buffer number!

There's a slight problem that I HAVE to now save a video for it to close properly, but I can sort this myself I'm sure.

Much neater now, but probably still not neat enough yet.

[cbutcher]

Looks nice. If the problem (at least as far as this question goes) is solved, make sure to mark the solution(s). I'd suggest messages 2 or 6, but you know which one(s) helped you most (this is much easier to type when you didn't have anything to do with the thread 🙂 )