Recording video output from Basler camera acA2500 60um with stroboscope

[michaelfriedmn]

Hello,

I am trying to output a video recording to an AVI file, but am encountering issues where only one frame is being recorded rather than a whole video. This issue arises intermittently and sometimes goes away when LabVIEW is restarted. I have attached the VI (and a block diagram image) I have been working with. Does anyone have any ideas on how to solve this problem? Any help would be greatly appreciated!

Michael

[Debrosse]

I haven’t had a chance of running your code but here are a few thoughts looking at your diagram:

-not sure you need the sequence frame. Dataflow will progress well without it. 

-you have both the camera interface and your equipment running in the same loop. Is there a reason you want them to be synced? If not, separate out them into seperate loops. Depending on the timing of your daq device, this might be the reason you’re only capturing 1 frame. A good idea when coding is to separate out your devices and test them individually. This way you’ll be able to confirm your camera works properly before you try to integrate a daq device with it. That is, design your code so you test individual parts (unit test) then integrate the working individual parts only when you confirm your unit tests are working properly. 

-your architecture depends on each other too much. When running cameras especially, you might run out of memory depending on your sampling rate and the number of items connected to the USB port your basler is connected to. To avoid this, consider a producer consumer design pattern for your camera. The camera will produce frames. They will then be consumed and saved in another loop and then stitched in a video in the final loop. This will make sure you don’t have dropped frames and prevent your ui from hanging. Your daq too can benefit from a producer consumer setup as well. Same as the camera 3 loops: stream, process, save data. 

-using producer consumer will require you to stop multiple loops. Look up the built in design pattern templates for examples. 

-finally, once you’re comfortable with producer consumer, you might want to look up a queued message handler design pattern for improvement in performance. 

Good Luck! 

[Bob_Schor]

I agree with the previous comments.  Here are some others:

• Delete the Sequence frame.  Data Flow (via the Error and Task Lines) to the work for you.
• Try to neaten/compress/straight-wire your Block Diagram.
• Are you using LabVIEW Project?  You should be, and if you did, you could define two Tasks that would handle all of the DAQmx "Configure" VIs, reducing your DAQmx calls to "Start Task", "Read" (or "Write"), and "Stop" (or "Clear") Task.  Look up "Learn 10 Functions in NI-DAQmx and Handle 80% of your Data Acquisition Applications".
• Your Camera outputs a 2590x2048 10-bit image, which comes to 10MB/image, or (at 60 fps) >600 MB/sec being transferred to memory.  This is right around the rated USB3 speed, so better not be doing too much else!
• IMAQdx saves images in its driver.  You have used the default "Configure Grab", which allocates buffers for 5 images used in a "ring buffer" configuration, so Image 6 is stored where earlier Image 1 was kept, meaning you need to get and process Image 1 before it gets over-written.  Speed is of the essence, and Producer/Consumer is probably mandatory.
• You use the word "stroboscope" in your title.  I think of a Strobe as something that has a very brief illumination.  I don't know how they work with video cameras -- do they "capture" an entire Image "all at once", or piece-meal (I don't really know)?  How you synchronize your illumination with the Camera may present interesting questions.

Bob Schor

[michaelfriedmn]

Hi Debrosse,

Thank you for your reply! After doing research on Producer/Consumer architecture, I am still a bit confused on how to implement the video recording/processing part of my block diagram into that design pattern. I do know that I will need one producer for acquisition and two consumers for image processing, and display/storage. All three of these operations are split up between different IMAQdx blocks (i.e. Open Camera, Grab, Close, etc.), but it is unclear to me how to split them up into the three different while loops, as well as what to connect to the Enqueue and Dequeue blocks. I greatly appreciate any guidance you can provide!

[Debrosse]

I’ll upload some snippets in a few days for you to consider.

Bob makes a good point about data throughput. 700Mbs/sec is very large. USB3 can only reliably process 450Mb/s per bus. You haven’t experienced this yet, but could be a serious issue for you going forward. Basler has configuration software. In it, there is a stress test module that will run your camera for something and tell you if it drops frames after a period of time. Worth you using so you trim your resolution or slow down your acquisition rate.

https://youtu.be/76Ox5kPCloM

Here’s a link to see how the Bandwidth manager works in Baslers Pylon software. Note: if your camera isn’t seen in the software, you’ll have to go into MAX and switch the drivers.

Net: start with the Pylon software to fine tune your camera and to confirm your caputure rate. Then switch drivers in MAX and setup for your labview project! Make sure to save your camera configuration in MAX. As Bob mentioned, using a Project is highly recommended so you can save this configuration info within.

[michaelfriedmn]

Hi Debrosse,

I'm still not quite understanding how to properly implement the producer/consumer architecture as you described. I've attached my most recent VI, and I am getting an error saying "the type of the source is variant" when linking the Dequeue element out to the AVI Create block. Also, Labview returns an error saying that Obtain Queue has an "unwired or bad terminal." I'm assuming the error is due to the data type not being specified. What changes do you recommend I make? Thank you for your help!

Michael

[Bob_Schor · Accepted solution]

OK, this is going to be a long reply, because there's a lot to consider in making an IMAQdx Producer/Consumer for making AVIs.  IMAQdx saves image data inside the Driver in something called a Buffer.  You code used the IMAQdx Configure Grab, which defines the default Camera as Cam 0 (I made you specify it), allocates only 5 Buffers (I allocated 30, which might not be enough), and starts the Camera.

Here's the code -- I removed everything except the Camera/AVI stuff, as that's a lot to cover.

The Producer is responsible for running the Camera and getting it to save Images in its buffers.  It doesn't have to pass an Image to the Consumer (and probably shouldn't even display it, especially at very high frame rates) -- it only has to pass the Buffer Number.  Buffer Numbers are U32s, but I coded the Queue for an I32, as I want to use a Buffer of -1 as a "sentinel", a signal to the Consumer that the Producer has exited (much nicer than forcing an Error).  We'll get to stopping the Producer in a minute.

The Consumer takes the Images (by the way, you never specified the Format for the Images, so they'll be the default, which I think is 8-bit Greyscale).  I specified Motion JPEG as the Codec rather than "whichever NI listed as first" -- feel free to change.  So what happens?  The Dequeue gets the next Buffer Number, which should be >= 0 (so that <0 is False).  Using this number, it pulls out that Image from the Camera (IMAQdx Get Image2) and passes it to the Write AVI.  You need enough buffers to make sure you read and save it before the Camera overwrites it -- if you allocate 30, and the frame rate is 30 FPS, you might think you have a whole second of buffering, but it's less than that (experimentation is the best way to find out how fast you can go).

So what happens when you push Stop?  This is an old "multiple-loop" problem, how to stop them properly.  In Producer/Consumer, the Producer gets the Stop, and "stops Producing", telling the Consumer "I've stopped, so you finish up and clean up for both of us".  The Producer sends this message to the Consumer by specifying an "illegal" Buffer, -1.  It also needs to "clean up" its own IMAQ Image.

When the Consumer gets the -1 buffer, it takes the True Case (not shown in the Figure, but it is empty of code, and just passes the True through to the Stop Indicator).  When the Consumer exits, it knows the Producer has exited, so it can Release the Queue safely, stop the Camera safely, close the AVI, and dispose of its IMAQ Image.  Done.

Bob Schor

[michaelfriedmn]

Thank you, Bob! It works great! 😄

[Harsh84]

Hello Bob,

Your explanation really helped to explain how to properly incorporate a producer/consumer loop. My question is what would be the best way to capture the images on command? So basically, the camera would be capturing images the entire time for the user to view. But when the proper conditions are reached, you would have the ability to hit a record button and record for the desired amount of time. And when the consumer loop has properly finished, it can go back to just viewing mode. 

I would think you would put a event structure, where the timeout event is to just capture images and the event when record is hit runs the producer/consumer loop.

Thanks,

Harsh

[Harsh84]

Or I was thinking, you can put a while loop around a case structure. The case structure will have two cases, one with just the viewing option and the other with the producer/consumer loop. Is it possible to link up the case selector with the stops of the other while loops so you are able to stop the viewing loop and it is able to switch over to the producer consumer loop in the next iteration of the outer while loop.