LabVIEW Vision won't create large AVI File

[DMJeff]

Good Afternoon,

I'm having an issue with storing data from my camera in an AVI file for later processing.

Hardware setup:

Point Grey Gazelle camera

2 Camera Link cables

PCIe-1433 frame grabber

Software:

LabVIEW 2013

Vision Module 2013

Computer:

Dell Inspiron

Windows 7 Professional

NTFS drive - 1 TB

16 GB RAM

PCIe bus does support x4 speed

Currently the camera is capturing approximately 280 fps.

I have a LabVIEW app that just captures the image and writes it to an AVI file.

I'm using 1 IMAQ buffer and the loop is running as fast as it has an image to store.

I found the best performance using the Y800 uncompressed gray scale codec.

First round of testing, with only 8 GB of RAM.

When I run for 10 seconds, all is good.  I get about 2800 images stored.

When I tried to run for 20 seconds, it would run okay for approximately 12 or 13 seconds, and then go very slowly (50 fps).

I assumed it was eating up the available RAM.

So, I replaced the 8 GB of RAM with 16 GB.

Everything seemed to be working as before.

I ran the data capture routine for the 10 seconds, it collected all the images, all is good.

I then tried to run the data capture for 20 seconds.

It seemed to run okay, the image was displayed on the screen, the speed gauge indicated it was running at about 280 fps.

When I opened the AVI file, it only had 3846 frames stored, about 13.7 seconds worth.

I ran the data capture again, for 30 seconds.

It seemed to run okay, the image was displayed on the screen correctly, the speed gauge indicated it was running at about 280 fps.

When I opened the AVI file, it only had 3846 frames stored.

When I checked the hard drive directory, both of these test files were the same size.

2,065,819 KB.

With the NTFS file system, there shouldn't be a limit to the size.

And, the LabVIEW didn't crash when it was executing the write to AVI function.

Is there a max size to the AVI file that I'm running in to?

Or something else?

Thanks,
Jeff

[MoviJOHN]

You are most likely experiencing a problem with memory leaking.  I posted a related issue a few days ago.

The solution depends on what your needs are.  If you can get away with using the MJPEG CODEC, then this will not leak.  If you are trying to output an 8-Bit grey scale image, and using a CODEC that expects color, like YYUV, then you need to cast the image as RGB before calling AVI Write Frame.

http://forums.ni.com/t5/Machine-Vision/IMAQ-AVI2-Write-Frame-Memory-Leak/m-p/2569449#M40014

[BlueCheese]

Definitely check the RAM use of the LabVIEW process to see if that bug is affecting you. Since I am guessing that you are doing greyscale images from the Gazelle and using Y800 mode, I am thinking this might not be your issue. You might just be running into behavior caused by the write caching of the OS. Typically the OS uses any RAM available to cache writes to the hard drive and make them appear to be completed even if they have not been flushed to disk. If your data rate you are writing exceeds the speed of your hard drive, then you would likely see this behavior of the writes continuing fine for a while and then slowing down significantly once the OS runs out of memory to continue buffering. In that case the memory usage would not be charged to the LabVIEW process so you would not see its memory increase.

Eric

[DMJeff]

Hi Eric,

Thanks for the thoughts.

When I had just 8 GB of RAM I certainly saw the image capture speed drop (from 280 to 50 fps) when the RAM was exhausted and the OS had to write to disk.

I was hoping that just bumping the RAM to 16 GB would allow the fast capture to continue for more than a few seconds.

When I started testing with the 16 GB, it looked good, and ran fast for over a minute.

BUT, when I went to open the file, to see what I got, the file contained only 3846 frames (about 13 seconds worth).

The strange thing is, the VI kept running, and displaying the images at a fast rate, even though they were not captured in the file.

I didn't get any error messages, no unexpected crash, no out of memory error, nothing.

Thanks,
Jeff

[BlueCheese]

Ah, I think you've hit the well-known 2GB limit for AVI files. There are some implementations that can go to 4GB but I'm not sure how well-supported that is.

EDIT: I think we can open another bug report CAR about not returning proper error codes when you exeed this limit. CAR 428570 created.

Eric

[DMJeff]

Hi MoviJohn,

Thanks, again, for your thoughts.

When I ran this same VI (read the camera, write the AVI, calculate time) to capture video using the MJPEG codec, it only recorded at 55 - 56 fps.

I need at least 280, and the customer is asking for more.

I'll try the file conversion idea.

It just seems strange to read an 8-bit gray scale camera, convert it to RGB, to save it in an AVI file with the 'Y800 uncompressed gray scale' codec, read it back from the AVI file, convert it back to gray scale, so that I can do the pattern recognition and edge detection processing on the image.

I'm concerned that all the image conversions are going to slow me down.

But, I'll give it a shot and see what happens.

Thanks for the idea,

Jeff

[DMJeff]

HI Eric,

Thanks again.

I thought it might be a size limit, but a quick web search revealed many posts indicating that AVI files can be larger than 2 GB.

Unless there is a LabVIEW limit to the file, that is different than the native AVI limit.

Thanks,
Jeff

[GuenterMueller]

Confirmation: There should be no 2GB issue with AVI files. Since more than 10 years I work with AVIs coming from a Digital-8 camera.  90 minutes of video make a 20 GB AVI that uses the DV codec.

[GuenterMueller]

I have produced several AVIs more than 10GB in size with LabVIEW 2012 SP1 64-bit and a NI 1433 connected to an AVT Bonito with 193fps. The AVIs were produced with uncompressed color data (RGB32). No coded was used.  The PC had 32GB of RAM ...

[BlueCheese]

It depends. Older implementations of AVI could not go past 2/4GB. Newer implementations based on OpenDML can go larger. I suspect that the difference in max size through the Vision tools might depend on which codec source you select from. If you choose one of the built-in Vision ones it does not use the system-provided AVI functionality (which is what the previous versions of Vision Development Module used). The built-in code might have different limitations than that from the system, Try selecting one of the system codecs instead.

Eric