LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Frame Rate Dropping When Uncompressed?

Solved!
Go to solution

Hi, 

 

I am having a problem with dropping frame rates when saving Uncompressed AVI rather than MJPEG. Note, there is no problem when saving MJPEG @ 30fps, but the frame rate drops to approx 5fps if I wire in the 'Y800 Uncompressed' codec (see screenshot).

 

This question is following on from my previous topic (https://forums.ni.com/t5/LabVIEW/Real-Time-Saving-AVI-Subraction-Image-Frame-Rate/m-p/3921842)

 

I'm now looking at the performance of the 'Y800 Uncompressed' vs 'Motion JPEG'. According to the following comparison, there should not be a significant performance drop (8ms plays 7ms write time). http://www.ni.com/example/30542/en/

 

Am I missing something obvious? Or is there an alternative to save the raw images for post-processing?

 

I also get a significant performance drop in saving if I use the lookup table, even when compressing to MJPEG.

 

The ideal goal would be to be able to view the 'log lookup table' version of the video in real-time, while saving the 'non-log' version (so that I can apply different lookup tables post-processing).

 

Cheers!

Matt

 

codec_ss.PNG

 

 

 

0 Kudos
Message 1 of 12
(3,175 Views)

The different compression algorithms are going to work differently based on the Image size (in pixels), depth (number of bits/image), and how they handle compression between Images (i.e. how they encode how a particular pixel changes from frame to frame).  I think the only reasonable thing to do is to test under the conditions that you are encountering.  Note that hardware can also make a difference (some video cards may support compression and decompression algorithms in their very fast memory).

 

Bob Schor

0 Kudos
Message 2 of 12
(3,142 Views)

Hi Bob,

 

I'm trying to capture 1000x1000 pixels at 8-bit depth @ 30fps (so each frame is ~1MB). I am then saving to an AVI with a playback speed of 30fps. 

 

Quite crudely, I've tried recording / timing each of the following settings for 10s (so would expect 300 frames in the saved video). I can open the saved video and check to see if there are ~300 frames to see if the saving is working, at least roughly.

 

Here's what I found (Compression codec / Log LookUp Table? / Result frames):

 

1.) MJPEG / NO / ~300 frames.

Frame rate okay, but post-processing with Log LuT looks rubbish since the saved 'raw data' is 'wrong'.

 

2.) MJPEG / YES / ~100 frames

Frame rate drops to 1/3rd. 'Log video' good, but frame rate is off. Also, don't have the raw data to apply alternate lookup tables since the reverse lookup table yields erroneous 'raw grayscale values'

 

3.) Y800 Uncompressed / NO / ~60 frames

Frame rate drops to 1/5th. If I could get the frame rate right, this would be fine to use since I could post-process the lookup table nicely.

 

4.) Y800 Uncompressed / YES / ~50 frames

Frame rate drops to 1/6th. I did this test just to see how it stacked. This would be the most preferable setup if I can get the frame rate right, as I can view the 'finished video' in real time as it saves. I could also 'reverse lookup table' to get the raw data.

 

 

 

Other questions:

 

- I guess I'm not quite understanding the whole 'producer / consumer architect'. I though the point was to pre-allocate buffers, it saves the passed images to RAM, and then the consumer saves it from RAM onto the HDD/SSD? My laptop has 16GB RAM and I wont need more than a few mins of video (6GB ~3.3 mins) hence this should be okay?

 

- Is there a better way of setting this up? I don't mind it saving in pseudo-real time. I.e. I dont mind pressing 'stop' and then it running on for a few more minutes to save to disk. Currently, as soon as I hit stop, the whole thing stops. I seem to be actually capturing/saving at a slower frame rate, rather than capturing the first '1/3rd' of the video (I've tested this by moving objects in at various points).

 

- I actually tried removing all of the 'viewing windows' to see if this helped. It didn't help significantly.

 

 

Many thanks for your help on this forum by the way!

0 Kudos
Message 3 of 12
(3,123 Views)

I think I've worked out the problem with my 'other questions', although I dont quite know how to implement it.

 

At present, my 'stop' for the consumer loop is just a button, so I guess I'm stopping the saving before it's finished.

 

Is there some way of implementing a boolean or something to say to the consumer "stop saving when there's nothing left in the queue"? i.e. once the producer stops, it sends out a signal for the consumer to stop (once it gets to this part of the queue)?

 

I know this is possible with buffer numbers (see old VI attached - apologies for the mess), but is it possible with passing the images? I dont mind 'dumping' the images to RAM (I've got about 12GB free usually, so enough for 6.5mins) and then them saving afterwards (at a slower rate).

 

If this is possible, is there a way of 'deleting the already saved images' from RAM to clear up space? Or would this play havoc with the timings?

 

Cheers!

 

Matt

0 Kudos
Message 4 of 12
(3,116 Views)

The "Stop" should trigger your producer to enque a "Stop" command to your consumer. That way, it will only see the "stop" when it has finished all other items in its queue.  The Producer can stop immediately if required, but don't destroy the Queue, the listener should always destroy the queue, because it's the last one to need it.

Message 5 of 12
(3,113 Views)
Solution
Accepted by topic author mhall

@mhall wrote:

Is there some way of implementing a boolean or something to say to the consumer "stop saving when there's nothing left in the queue"? i.e. once the producer stops, it sends out a signal for the consumer to stop (once it gets to this part of the queue)?

 

I know this is possible with buffer numbers (see old VI attached - apologies for the mess), but is it possible with passing the images? I dont mind 'dumping' the images to RAM (I've got about 12GB free usually, so enough for 6.5mins) and then them saving afterwards (at a slower rate).


I was going to say "Use a Sentinel, a unique value in your Queue that is sent when the Producer exits", but then saw you did exactly that when you passed the Buffer Number in the Queue.  If you want to pass an Image, it's not so convenient to make a "Unique" image, but it is easy (and cheap, and fast) to make the Queue be a Queue of a Cluster, the Image and a "Stop" Boolean (default False).  In your "normal" Enqueue, you bundle the Image into the Cluster (leaving "Stop" false) and enqueue it, and when the Producer exits, you enqueue the Sentinel, setting Stop to True.

 

In the Consumer, you unbundle the Cluster, wiring Stop to both a Case Statement (which processes the Image if Stop is false) and to the Stop Indicator of the Consumer.  This take essentially no time and little Block Diagram space, and works!  When the Consumer exits, you can Release the Queue.

 

Bob Schor

Message 6 of 12
(3,103 Views)

I've read your advice and wanted to check I understand. This is what I think you're saying:

 

- Currently, I am just en-queuing an image. Putting a 'sentinal image' in is difficult.

 

Instead, I should do this:

- Enqueue a cluster which contains the image, plus a 'Stop' boolean.

- When the producer is 'on' (inside the loop), the 'Stop' boolean is FALSE - this gets added to the queue.

- When the producer finishes (outside the loop), the 'Stop' boolean is TRUE - this is the final thing to be added to the queue.

 

- The consumer contains a boolean which checks the 'Stop' of the cluster. It keeps saving while 'false', and then stops and exits the loop when 'false'.

 

So, essentially it's like we are tagging each image as 'save this' or 'dont save this' by bundling it into a cluster?

 

I think this may (hopefully) fix my 'frame rate problem', as it will just run over until it finishes saving all the frames. I will let you know.

 

Once again, thanks for the help.

0 Kudos
Message 7 of 12
(3,095 Views)

@mhall wrote:

 

So, essentially it's like we are tagging each image as 'save this' or 'dont save this' by bundling it into a cluster?.


Not quite -- you are tagging each Queue Entry as "Continue processing as normal", which translates into Save the Image, or "Stop!  We Are Done!  No More Stuff Will Be Coming!, and, incidentally, there's no attached Image to this final entry, so don't bother saving it".  That's why I chose the name "Stop" (sometime I call it "Quit") instead of "Don't Save" -- I'm a believer in making code as Self-Documenting as possible.

 

Bob Schor

0 Kudos
Message 8 of 12
(3,087 Views)

Hi Bob,

 

I've implemented the cluster and I'm happy it now stops the consumer automatically (much tidier, and less likely to end in an 'unclosed' video file). So, thanks for this!

 

However, I'm still getting a significant drop in frame rate when I try and save with the uncompressed format, or when the LuT is active. The entire program seems to stop shortly after I press stop - i.e. the consumer doesn't seem to continue processing for long at all, which I thought it would if a big queue had been made.

 

Before I hit 'save', the far right display window will happily display at what looks like 30fps, regardless of what other functions are added (LuT, moving subtraction etc), so I know the image processing isnt slowing it down (significantly) when displaying. As soon as I hit 'save', it slows down.

 

I'm just not quite sure where the bottleneck is. Or should there even be one, since we are using a queue function?

0 Kudos
Message 9 of 12
(3,084 Views)

I don't have LV on this computer, but are you reusing Image-refs? Images are handled as references, so if you send the image ref to the queue and then reuse it it'll be overwritten. The solution is to either create a new ref every time (which'll allocate memory until you crash), keep an image ref buffer that's large enough, or send the image data to the queue instead of the ref. 

As i said, i haven't looked at your code, but those are fairly common problems.

/Y

G# - Award winning reference based OOP for LV, for free! - Qestit VIPM GitHub

Qestit Systems
Certified-LabVIEW-Developer
0 Kudos
Message 10 of 12
(3,081 Views)