Save Sequence Images with a given timing

[Emanuele_S]

Dear Diane,

I've got your point. Indeed I posted the program before reading your previous message (I posted two messages consecutively), I'll start working with LL ring now. I'll let you know how it goes.

Thank you again

Emanuele

[Emanuele_S]

Dear Diane,

I've modified the VI you sent me and now it works much better, you were asbolutely right. Now I'm able to save 10 images per second, even 20. However my program doesn't work correctly yet. In order to test if I had acquired all the images during teh recording time, I framed my hand in the last second of the acquistion (before of course my hand was not framed). As a result, I have a certain point in my image sequence in which my hand appears, then it disappears, then it apperas again and so on for a few times. It's like the last buffers are repeated and recorded!

I think I've made some conceptual mistakes in the loops. It might be possible that I didn't use properly the SHIFT REGISTER, but I don't know how it works. I tried to look for how it works but I haven't understood it.

In addition, when I press STOP teh program keeps on running and I have to stop it by clicking the red circular button in the Front Panel Menu Ba; I tried to look at the highlighted execution and it seems it doesn't do anything after I press stop. Does it have anything to do with the loop command that you connected to the error in the producer loop?

I think anyway we're getting closer to the final program, but there are still a few issues to fix.

Please let me know, the new VI is attached.

Thank you again.

Emanuele

[DianeS]

Hi Emanuele,

I'm very happy to hear that you are now able to acquire images at the rate you want!  Good job!

I made a small mistake in the VI I posted for you.  Go ahead and connect the output of the shift register in the top loop (the error output) to the "Release Queue" function.  Now the VI should stop when you press "stop".  I got ahead of myself a bit, thinking of ways to ensure that the bottom loop doesn't stop until all of the elements in the queue have been processed.  Then I decided to let you worry about that if you wanted to.  I'm sorry for the mistake.  Your bottom loop doesn't stop now because the queue reference never goes invalid to produce an error.

If you don't understand shift registers, may I suggest you take one of the online LabVIEW introduction courses?  Shift registers are fundamental and you'll need a good understanding of them.  There are two courses:

Three hour course

Six hour course

I'm not sure what's going on with what you describe about the images of your hand.  I do have a question for you.  On your front panel is an indicator called "Last valid buffer".  Does that number ever get stuck at a value?  If not, what does it do when you run your program?  I'm trying to ascertain whether your acquisition gets "stuck" or something, causing the last few buffers to be repeated.

By the way, the VI you attached is incomplete.  I assume that's not really the final version, because it doesn't run! 

Go enjoy the rest of your weekend! 

Diane

[Emanuele_S]

Dear Diane,

thank you for the links for the courses.As sson as I have time I''l try to follow the coruse.
You know, I've been looking for something simlar but I haven't found them, probably I didn't know where to look for.

My VI now runs better, I think you were right, before (in the previosu version eof the VI, the one with the wrong error connection) sometimes I noticed that the indicator last valid buffer got stuck, and this happened after I pressed stop. That might also be the reason why I got a repeated buffer at the end of the video.

Now I have another problem though: I tried to frame again my hand moving continuosly. I'm acquiring images at 20 fps and I have also set a buffer of 20 images. However some frames are skipped fro time to time. I can see it because my hand moves slowly and there are missing parts of the movement in the saved images.

What might cause thie problem?

Do I have to set the buffer better?

I attach the VI again. Please let me know.

Emanuele

[DianeS]

Good morning Emanuele (evening where you are),

I've modified your VI slightly to do two things:

First, I connected your error wire correctly to "Release queue".

Second, I put "Get queue status" in your bottom loop.  This will tell you if there are still items in the queue after you press "stop".

Of course the "last valid buffer" froze after you pressed "stop".  You stopped that loop.  Therefore you're no longer acquiring images.  That makes perfect sense.  My question was, does that indicator ever freeze while the loop is running?

I would increase your number of buffers a little bit -- I think 20 buffers is too few.  Give yourself a few extra.

Finally -- and I don't know if you can do this -- can you acquire images of a digital clock, one that shows 10ths of seconds, instead of your hand?  Do you have a stopwatch or something like that available?  It would be really great to have some timing information to help us with this last problem.  You'll know what the clock time should read, so it will be pretty straightforward to figure out what's happening with the acquisition by comparing what you SHOULD see with what you DO see.

Does that make sense?

So my modifications to your VI aren't intended to fix the problem...they're intended to give us more information to lead us to the problem.  Acquiring images of a running stopwatch would be a very helpful thing to do also.

What do you think?

Diane

P.S.  You're very welcome for the course links!  I hope they prove helpful!

[DianeS]

Forgot to attach the VI.  Here it is!

[Emanuele_S]

Dear Diane,

tomorrow morning (in my new day) I'll try to see whether your revised version of the VI works better. In the meanwhile I've tried to use the different buffer values, and the fact that some frames are skipped is for sure sensitive to this number.

I have also found an article on Ring acquisitions on the NI website

http://www.ni.com/white-paper/4001/en

It seems that the number of buffer does not have to be set particularly high, this article says "When acquisition is faster than image processing, keeping up is impossible, regardless of the number of buffers allocated. As a result, you typically need to allocate only three to five buffers. The buffers serve as a cushion for when the operating system multitasks, which sometimes results in processing taking longer. As long as the average processing time is less than the acquisition rate, the extra buffers help keep the acquisition from catching up to the buffer being processed."

This also cleary gives me a reason why several buffer wer skipped in the previous version with Grab (and so the number of images was less than expected) and of course also tells me that I'm on the right  track using the Ring fucntion (as I could easily understand reading the first paragraph of the article as well).

Today I only tried higher values, but tomorrow I'll also try lower ones, to see whether the problem is more evident or less evident. Probably it's not possible to get rid of this problem completely, I'll see what I can do. I'll let you know what happens.

As usual, thank you very much for your help again.

Have a nice day.

Emanuele

[DianeS]

Hi Emanuele,

As I stated in my previous post, my modifications are not intended to make the VI work better.  They're intended to give you some additional information about what's going on.

I'm glad you found a useful article and I think it's really great that you're researching these things on your own!  However, I must point out the following:

Using a producer-consumer architecture means that your image processing can be slower than your image extraction (acquisition of a single image from the buffer), while not affecting the speed of your image extraction.  That is the whole idea behind using a producer-consumer.  Your image acquisition/extraction can take place at one speed.  Your image processing can take place at a different (slower) speed.  The image processing time DOES NOT AFFECT the image extraction speed when you use a producer-consumer!

The statement in the paper that you quoted will be true if the image processing is in the same loop as the image extraction.  With a producer-consumer, it is not.  That's the reason I suggested a producer-consumer architecture in the first place.  By putting the image processing in its own loop, the image processing code does not slow down the extraction loop.

Let's say that your image extraction is faster than your image processing (so you are enqueueing faster than you are dequeueing).  What will happen is that the number of items in your queue will get larger and larger.  Both loops will still execute, each at its own speed.  In that case, if you stop both loops at the same time, you will lose whatever data is still in the queue.  The images have been extracted, but not yet processed.  When you look at "Get Queue Status", you will see "# pending remove" will get larger and larger.

Let's say that your image processing is faster than your image extraction.  Your image processing loop will wait patiently until your extraction loop enqueues an image, then it will dequeue that image and process it.  In this case, "# pending remove" will always be 1.

One final point:  the number of buffers you configure does matter.  Here's an example which I hope will illustrate why:

Let's say you are acquiring images at 20Hz (one buffer every 50msec).  It will take 1 second to fill all 20 buffers.

BUT, let's say your image extraction loop (what we've been calling the acquisition loop) takes 100msec to execute.  That means that for every 2 buffers you fill, you will only extract 1.  Like this:

Iteration 0:  fill 2 buffers, extract 1.  19 empty buffers remaining.

Iteration 1:  fill 2 buffers, extract 1.  18 empty buffers remaining.

...

Iteration 19: fill 2 buffers, extract 1.  0 empty buffers remaining.

Iteration 20: fill 2 buffers, extract 1.  Overwrite 1 buffer that already contains an image.

So in this example, after 20 loop iterations (2 seconds of image acquisition), we're starting to overwrite existing data because our extraction loop is running too slowly.  How long does it take "IMAQ Extract Buffer" to execute?  We don't know.  Keep in mind that you are acquiring images in hardware, but you are extracting them in software.  Thus it is perfectly possible that you aren't extracting the images quickly enough to prevent overwriting buffers that already contain data.

If that turns out to be the case, what to do?  Well, you can configure more buffers.  If you configure more buffers, it will take longer to overwrite a buffer which already contains an image.  Say, given the timing in the example above, you configured 100 buffers instead of 20.  Now it will take 10 seconds before you start overwriting buffers which already contain images.

I have attached three VIs for you to run, which I hope will give us some more information.  The first one is a producer-consumer.  I've added a calculation for the image extraction loop speed, so you'll be able to see how much time it takes that loop to execute.  Also, I've added some more information for you.  Run the VI.  After you press "stop", compare the number of images enqueued.  It should equal the number of images dequeued plus the number of images pending removal from the queue.

The second VI is also a producer-consumer, but it does not contain any of your image processing. 

The third VI eliminates the consumer loop and all of your image processing.  The calculation for the acquisition loop speed is still there, so you will still be able to see how long it takes that loop to execute.

My questions for you:

1.  How long does it take the image extraction loop to execute in each VI?

2.  Is there a difference in extraction loop time between the three VIs?  If so, what is it?

Keep up the good work!

Diane

[Emanuele_S]

Dear Diane,

I ran all your VIs. In all of them I set 100 as number of buffers, frame rate set 20Hz.

I have the following answers for you:

1) the loop execution time oscillates between two values, 46.875 ms and 62.5 ms (sometimes it also goes up to 78.125, probably depending on the tasks executed by Windows). I think this should give me and indication on the number of buffers I have to set to prevent the overwriting problem you mentioned;

2) The loop execution time is the same for all of the VIs, this is what I expected as you said that the producer-consumer structure doesn't have to affect the extraction time.

However, there is something that doesn't correspond to what you expected, because the number of dequeued element is not equal to the number of enqueued elements (the numer of saved images in the folder is 2 images less than the number of dequeued elements) and the value of pending remove is constantly 0, but you said that the sum of pending remove and dequeued elements should be equal to number of enqueued elements.

Details of the VI runs:

Producer-Consumer with Image Processing

N. Buffer: 100

Loop Exec. Time: 46.875 ms - 62.5 ms (sometimes 78.125)

last valid buffer 197

pending remove: 0

images enqueued: 198

images dequeued: 101

Producer-Consumer with No Image Processing

N. Buffer: 100

Loop Exec. Time: 46.875 ms - 62.5 ms (sometimes 78.125)

last valid buffer 196

pending remove: 0

consumer loop iterations: 197

images enqueued: 197

Only Producer

N. Buffer: 100

Loop Exec. Time: 46.875 ms - 62.5 ms (sometimes 78.125)

last valid buffer 199

images:200

I hope this can helps us understand better where the problem is.

Thank you again for your help.

Have a nice day

Emanuele

[DianeS]

Hi Emanuele,

Thanks for running all those VIs and reporting back!  Now we have a lot more knowledge than we had before, and we learned some interesting things.

Next let's see where all those images are going.  I'm bothered by the fact that you're enqueueing 200 elements (or thereabouts) but only dequeueing 100 (or thereabouts).   So, I've added some code so we can see how long it takes the consumer loop to execute.

I believe I made one mistake.  I should have wired up "# of items in queue", not "# pending remove".  I apologize...my brain clearly was not working properly.  Can you run this VI and tell me how many items are in the queue after you press "stop"?

I think we're closing in on it, though!   You're doing great.  I'm sorry if all this seems really tedious, but I do feel like it's leading us to the answer -- and a good solution for you.  Have a great evening!

Diane