IMAQ optical flow motion estimation fails after a period of analysing real-time images

[ngulungundu]

I am running LabVIEW SP1 (32-bit) in Windows 7 Enterprise SP1 (64-bit) on an HP Elitebook 8540p laptop with a dualcore i5, and 8 Gb of RAM. 

I have a largish VI that logs data from various sensors, and controls a few motors and actuators, through a pair of cDAQ 9174s and various modules. I write the results to a tdms file from a combination of queues and local variables. Below is a screenshot of my overall VI (from the navigation window), showing its 8 parallel loops:

In addition to this, I have two USB webcams that I use to do motion estimation, using the IMAQ optical flow Horn and Schunck VI. Below is a screenshot of my motion estimation diagram:

As you can see, I use IMAQdx to grab images from two different cameras. On first run, I obtain regions of interest for use in the Optical Flow (HS) VI. Thereafter, I get images from each stream, which I turn into black and white frames, and continuously update the current and previous images using a shift register. Then, for each pair of images, I force them to the same size in separate loops, and use IMAQ Optical Flow (HS) VIs to estimate the movement between them. Finally, I  store the mean and standard deviation of the velocities in each dimension in local variables to be logged in the TDMS.

As well as the above screenshot, I have attached the test VI on which the above is based-- ie without all of the other DAQ control and logging that my main VI does. The main difference between the attached VI and the loops included in my main VI is that in the attached VI I'm visualising the velocity by overlaying arrows on the streamed images. I don't do this in my main VI to save resources. 

With all of that said, my problem is the following: Initially, the motion estimation seems to work perfectly. However, after between 5 and 10 minutes (although sometimes up to an hour or so), the motion estimation fails without producing an error. The image on the left below is the velocities chart with the motion estimation working correctly, and on the right is an example of failed motion estimation:

                       

Sometimes the estimated velocity collapses down to zero, as it has in the upper chart above, and other times it enters an exponential slide-- resulting in velocities of the order 1e34 or 1E-38. Below are a few examples of the sort of spurious data that is produced when the motion estimation fails:

Initially, I thought that this might be a performance issue-- my VI uses 95-100% of my CPU time and about 4 Gb of memory-- but while increasing delays in all of the parallel loops frees up processor time, it also usually results in the velocity estimation crashing sooner. Also, memory usage doesn't seem to increase significantly over a few hours' run time. I've also tried replacing the local variables calculated in the motion estimation loops with queues, but the result is that the calculated velocities are zero in all cases. I've been searching the fora with no luck, and so would appreciate any suggestions that you have.

Finally, just to be clear, the problem of the motion estimation failing doesn't  occur in the attached VI, which seems to be capable of running indefinitely (or at least for several hours). This means that you won't be able to replicate the problem, but you can still use it to see which subVIs I've used.

[Vsenior]

Hi ngulungundu,

Your post is long and pretty involved here. I don't know exactly what's causing this. Is there a particular reason you can't attach the VI that doesn't work? Sending over a VI that doesn't reproduce the issue doesn't really help with replicating your problem. 

I can make a few comments based on what you'veput in this post though, perhaps that might help. 

The fact that your VI works for a while then seems to lose track of things implies that there is some sort of memory leak problem going on. you can test that by using a windows application called performance monitor. To find it go to the search menu under "start" and search for "perfmon". Go to "performance monitor" then press the plus sign to add something that you want to monitor. In the list, expand "process" and select "% processor time" and in teh "instances" box that then fills up, select "LabVIEW". This will then monitor how much processor time LabVIEW is taking up. Run your program and watch the monitor to see what it does. You can even take a log that you could save and attach to this post. If your memory use increases as the program goes on, you've got a memory leak. 

I know you mentioned that your program uses a lot of memory, and that's not too surprising, as you have a lot of parallel loops and this is a vision application. These tend to have to use a lot of memory anyway as images themselves take up lots of memory. Slowing the loops would probably just slow down the image processing causing a backlog of data that will probably crash the VI. 

One other thing that you mention is that you've used a lot of local variables. The best use of local variables is reinitialising the controls once per run of the program. You've used them a lot to pass data around the code, with, as far as I can see, reading and writing in multiple places. In LabVIEW, you should try and avoid passing data around with local variables. It's not considered good programming practice as the more of them you use the more likely you are to get race conditions that can cause your data to be wrong (at best), and this tends to happen in very unpredicatable ways, meaning your code doesn't always display the same issues, which makes it hard to trace problems. You should use queues or notifiers instead and avoid local variables as they are unpredictable and too easy to overwrite data that you want.

Really without the full VI (the one that crashes) it's going to be very hard for me to say anything more specific as the code you've attached works fine, so it's probably not here where the problem lies. 

If you'd like me to take a look, please attach it. 

Thanks

Vsenior

[Vsenior]

Hi ngulungundu, 

I understand you've been in touch with my colleague Hans in Applications Engineering about this. 

Please continue to email him rather than replying to this forum, He's going to look after the problem for you as it's easier to do that than work over the forum. If you want to post the eventual solution here when you're finished for posterity, please do.

All the best

Vsenior

[ngulungundu]

Hello Vsenior, thanks for responding to so long and involved a post-- because I wasn't sure what is relevant, I erred on the side of datadumping. I'll continue this, as you suggest, with Hans.

[BlueCheese]

Any chance you can try 64-bit LabVIEW instead? This would help rule out memory leakage as the memory available to he process effectively becomes unlimited (it just starts swapping to disk).

[ngulungundu]

Thanks for the tip, BlueCheese. I switched to 64-bit, however, and the problem still persists-- I'm working on replacing local variables with queues and notifiers, and on combining the image acquisition and analysis into the same loops so that I don't have to pass a reference.