strange labview hang

[instrumento]

Huge 7.1 labview program calls a matlab generated dll and randomly hangs, i.e. sometimes it keeps on going, sometimes it stops. 

Labview windows are all blank, code just sits there.  I've tried this both with marking that dll in the ui thread and as reentrant, makes no difference. 

Can't really tell with process explorer which threads are just spinning waiting for something.

Didn't use to happen before we started calling this matlab dll, but it could just be that now the process is even bigger and not necessarily an interaction between labview and matlab.

Funny thing is ... opening a plain old windows explorer window (by clicking in a icon in the taskbar, haven't tried other methods) which should be a total unrelated process, unblocks the whole thing and the process continues normally.

Not even sure where to look to debug this.

[Ben]

---

instrumento wrote:

Huge 7.1 labview program calls a matlab generated dll and randomly hangs, i.e. sometimes it keeps on going, sometimes it stops. 

Labview windows are all blank, code just sits there.  I've tried this both with marking that dll in the ui thread and as reentrant, makes no difference. 

Can't really tell with process explorer which threads are just spinning waiting for something.

Didn't use to happen before we started calling this matlab dll, but it could just be that now the process is even bigger and not necessarily an interaction between labview and matlab.

Funny thing is ... opening a plain old windows explorer window (by clicking in a icon in the taskbar, haven't tried other methods) which should be a total unrelated process, unblocks the whole thing and the process continues normally.

Not even sure where to look to debug this.

---

That one is easy!

1) Toss the dll and make sure it works OK without the dll.

Bonus idea (no charge)

2) If the original is still OK withou the dll, write a stand-alone VI to just test the dll.

Ben

[instrumento]

My statement of "not sure where to look" reflects the fact that running an INDEPENDENT program, in effect "explorer /e,c:" releases what looks like a deadlock in our program, and that I find very strange.  Can you suggest how that can happen?  I can only think that this changes the scheduling of some threads, and as we don't do any complicated threading, besides whatever the labview and matlab runtimes decide to thread,I don't know how to fix it.

[instrumento]

And of course our code works without the DLL, I just so stated in my original post.

And of course the DLL works, and never hangs, with simple tests.

[Ben]

---

instrumento wrote:

My statement of "not sure where to look" reflects the fact that running an INDEPENDENT program, in effect "explorer /e,c:" releases what looks like a deadlock in our program, and that I find very strange.  Can you suggest how that can happen?  I can only think that this changes the scheduling of some threads, and as we don't do any complicated threading, besides whatever the labview and matlab runtimes decide to thread,I don't know how to fix it.

---

Without doing crash dump and analyzing it may be hard.

LV use Cooperative multi-tasking where each thread will run as long as it is remains computable. You may have read the suggestion to make sure there was at least a "0 ms wait" in every loop because of this.

If the hang is due to code never releasing the CPU then when you invoke another program and Windows has to map that into memory, the hanging process may be going into a resource wait state therby releasing the thread.

Arelated situation is you are running a LV app and then open another while the first is running. The running app will hang while the new code get mapped into memory.

Just wild speculation on my part.

Does your "simple dll" test use multiple loops so you can see if dll is hanging the other loop.

I may also have ommited mention that the UI thread is single threaded so anything that dominates that thread will hand all other LV GUI updates.

Ben

Message Edited by Ben on 06-11-2010 01:05 PM

[instrumento]

Looks like we went from easy to hard in a couple of posts 🙂 

I've looked at all the threads under Process Explorer and couldn't see anything out of the ordinary (though there's lots of them and I may have missed something). Curiously starting Process Explorer doesn't "fix" it.

The CPU is idle when this happens, the CPU load is divided between Process Explorer itself and the system idle processes. CPU time for Labview is virtually zero so I doubt we're spinning on a loop.

The only thing I can see is trying is to move the DLL call into a separate process.

[Ben]

---

instrumento wrote:

Looks like we went from easy to hard in a couple of posts 🙂 

I've looked at all the threads under Process Explorer and couldn't see anything out of the ordinary (though there's lots of them and I may have missed something). Curiously starting Process Explorer doesn't "fix" it.

The CPU is idle when this happens, the CPU load is divided between Process Explorer itself and the system idle processes. CPU time for Labview is virtually zero so I doubt we're spinning on a loop.

The only thing I can see is trying is to move the DLL call into a separate process.

---

That could be the sign of a deadlock where the code is waiting for something to become available that only becomes available when you "jiggle the handle".

Still guessing while hoping I am not hurting you...

If the dll is actually not thread-safe and marking at such could be a factor so take a look at this thread to try running your dll in a single thread that is not the UI thread.

Now if that does not help or reveal something novel, I am going to setp bak and let others try to help. THe last thing I want is to lead you down the primrose path.

Ben

[instrumento]

Finally had time to review this.  Created a small vi to test the DLL, got it to replicate the hang, and eventually found a silly way around it.

Summarizing, we call a MATLAB generated DLL that uses the MATLAB runtime to model/minimize some function, and optionally plot some stuff and dump the plot to a file.  The way MATLAB works, according to the guy who wrote the DLL, there is no way to generate the plot and dump the image to a file without generating a window that shows that image.

In normal mode the image just pops up and covers most of the screen.  Eventually our code hangs inside the MATLAB generated DLL.  At this point sometimes the LabVIEW windows go unresponsive (our standard program) or can be manipulated (my test case) but the code hangs.  At this point, opening a Windows Explorer window unfreezes the DLL, but restoring a previoulsy minimized Windows Explorer window doesn't, as well as opening a variety of other programs, e.g. Process Explorer, Irfan View, Firefox, Internet Explorer doesn't seem to do it either.  It needs to be a new Windows Explorer window.

Setting the DLL to not generate the plot fixes the hang, though we don't get the file dump.  Setting the DLL to open the window offscreen also fixes it and we do get a file dump, and thus this is the way we're going to go as no one needs to see those images real time (well, I'll be running the code all day to make sure it doesn't hang, but with the image on screen it hangs quite quickly.)

Don't think there's anything we can do by modifying our LABVIEW or MATLAB code, but this is some strange interaction between the two runtimes and contention for the display.

[ArchivedPointOnePa]

instrumento,

Have you gained any additional insight/understanding of this issue since your last post in July 2010?

I'm fighting a similar problem, and I'm wonder if the ActiveX Server DLL that I'm using is attempting to pop-up a dialog box that is deadlocking with the Labview GUI.  Repainting of the Labview application window unlocks the application and lets it proceed from the point of hanging.

Other related post from 2006:

http://forums.ni.com/t5/LabVIEW/My-Labview-EXE-not-computer-goes-to-sleep-mouse-jiggle-wakes-up/m-p/...

[instrumento]

Wasted enough time on that.  Don't think that I, as an end user, can guess what's deadlocking.

In our case having Matlab draw to a window "off screen" i.e. such that no part of the window is mapped into the actual screen, solved our deadlock and allows us to generate a dump from Matlab for subsequent inspection/logging purposes (besides gives us the results of some interpolation that doesn't need the screen dump).

The only change we made to the code was in those window coordinates and the problem went away.  Not sure where the deadlock was at all, but at this point I don't care or have the time to think about it.