Multithreaded Instrument Communication

[jarc56]

I am trying to write a multithreaded instrument reset subroutine in LabWindows/CVI.  Each thread performs an individual reset on an instrument using its own instrument wrapper reset function.  When running the resets sequentially (not multithreading the resets), there are no issues communicating with the instrument and resets are performed appropriately.  However, when the same reset function calls are implemented in separate threads, I find mixed results as far accomplishing the resets.  Rarely do all instruments are reset properly and I have seen times when an instrument or the bus becomes "locked".  The whole idea behind resetting the instruments via multithreading is to cut down on the execution time of resetting the test station.  I cannot seem to corner the problem here.  I do not know if the problem exists in the order of creating the threads, waiting for certain threads to complete, bus communication getting lost due to too many commands sent or commands being sent simultaneously .... etc.  Any suggestions to this problem would be greatly appreciated.  I also could supply more information if needed.

Thanks!  

[tmaxprt]

Hi

I experienced the same issue with CVI 7.1 and multi threading.  What version of CVI and what API do you use to perfrom instrument IO?  Maybe we are both experiencing a similar issue.  I use VISA  and  noticed from my SPY traces that it looks like a thread swap may occur  before ViPrintf  completes performing io (via ViWrite)

I suspect that the version of VISA and/or CVI has an issue with this. 

My original post: 

http://forums.ni.com/ni/board/message?board.id=180&thread.id=38029

[jr_2005]

This sounds like the classic case of simultaneous software access to some single underlying hardware component, with the inevitable result that the hardware operation becomes confused. Depending on the software you are using, you need to implement some form of resource lock in each of your threads, so that the critical core hardware control is only in use by one thread at a time. This could be via IVI_Lock()/IVI_Unlock(), or by using SDK Critical Section or Mutex techniques, for example. Check out "Interprocess Communication" in the SDK for more ideas.

JR

[jarc56]

I am using VISA as well, with CVI 8.1.  I also looked at the SPY traces and it leads me to believe that what JR is saying is true.  I am going to attempt the lock/unlock method and see if that helps.  The only problem here is that the reset subroutines create the sessions themselves and unfortunately i am bound to using those routines.  I know for sure that they do not lock sessions for instrument i/o.  If I find that locking the i/o sessions  works, I may can put in a change for the wrappers currently in use.  Again thanks for the help!

[menchar]

What type of HW interfaces are being used with the instruments?  Serial?  GPIB?  Ethernet?   PXI?  A mix of several types?

The underlying HW libraries from NI are all 'threadsafe" (e.g. serial library, TCP/IP library, GPIB library).  This generally means that any single i/o operation to/from an instrument won't get disrupted by threads competing for access to the interface / instrument, and i/o will automatically be interleaved by the I/O library.

If you are trying to do two different things that involve multiple i/o transactions on the same device, you can easily get crossed up despite the successful interleaving of the individual transactions. 

But this is all still vulnerable to the ordinary fatal-embrace / deadlock situations.  This involves having multiple threads requesting simultaneous use of multiple resources, and having the logic crossed up so that two threads both want a resource that the other thread has but won't release the resource they have.  So neither can progress and both get hung up.

As a previous poster stated, hardware state is hardware state:  multi-threading doesn't overcome fundamental issues of hardware state:  the hardware (any one instrument) doesn't know or care (to anthropomorphize a piece of electronics 😉  that you're commanding it from multiple threads - it is going to react to the sequence of commands it receives in the order it receives them.

In principle you should be able to command a reset serially by simply commanding each instrument to reset / initialize with a single write transaction and going on to the next instrument.  If it's GPIB, you can issue a universal device clear and get all connected devices to reset in response to a single command.

What  is your concept as to how you're going to save significant time by resetting instruments from multiple threads?  If a "reset" of an individual instrument requires multiple transactions, and if the instruments have effectively separate interfaces (or are separately addressed devices on a GPIB or some other parallel bus structure) then you should be able to overlap the resetting of each device and save some time, but this makes me wonder what you're doing that this savings would be worth the effort of constructing multiple threads simply to reset the instrument.

Having said all of this, I have designed systems that could do this, with a dedicated thread / processe managing all of the i/o for a device. 

Menchar