Just one more tidbit...after analyzing another group of acquisitions it is clear that:
1. Consistent data sample dropouts (new samples either not taken or placed in read buffer) for 150 microsec, after which samples resumed. All channels in the chassis are affected.
2. All 3 clients scan exhibit this behavior, usually just one/acq but sometines 2.
3. On one occasion intra-chassis channels were in phase while inter-chassis channels were out of phase, but none exhibited dropouts.
4. About 50% of acquisitions were perfectly good.
5. I'm thinking either sync or sample clock breakdown. Or DAQmx issues. Or maybe the NI-sync connections. Or mixing in DAQmx timing subVIs with NI-sync. Or...
Thanks for the additional detail on this issue. Based on these concerns, I have several ideas for troubleshooting or resolving this issue.
Here is a knowledgeBase (KB) article with troubleshooting steps for the 89130 DAQmx error. Along with this KnowledgeBase, I would recommend updating DAQmx and NI-Sync to the latest possible versions (9.6 and 3.3.5 respectively) as well as following the troubleshooting steps in the KB.
DAQmx Error Code -89130: Device is not Available for Routing:
Also, there is a KnowledgeBase regarding the CLKIN port of the 6553 to a PXI chassis that may be useful.
Routing CLKIN of PXI-6653 to PXI_CLK10_IN of PXI Chassis:
There is an example VI that is very similar to what you are attempted to create, I would like to try it and see how the acquisition responds. Please run this VI, and in particular look at steps 16-20 on the block diagram where the terminals and clocks are disconnected. You may have this example VI already, but I have also attached it for easy use.
Share PXI_CLK10 and Synchronous Trigger Between Chassis
Finally I would like to clarify the current setup with the pxi chassis, could you describe the hierarchy of pxi chassis with the cards installed. At first I thought that the host computer was attached to the main controller chassis (a 1045) that connected to three client pxi's that are 1033 chassis, but your previous post made me unsure.
As a last note, I will be bringing in another Applications Engineer to help work on this issue with us in order to bring this issue to resolution quickly.
I've attached a png file of the basic layout, that will hopefully assist you guys in visualizing the system.
I had read the KB article about the 89130 error fix that lead me to the device resets that you later had not recommended. I thought resets were for transitioning back to DAQmx from using Traditional NI-DAQ driver sets?
I'm not sure the PXI-1033 box we have has the backplane switch mentioned in the routing CLKIN article...we are using the OCXO oscillator on the 6653 for CLKIN10.
The latest DAQmx version (I think) that the clients (WinXP, LV8.5, MAX 4.7.1) can use is the current installation (9.2.2). I had tried to install the same version (9.4) that the controller (Win7 64-bit, LV 2011, MAX 5.0.0) uses but it would not install on the XP clients. There was a table on your website that listed all the compatibilites between DAQmx versions and Windows OSs but i can't find it now.
I'm incorporating the terminal disconnect/session closing shown in your example. Do you think there is some collision between unclosed sessions/routings and current requests when the VI is launched?
Added assistance is always a help to accelerate solutions, so a second set of eyes is welcomed...I've really got to get these boxes humming!
A quick question about how to close old session (niSync) handles. I thought LV (running 8.5 on clients) closed old handles when LV was closed. Why do I see the instrument handle name with a (I guess) session number in parenthesis? I take this to mean the previous session did not close properly. Can't find any info on this in the discussion forum or KB. Thanks Joel
If you see an instrument handle name staying around after a VI has been run, it means that niSync Close.vi was not called for that session. The niSync Initialize VI creates an NI-Sync session and then calls IVI New Session.vi which associates the resource name with the NI-Sync session and returns an IVI session handle. The niSync Close VI destroys the NI-Sync session and deletes the IVI session. When a VI is aborted or stops with an active NI-Sync session, the NI-Sync session gets cleaned up, but the IVI session will remain.
Just as a side note: When you create multiple sessions to an NI-Sync device, the number in parenthesis of instrument handle name is used to make each name unique and does not necessarily reflect the session number.
I took a look at the G550_ClkTrig_config4.vi you posted. The following are a few suggestions to help out with the synchronization:
1. When you route PFI0 to PFI[1..3], you should remove the synchronization clock input to the niSync Connect Trigger Terminals VI. By default, NI-Sync creates asynchronous routes which are preferred for distributing clocks. Synchronous routes were designed to align triggers to a rising or falling edge of a clock and only update the output terminal on the specified clock edge. When using the DDS as the synchronization clock source for the terminals, the 10Mhz clock wired to PFI0 will be aliased by the DDS frequency on the output terminals PFI[1..3].
2. As for the start trigger, I would suggest that you make the paths to the DAQ devices symmetric. Each asynchronous route through a 665x device generally delays a signal in the order of 10s of nanoseconds. To better align the trigger to the client chassis, keep the software trigger route to the backplane, but replace the software trigger route to PFI5 on the 6653 with a route from the backplane to PFI5. For now, I would make the routes from PXI_Trig1 to the PFIs asynchronous.
3. I am not sure how well you need the chassis synchronized to each other. Though to achieve nanosecond level of synchronization, you will want to match path delays for clocks and triggers. For example, by routing the PXI-6653s OCXO to PXI_Clk10_In and then exporting PXI_Clk10 to the client chassis, the chassis will have a phased locked 10Mhz clock but not phase aligned. To ensure that your clocks are both phase locked and phased aligned, you can make the following changes:
a. On the PXI-6653, route OCXO to ClkOut
b. Wire ClkOut to a 4-way splitter
c. Wire the outputs of the splitter to ClkIn of each system timing module in a system timing slot (including the PXI-6653). Make sure each wire is matched trace length.
d. Then route ClkIn to PXI_Clk10_In
Similar techniques can be applied to the trigger and sample clock routes that don’t necessarily require an external splitter.
I hope this helps,
Your suggestions made good sense so I changed the code and cabling to what (I think) you were recommending. Although not specifically stated, I also disconnected all synch clock sources for all trigger connections, both server and client connections. Attached are the VI, diagram of the system as it now exists, and read block error. I got the following results:
1. The master synch clock (OCXO) is measured at 20 MHz (was 10MHz) at PFI0 on 6653).
2. Client 1 measured sample clock was 120 khz (should be 250 kHz according to DDS settings).
3. Client 2 measured sample clock was 250 khz as it should be.
4. I still get the same data discontinuity after read block #1 in recorded data, but it maybe app related instead of clocking error...
Hopefully I have made some block-headed error that you will immediately spot...
I noticed in the previous VI version I had left the synch clock inputs wired on the 2 1033 slave 6651s...these have been disconnected and also on the client case were the trigger was wired. The results are the same...