Buffered event counting. Why can't I explicitly sequence generating the Sample Clock Pulse and reading the counters?

[Kevin_Price]

cebailey,

I modified your latest vi's a little bit by removing all the Wait calls (more on this later) and maintaining the error data throughout.  I only had a few minutes and am not currently at hardware so there's a chance there's some small problem, but hopefully not.  Now, addressing some of the questions / comments:

A. Who am I and Why am I here? 
   Just a guy who's happened to do a fair amount of data acq with LabVIEW and NI boards, with more emphasis on counters than most.  I learn from and like to contribute to the forums (I learn by thinking through answers too, after all).  And I come from a family with a lot of teachers.  That's about it.

B. Timeout on restarts.
   Your first posting about this stuff was in an infinite loop.  The only way to stop and restart was to use the "Abort Execution" stop sign menu button, which is generally a bad practice.  In recent versions, LabVIEW has gotten much better at automatically cleaning up the mess that is left behind, but it can take some time to do its thing behind the scenes.  The only way I saw timeouts was to restart immediately after an "Abort".  I figure LV hadn't had time enough to clean up yet.  Odds are, one of the DAQmx calls generated an error that was ignored because the error data was left unwired.
   Your more recent posting puts an explicit Stop and Clear in the outer loop, so the next iteration of the loop can't start until all the housekeeping associated with Clearing a task has completed.  So it's not surprising that the restart on the next iteration is always succeeding.

C. Error data & sequencing
   I *do* get your point.  But I think the "common wisdom" about using error wires for sequencing whenever possible is still a good one.  The reasons why are often left unstated but go something like this:  the error wires have an important purpose, and should almost always be wired and handled.  Error handling is a key part of any significant application.   Well, since I've got to deal with those error wires anyway, and since I typically have need to provide explicit sequencing in some places, I can use the error wires to enforce sequence and kill 2 birds with one stone.  The same wire routing is both a preferred way to minimize the # of separate error threads AND is a useful way to provide sequencing.
   I think the general perception of sequence structures is that they are a large mallet.  There's *usually* a different, and frequently better (more flexible, less diagram clutter, less redundancy) tool for the job.

  It really is important to handle your errors though.  You have probably missed a lot of clues during your troubleshooting by ignoring them. 

D. Sequence structures for numeric calcs.
   It's at least 99.99% likely that your observation has a different explanation.  Those numeric operations simply DO NOT have the kind of dataflow issues that you thought were responsible.  Not trying to scold you or anything, but just trying to deal with issues at the most basic level.  I know how a difficult troubleshooting problem can lead to trying all sorts of ideas, including many that you don't think *should* work but it's all you can think to try at the moment.

E. Modified vi's
   I removed all the 1 msec Waits.  If other aspects of your system's response needs these delays, that's (to me) a whole other issue that I can't really help with.  You can work on that later.  What I *can* try to help with is the behavior of the data buffering of the counters while generating a "clock" signal with a DO line.  And at least for *that* aspect of operation, the delays absolutely should not be needed.  My earlier examples worked flawlessly for me without them on my PCI hardware.  So at least now that you're on a PXI system (got my doubts about your USB hardware), let's stick with basics.
   I also added error chaining through all the DAQmx calls.  One subtle thing I did was to generate a False on the DIO before starting the counter tasks.  Just in case the DIO line glitches somehow when first configured for output.  This way, the counters aren't yet paying attention if/when such a glitch occurs.

-Kevin P.

[Crystal]

Hi, Chris

Once again, {I'm blaming the 3 hour time difference between us} Kevin has beaten me to the punch.  I completely agree with everything he's said.  I can only apologize for anything the Basics courses taught that we are disagreeing with.  I can guarantee that when I wrote and taught the classes that you would have gotten more thorough information with practical examples of all the points Kevin made.  I can also recommend the books "LabVIEW for Everyone" by Jim Kring and "The LabVIEW Style Book" by Peter Blume.  They both go into the descriptions of sequences and error handling in much more detail where your issues are concerned. 

I also hope it doesn't sound like Kevin and I are being overly critical of your code.  We were both new at LV at some point, too, and  made all the same mistakes (at least I did).  It took a while before I made the complete paradigm shift from conventional programming to graphical dataflow programming. 

Stick with it;  I swear this stuff works and works well.

[cebailey]

Thanks, both of you!

Kevin, I remain awed by your participation. I'll try the VIs on the PXI system when I can.

Crystal, I'm using King's book (3rd ed.), and have found almost nothing about error flow sequencing. The only relevant line I've found so far is: "When you add error in and error out I/O terminals to your VIs, you allow calling VIs the opportunity to chain the error cluster wire to create dataflow dependencies between subVIs" on P. 295 in the chapter on Arrays and Clusters. This is minor but relevant, yet doesn't come close to Kevin's quite reasonable explanation. I looked through the entire chapter on Sequences and didn't find anything suggesting wiring errors instead. About dataflow, I find only "Its execution is based on the principle of dataflow, in which functions execute only after receving the necessary data" on P. 5 (the only index entry for "dataflow"). Where's the "much more detail"? I don't have Blume. I DO hope that if the error flow trick really is good practice, I'll start to see it as such!

>...sound like Kevin and I are being overly critical of your code...   ---   No, no, of course not. I think at this point you're only critical of me ignoring errors, in which you're perfectly correct, and the error flow sequencing, which for good or ill I still see as debatable. You have criticized some things which I shouldn't have to be doing, of course, but the whole discussion includes the question of why these seem to fix or change behavior when they shouldn't be needed at all.

>I swear this stuff works and works well.   ----   I dunno, I dunno. I'm still waiting to experience that. Surely I'm trying hard enough, right?

[Crystal]

Hi, Chris

Yes, I'm going to have to apologize again.  Kring's book is a great reference, but it doesn't actually go into many details.  That's probably a good thing because it's already over 1000 pages.  It is a good place to start, but doesn't tell the whole story.  Blume's book has an entire chapter devoted to Error Handling.  He starts out by recommending you trap all errors by wiring all the error terminals.  The he goes into how trapping isn't enough and how you *really* only need to trap certain errors such as any sort of I/O.  What you have to do is find efficient ways to report errors and/or handle them in the method that works best.  For instance, receiving the timeout on an early DAQmx VI pretty much obsoletes the rest of your diagram, so the program should probably immediately report that error and either attempt the DAQ operation again or just end depending upon whether you are actually monitoring the app or if it's running unattended.  He recommends not using error dialogs if the app is unattended, but to programmatically prioritize the errors and either clear them out and try the operation again, or send the messages to an error log.  You can even have the program e-mail the error log to an admin or page people should certain high-priority errors occur.  Most of Chapter 4 is devoted to dataflow and showing how sequences are unnecessary in some situations versus others.  He talks through about ten examples and you can see how the programs are much easier to read, understand, and debug based on different programming architectures. 

You are definitely trying enough, in fact, you might be trying too hard.  I've had periods of massive frustration over a seemingly simple problem and eventually they work themselves out.  Dataflow is an extremely simple concept but changing the way you program from a sequential text-based way of thinking to a parallel dataflow way can be painful and difficult.  It took me a long time while I've seen some people immediately get it after finishing their first application.  The more you use LV on real-world problems, the more it will make sense.  It really helps to use this forum as you did.  Talking through problems/issues with other people who have been there and done that is extremely helpful. 

Kevin, you completely rock!  You are highly appreciated here!

Another good resource for LV programming is www.lavag.org.  They have some great wiki articles on many topics.  

 Of course, I'm available as a resource any time.  Just give me a holler if you want me to come out for a visit or anything.  I plan to stop by the next time I'm back East just to meet you face to face and see what all you are working on.

[Kevin_Price]

Had a chance to try out my most recent vi's on a PCI-6259.  At first I *did* see a timeout error -- but it turned out to be that I forget to use the "right" DIO line for the pulse.  As soon as I set it to port1/line0, no more timeouts ever.  (I've got the terminal block completely disconnected from my board for my testing.  Using port1/line0 gives me an internal signal connection to PFI0, removing all dependence on external wiring.)  Also due to my lack of external wiring, the other thing I needed to do was define the 100 kHz timebase as the edge count terminal using a DAQmx Channel property node.  You can copy and paste from one of my earlier posted examples.

Could you test this way on your PXI?  The code you post keeps reverting back to port0/line1 and using the default Source pin as the terminal for edge counting.  I'm not convinced (yet) that your external wiring and connections can be eliminated as suspects.  So could you remove the cable from the board to take all the external wiring out of the picture, then do the things I described above?  My board ran for something like 12000 starts at 100 randomly-timed reads per start.  That's over a million total counter reads with no timeouts.  Yours should do this too.  I'm still not sure you've ever 100% confirmed that you tried the USB board with all wiring disconnected.

I really do understand why you keep wanting to test with something more like your full-up real system.  After all, if it works, you're all done.  But in the long run, you'll benefit about 90% of the time by resisting that temptation.  Break the problem down into small, non-interacting, easily-observed pieces.  It always takes extra time to do this.  But it usually takes *less* extra time than if you didn't.  Right now the core issue is to force count values to get latched & buffered immediately whenever you happen to generate a True-False strobing pulse on a digital output.  If you run an example that's been proven to work in the absence of all external wires, you'll be helping to zero in on the root cause.

-Kevin P.

[cebailey]

Kevin, I'm trying the VIs you posted most recently.

>use the "right" DIO line for the pulse ... port1/line0   ----   Yes, I changed this.

>define the 100 kHz timebase ... Channel property node.  You can copy and paste from one of my earlier posted examples  --- Yes, I copied this from your "query numsamps" VI.

>Could you test this way on your PXI?   --- Yes, I will do that next, with the cable disconnected, if the system is available. The PXI system = the full up hardware system = "Multi".

I did just test it on my USB system = my development system = "Laplace", after removing all the physical wires. It immediately gives the timeout error. I am posting that here as "...LaplaceUSB.vi". Do you see anything I missed?

>...not sure you've ever 100% confirmed that you tried the USB board with all wiring disconnected...   ---   No, I didn't try USB without wiring until minutes ago, but I have now, and it timed out immediately, as per above.

>I'm not convinced (yet) that your external wiring and connections can be eliminated as suspects.  ---  This is reasonable. I'll be doing internally-wired-only tests with the cables off for the time being.

>I really do understand why you keep wanting to test with something more like your full-up real system.   ---   Hah! I think you don't! I was using external physical wiring out of ignorance  because I'm unfamiliar with the internal routing with PFI lines and felt more secure with the external physical wiring. I didn't have a useful feel yet for what sorts of things to try to do that way, for example it didn't occur to me to use Port 1 for the digital out so it can map to a PFI line to the source input. But I'm getting more comfortable with the PFI business now.

>core issue is to force count values to get latched & buffered immediately whenever you happen to generate a True-False strobing pulse on a digital output    ----   Exactly. Exactly. Exactly.

[cebailey]

Kevin, YOU GOT ONE!

Your posting which I edited per your advice and just reposted, now works on the PXI system (after "Laplace*" is replaced with "Multi*")! I watched it run 500 start loops.

This got me wondering about Distribution...Multi which was running before on PXI. You may recall I didn't trust it because I had to add an extra pulse before starting the loop and was worried there was an uncertainty of 1 in the vintage of the count results. A few days ago I posted "Then I discovered that about 15% of the time, it dies on the first iteration when starting, with a timeout, at random as far as I noticed. I thought to add a pulse on the digital line before the loop starts in the calling, test program Distribution....vi." when I introduced this change.

Maybe the state of the digital line was indeterminate in my Distribution, either electrically or depending on how I had driven it during a previous test... until I added the pulse, which also fixed the indeterminacy. Your recent posting specifically asserts the line low, which I didn't do before (looking back this seems very bad practice)! So I changed the older Distribution...Multi.vi to drive the line low twice, instead of high & low, and it still worked. The pulse I had concluded was fixing things wasn't really the difference that fixed things, but explicitly setting the state of the line may have been.

I also tried taking both those drives out, hoping to break it, but that did not break it (however proving this was the problem also relies on forcing the line to take an indeterminate state, and who knows what history may have been doing that - there's no fuzzy Boolean constants available in LV). Maybe I'll add something to randomly set the line high or low before the loop; I would expect that to break it.

So it's gonna run over the weekend while I tally diff's and starts.

I don't see anything worrisome or indeterminate about the VI now.

I am still curious why it didn't work on the development system, which is way easier to get on while maintaining file and Web access. There's a NI field engineer visiting next week, so maybe he can put his hands on it and figure something out.

And I think I have to add Duplicate Count Prevention to this, because my event rates can sometimes be much lower than my sampling rates, right?

Thanks!!

[Kevin_Price]

   Your recent post with LaplaceUSB in the name worked flawlessly on my PCI-6259 after adjusting only the device name.  Hundreds of loops, no timeouts.  Hang onto these various examples so you can go through things with the NI field engineer next week.  I've had very little exposure to the USB boards but I find it hard to believe that your timeouts should be considered expected behavior.  Something seems mighty screwy there.

   Glad the PXI is finally showing expected behavior.  Hope that continues after you wire your real equipment back up again.

   Re: Duplicate Count Prevention.  Over at my end, DAQmx sets this up appropriately by default.  In my last post, after changing the port1/line0 thingy but before specifying the 100 kHz internal timebase, the code looped repetitively with no errors but every latched and buffered counter value was identically 0.   (Since I've got nothing wired, there were no Source edges).  This indicates that Duplicate Count Prevention (a *very* unintuitive bit of terminology, in my opinion) behaves the way you need it to.  The other mode would essentially ignore future sample clock signals until a Source edge came in to enable the circuitry somehow.

   I guess I never asked before, but it may matter what DAQmx version you're running.  The default settings and behavior for Duplicate Count Prevention has varied at 2 or 3 different versions of DAQmx and for different boards.  Because of that, I don't even try to remember the right setting for different situations -- I just run test cases to make sure I get the right behavior.  I *think* it's been stable and consistent since 7.4 or thereabouts, probably about 2 years ago.   I'm running 8.3 on the system I used for testing.

  A quick check you can do is to remove that property node that references the 100 kHz timebase on one of the counter tasks.  If DCP isn't set right, then you could get timeouts for the counter that no longer counts the 100 kHz signals.  If it doesn't timeout, and it just keeps returning 0 values as fast and as often as you strobe, then you're OK.  But to be on the safe side and guard against changing DAQmx versions, maybe it's best to explicitly set the DPC property to the correct Boolean value.   I don't recall whether the correct one is "True" or "False", but you can find out quick enough by trying each.

-Kevin P.

[cebailey]

>This indicates that Duplicate Count Prevention (a *very* unintuitive bit of terminology, in my opinion) behaves the way you need it to.   ----   I plan to try counting transitions on a digital output line that I also control, and to create a vi that generates a somewhat random distribution of counts that includes 0, and tests whether the counter detects the number that the vi knows it created. The terminology, I take it, suggests that this mode prevents re-reading or duplicating the last nonzero count during what should register as new zero counts. Since I think all counters should be able to count to zero, I think the mode should have been termed Not Messed Up Mode. I do see that the counters can count to 8, even though we did not have to specify Counting To Eight Still Works Mode.

[cebailey]

Kevin,

I'm running DAQmx version 8.1 on the PC that uses the USB system Laplace. I think the last update was about a year ago but couldn't be more than 2 years. The PC using PXi Multi very likely has a newer DAQmx. It is kept pretty nearly up to date. I think we installed new versions of all NI software there this summer.

On the PXI system I have written another testing VI that calls on ReadCountersOnce. It generates a known number of pulses in an inner loop using another, new digital output port, which is coupled by PFI into both counter source terminals, so buffering and events are both their own digital output bits. I have yet to get an incorrect count or timeout on either counter, in many reps, even with 0 as the number of pulses in isolated or consecutive counts. This seems to me to indicate that Duplicate Count Prevention is either in play or unneeded, though I did nothing to specify its use.

There's some tidying up I'd like to do, checking all my versions and collecting the most interesting VIs and fixing two off-point problems with this last VI and getting things together for Friday's visit. But it seems to me my problem is fixed, or rather nonexistant, for the full-up hardware system, and I could create a dummy counting VI for the development system if need be. Maybe I'll post some of this wrapup.

Kevin, thanks tremendously for helping figure this out! By chance would you be willing to pass me your email address? Mine's visible in my profile. In any case, I'm much in your debt - Thank you!