NI PCI-8430/8 Crashes in Vista x64 with a BSOD

[geneautomata]

Hi All,
I have and issue with a NI PCI 8430/8 serial card.

In our unnatended automation setup,  I control 5+ devices simultaneously using a serial port card.

I have been using a generic chinese pci card with 4 ports with no problem.

When I replaced it with the 8430, hoping for better performance, I get a BSOD when I try to operate all devices at once using labview.

I have searched the forums and the website but I find the documentation for the NI PCI-8430 very scarce, there is not even a proper user manual.
 
Evidence:

1) New NI PCI-8430/8 serial card

2) newly installed Vista x64 SP2

3) newly installed LV 8.6.1 from the Feb09 disks

4) new core i7 6GB ram computer

5) no warning signs in device manager or MAX

6) runs the controlling VI fine for about two minutes and then BSODs with error "IRQL_GT_ZERO_AT_SYSTEM_SERVICE"

7) computer has otherwise runned fine for a month with the same code with a generic chinese 4-port card with no reboots (24/7 automated robots).

It has to the the drivers or something like that - this is serious and we need to ix it or will have to return the card - big setback for us. The whole
point of spending $600(!) for somthing we were doing with $20 is more reliability !

🙂

Any advice will be appreciated, I have tried most of what I can think of, including reinstalling windows from scractch.

Thanks!

-ignacio 

[Steven_T]

Hello,

I think this is very concerning.  You are correct with your thinking that this may have something to do with the NI-Serial driver, however the blue screen never mentions any NI driver components.  We will need to troubleshoot in order to see if this is indeed a problem with NI-Serial.  I have a few questions for you.

1.  What version of NI-Serial do you have? (In MAX expand the Software tree item in the system tree, clicking on the NI-Serial item will give you the version information.  I'm expecting NI-Serial version 3.4 or 3.3.)

2.  Have you contacted NI Support via Phone or Email?

3.  Have you generated a Kernel Crash dump from this crash?

For item 3 follow this KB: http://digital.ni.com/public.nsf/allkb/581127525C80606A862570BE0003111D?OpenDocument

This file will likely be very large and you will have to provide it to us using our ftp server.  (ftp://ftp.ni.com/incoming)  Once it is uploaded to the server, we will have access to it, but we will need the exact version of NI-Serial you are using for it to be of much use.

To provide better troubleshooting steps and to get more information, I would recommend that you contact NI-Support.  Please reference this discussion forum and have the applications engineer contact me for additional steps.

If you are unable to get support please let me know.

Thanks,

Steven T.

[geneautomata]

Thanks Steven for your reponse! I have contacted NI for support and I am working with them to isolate the problem, but it is going a bit slow because they want me to work with my code to isolate the problem. But I know the code is fine because it has been working fine all along with cheaper hardware.  Attached is the MAX report, anything you can see? I will try to do a dump later today.

Thanks for your help,

-ignacio

 PS: From what I gather, drivers execute in the same space as the kernel so some exceptions will not explicitly mention the driver itself. You are right that a dump could be ultimately the only way to pinpoint the culprit. But circumstantial evidence already implies the card software implementation is the problem since I have been though a full reinstall and recreated the problem and the problem is not there if you use the exact same hardware/software ecosystem with one of the generic cards... if that makes sense.

[Steven_T]

Hello,

Thanks for the report.  It shows that you are using NI-Serial 3.4, which is currently the latest available. 

I look forward to getting the kernel dump when it is available.  If you can provide your service request number, I would be happy to work with the applications engineer to debug this issue.

Thanks,

Steven T.

[geneautomata]

Hi Steven,

Tried to generate a kernel dump, but I cannot find the file memory.dmp. I am under the impression that x64 dumps with 4gb+ of memory are not trivial. It is not clear from the MS site that they can be done. On a different note, I went to a store and bought a couple of generic PCIe serial cards (2-port each) and installed them and ran my software succesfully. So it is not my code I would guess. Something with concurrent access to the driver from multiple threads I think. Any ideas?

-ignacio 

[JasonSmith]

You can certainly generate dumps from Vista x64. A kernel memory dump does not include the full 4+ gigs of RAM, it will dump out just the kernel memory, which will be only a small portion of that. The one caveat I am aware of is that you will not be able to generate a memory dump if your virtual memory paging file is smaller than the size of the dump file. For a full memory dump this would mean you would need a 6 Gig paging file, but for a kernel memory dump it could be much smaller. The dump file by default will be placed at C:\Windows\MEMORY.DMP. You will know if it is generating the dump file because when it bluescreens it should tell you that it is generating a dump file along with the percent completion.

Sometimes Windows will tell you something like "Windows has recovered from a serious error" when you reboot the machine. When this happens, it should generally point you to the location of a minidump file--I believe they normally reside in C:\Windows\Minidump. While a minidump is not nearly as useful as a kernel memory dump, sometimes it is enough to spot the problem.

All that being said, not every bluescreen can generate a crash dump. The one you posted is not a "regular" bluescreen, in that it doesn't really have any debugging information such as the name of the driver responsible. This may indicate that it could be one that won't generate a dump file. 

We do test the exercising of all ports on the board simultaneously, but it is impossible for us to test the exact scenario you are using. In addition to Reads and Writes, what other aspects of the serial driver do you use? Examples would be anything you are doing with events or property nodes.

-Jason S. 

Message Edited by JasonS on 06-11-2009 08:42 PM

[JasonSmith]

Ignacio-

We just posted NI-Serial 3.5 to our driver download section. Would it be possible for you to install this and see if the problem persists? We have never encountered this issue in our testing, but PCI support in this version of the driver has been revamped dramatically, and I find it unlikely that you will still experience the same problem if this was due to NI-Serial.

-Jason

[geneautomata]

Great! Will try as soon as I get a chance. I noticed you dropped support for Windows Server 2003. Does the driver run on Windows Server 2008? Are there any plans to support Windows Server 2008?

-ignacio 

[JasonSmith]

Currently we are not doing any extensive testing with Server 2003 or Server 2008. Do you currently use either of these operating systems with NI-Serial products?

I would be interested to know the use case you have which would require you to use NI-Serial with one of the server operating systems, instead of Windows XP or Windows Vista. If you used them, would you be using 32-bit or 64-bit?

If server support is important to you, I would recommend that you take a few moments to file a product suggestion so that we can have your information on file for future reference.

-Jason S.

[geneautomata]

Thanks Jason for your reply! I have used Windows Server 2003 a lot in the past with NI products. The main reason being that I have assembled high speed control and data acquisition systems that required the speed and robustness of the server product, including better support for database interfaces and memory management. For example, Windows Server 2003 PAE support means we have been able use 16Gb of RAM for a solution where only 32-bit drivers were available for some of the cards (Dual Quad core 5420 Xeon + Magma 14-PCI slot backplane). Windows XP at the time just could not handle all the cards as well, nor the number of processors or the memory amount or the RAID hardware.

The main reason I am interested in using Windows 2008 Server these days is of course 64-bit support and maybe the possibility of using virtualization to isolate tasks and speed up development. Labview seems just too interwined with the OS and I would like to have LV running in virtual machines so I can swap computers in the future or test client-server implementations in our main production computer. Is NI considering supporting any aspect of virtualization?

I am not sure if this is correct, but, in my experience I have found server products to be a better platforms for data acquisition than a desktop OSs. Especially when big volumes of data are processed in realtime. Windows Server 2008 has very nice virtualization and I was about to do an install to test this.

-ignacio