How does VISA detect PCI/PXI interrupts (under the hood)?

[tavta]

Hello,

Does VISA poll the PXI devices in order to determine if a PXI interrupt occurs? This is what I suspect from my system's behaviour and I was unable to get any useful informationon on this matter online.

I am trying to configure interrupts from a third-party PXI card in a NI-PXI-1042Q crate with a NI PXI-8106 LabView RTOS controller using VISA.

I have succesfully made the card visible in MAX and included interrupt detection/discarding info in the .inf file for the card.

My situation is the following:

The card is able to interrupt if an "IRQ_enable" bit is set in one of its registers. Interrupts are emitted on the INT#A IRQ line. They can be triggered by software via asserting an "IRQ_on" bit on the card. You can clear the interrupt by deasserting any one of these two bits.

The behaviour I expect is the following:

- if the IRQ_enable bit is asserted, an interrupt will be emitted if I assert IRQ_on. (this works: one CPU core's total load and ISR goes to 100% and I can clear it by deasserting e.g. IRQ_on)

- upon interrupt assertion, VISA will check which device emitted the interrupt and notify me (I have a separate VI with VISA wait on event)

The problem is that I did not include the IRQ_enable bit in the interrupt detection sequence. Therefore, VISA detects an interrupt even if I just set the IRQ_on bit, with the IRQ_enable bit deasserted! Even though no actual interrupt occured, VISA notifies me of a PXI interrupt event.

When an actual interrupt occurs (the IRQ_enable was asserted prior to asserting IRQ_on), VISA succesfully clears the interrupt using the clearing sequence I specified in the .inf file.

Please, could someone explain this behaviour? What I find disturbing is that I did not expect any polling to take place on VISA's part - I expected it to only come alive when an actual interrupt occurs. 

Thank you in advance, 

Rok Tavčar

[ericg]

Hi,

Good question about how NI-VISA handles PCI-based interrupts under the hood.  Contrary to what may appear, NI-VISA will only respond to (and notify your application of) an interrupt when the interrupt actually asserts.  It does not poll any HW register.

NI-VISA relies on a user-specified interrupt handling sequence to service a device's PCI interrupt.  The interrupt sequence includes two major steps:

A. Determine if the device is interrupting (since PCI interrupts can be shared).

B. If the device is interrupting (i.e., step "A" returned "TRUE"), then squelch the interrupt.

For both steps, you specify a sequence of register operations, including reads, writes, and mask-and-compare operations.

Using these register sequences, which you specify using the NI-VISA Driver Development Wizard, when VISA loads, it will install a kernel-mode ISR with the OS for your device.  This ISR will only execute when the physical interrupt that the device is tied to interrupts.  This can happen because your device is interrupting, or because a device that's sharing the same physical interrupt line with your device is interrupting.

In your specific case, are you certain that your device isn't capable of asserting an interrupt, even when the IRQ_enable bit is not set?  It should not be possible for VISA to execute your interrupt handler (and notify your application via Wait on Event) unless your device is physically interrupting.

What do your interrupt handling sequences (specified with the Driver Development Wizard) look like?

Thanks,

Eric G.

National Instruments R&D

[Robert__R]

Another comment on one point Eric mentioned: PCI interrupts are shared.  This means that the interrupt handler is being called whenever anything on the same interrupt line asserts its interrupt.  Since you didn't include the IRQ_enable in the interrupt detection sequence, the interrupt handler will incorrectly determine that your board is asserting an interrupt.

- Robert

[tavta]

Hi Eric, thank you for your reply!

I'm new to interrupts and the information I find online is somehow overwhelming, making it hard to find concise info for a specific setup.

---

Using these register sequences, which you specify using the NI-VISA Driver Development Wizard, when VISA loads, it will install a kernel-mode ISR with the OS for your device. 

---

This is particularly valuable to me as I was not sure where the ISR comes into play and what sits on the lowest layer of interrupt detection.

---

..... This can happen because your device is interrupting, or because a device that's sharing the same physical interrupt line with your device is interrupting.

In your specific case, are you certain that your device isn't capable of asserting an interrupt, even when the IRQ_enable bit is not set?  It should not be possible for VISA to execute your interrupt handler (and notify your application via Wait on Event) unless your device is physically interrupting.

---

I had a FlexRIO module on the bus, but the "interrupts" are detected by VISA even after I removed it from the machine and my card is the only one on the bus.

What makes me think that the device isn't asserting interrupts without the IRQ_enable bit is the following:

- if I use no "wait for event" VI, and trigger an interrupt with IRQ_on while IRQ_enable is asserted, one CPU core and its corresponding ISR go to 100%, indicating that a hardware interrupt occured and nothing handled it.

- if IRQ_enable bit is 0, nothing happens to ISR or CPU load when IRQ_on is asserted.

---

What do your interrupt handling sequences (specified with the Driver Development Wizard) look like?

---

Everything about controlling interrupts from the card involves the same 8-bit hardware register (INT_CTRL) at BAR0, offset 0x4c:

My (one) Interrupt detection sequence only has one step: 

Compare a byte in BAR0 at offset 0x4C. Mask = 0x80, value must be 0x80. This is bit7 that gets asserted by software and triggers an interrupt if bit6 of the same register is asserted. I do not check the value of bit6 in any step. (I do not deem it necessary)

Interrupt Removal:

Masked R/W. 8bit register in BAR0 at offset 0x4c. Mask: 0x7f. This should deassert bit7.

Interrupt disarm: 

Masked R/W - same as above with mask 0xBF, clearing bit6.

--------

When a real interrupt is generated by the card and handled by VISA, it seems only the "Interrupt disarm" sequence takes place, because bit7 stays asserted after VISA handles the event. Under what condition does the Interrupt removal sequence happen?

+ another question: The VISA VIs such as "discard event" and "disable event" are on a level higher than ISR, right? They do not use any information determined in the .inf file? I am quite sure, but I would like a confirmation:)

Cheers, 

Rok

[tavta]

@Eric: Thank you for your comment... I now see that including IRQ_enable will be necessary in the case when multiple devices are connected on the bus. But in the case of a single device, I would expect that interrupt sharing shouldn't pose a problem. Can things happen on the IRQ lines that do not originate from the PXI devices?

[tavta]

Robert sorry, my last post was ment for you, not Eric...

I still don't understand why VISA executes the interrupt detection sequence even though no actual HW interrupt occured. If I assert bit7 (of Interrupt control register on the PXI card) and bit6 is 0 (PXI interrupts are disabled), no interrupt is actually generated, but VISA still detects that bit7 was asserted (i only check bit7 in the interrupt detection sequence).

What exactly happens when "VISA enable event" VI is called? Does VISA write anything to the PXI card? Or any other LV VIs - do they use the sequences, defined in the device's .inf file?

The card uses the PLX 9030 pci bridge.

I hope you have some ideas, I'm in the dark here... I really want to know what is going on the lowest level in VISA...

Regards, Rok

[Robert__R · Accepted solution]

Rok,

Boards don't need to be in the same chassis to share an interrupt.  The legacy interrupts (Int A, B, C, D) are often shared across all PCI/PCIe devices in a system, so onboard devices on your controller may use the same interrupt line.  Because the interrupt lines are swizzled/rotated at each slot, try your device in 4 adjacent slots (i.e. slot 2, 3, 4, 5) to find one that isn't sharing an interrupt with something else**.  It's likely that at least one of the slots will be alone on an interrupt and you can see VISA react in a way that makes more sense to you.

I've seen cases like this where nothing in the chassis is interrupting, but ISRs are being called for them because of something in the host PC was interrupting (I use a host PC with MXI).  It can be surprising, but it's normal.

I don't know about the VISA end of it, but here's how I think it works.  Since you don't tell VISA how to enable the interrupt, it doesn't know how to.  Therefore it's up to you to enable the interrupt after you call "VISA enable event".  VISA will squelch the interrupt when it comes and notify you so you can properly deal with it, then you call "VISA enable event" again and rearm the interrupt.

** This works because an 8 slot chassis doesn't have bridges, and the interrupts will go through a full rotation in 4 adjacent slots.  Trying your board in those 4 slots assures that you've tried all interrupt lines available to you.  Larger chassis have PCI-PCI bridges to join segments, and the bridge may cause an extra rotation between a pair of slots.  Also, this is not intended as a fix for your problem, just as an aid in understanding interrupts and how VISA handles them.

- Robert

[tavta]

Hey Robert, thanks! 

This explains why VISA sees "phantom" interrupts. Now I can implement proper IRQ detection sequences and sleep calm at night.

[ericg]

---

My (one) Interrupt detection sequence only has one step: 

Compare a byte in BAR0 at offset 0x4C. Mask = 0x80, value must be 0x80. This is bit7 that gets asserted by software and triggers an interrupt if bit6 of the same register is asserted. I do not check the value of bit6 in any step. (I do not deem it necessary)

Interrupt Removal:

Masked R/W. 8bit register in BAR0 at offset 0x4c. Mask: 0x7f. This should deassert bit7.

Interrupt disarm: 

Masked R/W - same as above with mask 0xBF, clearing bit6.

---

Rok,

Just a couple of final comments on your interrupt detection sequence.   In general, NI-VISA will NOT notify you of an event for your device (through an event handler callback or the event queue) unless your interrupt detection sequence returns TRUE.  So, your interrupt detection sequence (and the hardware that it supports) should be designed to not return TRUE unless your device is actually interrupting.  This should eliminate "phantom" interrupts that result from your device sharing an interrupt with another PCI device.  In the shared IRQ case, even though the OS may be calling your device's ISR under the hood, your interrupt detection sequence should return FALSE if your device isn't interrupting, and NI-VISA will ignore the interrupt and not notify your API-level event handlers.

If you are seeing interrupts occur in your application even when your device is not interrupting, you may want to verify your interrupt detection sequence.  Also, if your hardware does not support a "I am interrupting" bit, you may want to consider adding this feature.

Hope this helps,

Eric G.

[tavta]

Hi Eric,

I realize it now. It is possible to find out  whether an interrupt is active or not by reading some bits in a PLX pci bridge's register. As we have control over the card's firmware (onboard FPGA), we'll also define IRQ IDs to show the interrupt source, if needed. In such a case we'll define multiple interrupt detection sequences for VISA, which will check 1. if the device is interruptding and 2. what is the source of the interrupt. As I understand, it is possible to find out which sequence returned TRUE from the VISA wait for event VI.

Thanks again for your support,

Rok