Skip to contentSkip to conversation

PCI,pciehp: Don't handle PDC for cards with attention button

PATCH In review Yinghai Lu 3 participants 5 messages Feb 17, 2017 PCI

lore
Share & tools
lore

Yinghai Lu

author
#0 lore raw

PATCH 1/1 PCI,pciehp: Don't handle PDC for cards with attention button

On one system during power off slot, find card get power on right away.

 # echo 0 > /sys/bus/pci/slots/1/power
 pci_hotplug: power_write_file: power = 0
 pciehp 0000:16:00.0:pcie004: pciehp_get_power_status: SLOTCTRL a8 value read 11f1
 pciehp 0000:16:00.0:pcie004: pciehp_unconfigure_device: domain:bus:dev = 0000:17:00
 pci 0000:17:00.0: PME# disabled
 pci 0000:17:00.0: freeing pci_dev info
 pciehp 0000:16:00.0:pcie004: pending interrupts 0x0010 from Slot Status
 pciehp 0000:16:00.0:pcie004: pciehp_power_off_slot: SLOTCTRL a8 write cmd 400
 pciehp 0000:16:00.0:pcie004: pending interrupts 0x0108 from Slot Status
 pciehp 0000:16:00.0:pcie004: Slot(1): Link Down
 pciehp 0000:16:00.0:pcie004: Slot(1): Link Down event ignored; already powering off
 pciehp 0000:16:00.0:pcie004: pciehp_green_led_off: SLOTCTRL a8 write cmd 300
 pciehp 0000:16:00.0:pcie004: pending interrupts 0x0018 from Slot Status  <======
 pciehp 0000:16:00.0:pcie004: Slot(1): Card present
 pciehp 0000:16:00.0:pcie004: pciehp_get_power_status: SLOTCTRL a8 value read 17f1
 pciehp 0000:16:00.0:pcie004: pending interrupts 0x0010 from Slot Status
 pciehp 0000:16:00.0:pcie004: pciehp_power_on_slot: SLOTCTRL a8 write cmd 0
 pciehp 0000:16:00.0:pcie004: pciehp_green_led_blink: SLOTCTRL a8 write cmd 200
 pciehp 0000:16:00.0:pcie004: pending interrupts 0x0010 from Slot Status
 pciehp 0000:16:00.0:pcie004: pciehp_check_link_active: lnk_status = f103
 pciehp 0000:16:00.0:pcie004: pending interrupts 0x0108 from Slot Status
 pciehp 0000:16:00.0:pcie004: Slot(1): Link Up
...

Somehow PDC bit get set, and during handling interrupt that is caused by CC, that PDC also get handled, and the card get powered on again.

In pcie_enable_notification(), we already only enable notification for PDC when attention button is not there.
So we can safely add checking in pciehp_isr() to skip that PDC handling.

---
 drivers/pci/hotplug/pciehp_hpc.c |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
drivers/pci/hotplug/pciehp_hpc.c +3 -2
Index: linux-2.6/drivers/pci/hotplug/pciehp_hpc.c
===================================================================
@@ -618,8 +618,9 @@ static irqreturn_t pciehp_isr(int irq, v
 		present = !!(status & PCI_EXP_SLTSTA_PDS);
 		ctrl_info(ctrl, "Slot(%s): Card %spresent\n", slot_name(slot),
 			  present ? "" : "not ");
-		pciehp_queue_interrupt_event(slot, present ? INT_PRESENCE_ON :
-					     INT_PRESENCE_OFF);
+		if (!ATTN_BUTTN(ctrl))
+			pciehp_queue_interrupt_event(slot, present ?
+					INT_PRESENCE_ON : INT_PRESENCE_OFF);
 	}
 
 	/* Check Power Fault Detected */
#1 lore raw

Hi Yinghai

Which version of linux did you apply this?

I'm not sure if you can ignore PDC when ATTN isn't present. Surprise hot-add on systems with Power Control would depend on PDC.

There is some new code to deal with both Presence detect and DLLSC events since 4.10-rc1.

drivers/pci/hotplug/pciehp_hpc.c static irqreturn_t pciehp_isr(int irq, v · line 626

Quoted 11 lines · Yinghai Lu
---
 drivers/pci/hotplug/pciehp_hpc.c |    5 +++--
drivers/pci/hotplug/pciehp_hpc.c +3 -0
+		if (!ATTN_BUTTN(ctrl))
+			pciehp_queue_interrupt_event(slot, present ?
+					INT_PRESENCE_ON : INT_PRESENCE_OFF);
 	}
 
 	/* Check Power Fault Detected */

This is what we have since v4.10-rc1

    /*
     * Check Link Status Changed at higher precedence than Presence
     * Detect Changed.  The PDS value may be set to "card present" from
     * out-of-band detection, which may be in conflict with a Link Down
     * and cause the wrong event to queue.
     */
    if (events & PCI_EXP_SLTSTA_DLLSC) {
        ctrl_info(ctrl, "Slot(%s): Link %s\n", slot_name(slot),
              link ? "Up" : "Down");
        pciehp_queue_interrupt_event(slot, link ? INT_LINK_UP :
                         INT_LINK_DOWN);
    } else if (events & PCI_EXP_SLTSTA_PDC) {
        present = !!(status & PCI_EXP_SLTSTA_PDS);
        ctrl_info(ctrl, "Slot(%s): Card %spresent\n", slot_name(slot),
              present ? "" : "not ");
        pciehp_queue_interrupt_event(slot, present ? INT_PRESENCE_ON :
                         INT_PRESENCE_OFF);
    }

Cheers
Ashok

author
#2 lore raw
Quoted 3 lines · Ashok

Hi Yinghai

Which version of linux did you apply this?

current linus tree + pci/next

Quoted 3 lines

I'm not sure if you can ignore PDC when ATTN isn't present. Surprise hot-add on systems with Power Control would depend on PDC.

in pcie_enable_notification() we don't enable PDCE when ATTN is not there.

        cmd = PCI_EXP_SLTCTL_DLLSCE;
        if (ATTN_BUTTN(ctrl))
                cmd |= PCI_EXP_SLTCTL_ABPE;
        else
                cmd |= PCI_EXP_SLTCTL_PDCE;
        if (!pciehp_poll_mode)
                cmd |= PCI_EXP_SLTCTL_HPIE | PCI_EXP_SLTCTL_CCIE;

        mask = (PCI_EXP_SLTCTL_PDCE | PCI_EXP_SLTCTL_ABPE |
                PCI_EXP_SLTCTL_PFDE |
                PCI_EXP_SLTCTL_HPIE | PCI_EXP_SLTCTL_CCIE |
                PCI_EXP_SLTCTL_DLLSCE);

        pcie_write_cmd_nowait(ctrl, cmd, mask);

So we should handle PDC at all.

Thanks

Yinghai

#3 lore raw
In reply to Yinghai Lu
Quoted 43 lines · Yinghai Lu

On one system during power off slot, find card get power on right away.

 # echo 0 > /sys/bus/pci/slots/1/power
 pci_hotplug: power_write_file: power = 0
 pciehp 0000:16:00.0:pcie004: pciehp_get_power_status: SLOTCTRL a8 value read 11f1
 pciehp 0000:16:00.0:pcie004: pciehp_unconfigure_device: domain:bus:dev = 0000:17:00
 pci 0000:17:00.0: PME# disabled
 pci 0000:17:00.0: freeing pci_dev info
 pciehp 0000:16:00.0:pcie004: pending interrupts 0x0010 from Slot Status
 pciehp 0000:16:00.0:pcie004: pciehp_power_off_slot: SLOTCTRL a8 write cmd 400
 pciehp 0000:16:00.0:pcie004: pending interrupts 0x0108 from Slot Status
 pciehp 0000:16:00.0:pcie004: Slot(1): Link Down
 pciehp 0000:16:00.0:pcie004: Slot(1): Link Down event ignored; already powering off
 pciehp 0000:16:00.0:pcie004: pciehp_green_led_off: SLOTCTRL a8 write cmd 300
 pciehp 0000:16:00.0:pcie004: pending interrupts 0x0018 from Slot Status  <======
 pciehp 0000:16:00.0:pcie004: Slot(1): Card present
 pciehp 0000:16:00.0:pcie004: pciehp_get_power_status: SLOTCTRL a8 value read 17f1
 pciehp 0000:16:00.0:pcie004: pending interrupts 0x0010 from Slot Status
 pciehp 0000:16:00.0:pcie004: pciehp_power_on_slot: SLOTCTRL a8 write cmd 0
 pciehp 0000:16:00.0:pcie004: pciehp_green_led_blink: SLOTCTRL a8 write cmd 200
 pciehp 0000:16:00.0:pcie004: pending interrupts 0x0010 from Slot Status
 pciehp 0000:16:00.0:pcie004: pciehp_check_link_active: lnk_status = f103
 pciehp 0000:16:00.0:pcie004: pending interrupts 0x0108 from Slot Status
 pciehp 0000:16:00.0:pcie004: Slot(1): Link Up
...

Somehow PDC bit get set, and during handling interrupt that is caused by CC, that PDC also get handled, and the card get powered on again.

In pcie_enable_notification(), we already only enable notification for PDC when attention button is not there.
So we can safely add checking in pciehp_isr() to skip that PDC handling.

---
 drivers/pci/hotplug/pciehp_hpc.c |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

Index: linux-2.6/drivers/pci/hotplug/pciehp_hpc.c
===================================================================

drivers/pci/hotplug/pciehp_hpc.c static irqreturn_t pciehp_isr(int irq, v · line 623

Quoted 8 lines
drivers/pci/hotplug/pciehp_hpc.c +3 -2
@@ -618,8 +618,9 @@ static irqreturn_t pciehp_isr(int irq, v
 		present = !!(status & PCI_EXP_SLTSTA_PDS);
 		ctrl_info(ctrl, "Slot(%s): Card %spresent\n", slot_name(slot),
 			  present ? "" : "not ");
-		pciehp_queue_interrupt_event(slot, present ? INT_PRESENCE_ON :
-					     INT_PRESENCE_OFF);
+		if (!ATTN_BUTTN(ctrl))
+			pciehp_queue_interrupt_event(slot, present ?
+					INT_PRESENCE_ON : INT_PRESENCE_OFF);

I don't think it really makes sense to tie PDC handling with the attention button. It might happen to avoid the problem on your system, but I don't see the logical connection between them.

Can you reproduce this by disabling pciehp and driving this sequence manually with setpci? I suspect that we are tripping over our own feet because we read PCI_EXP_SLTSTA once, clear it (probably too early), then queue multiple events, then process those events possibly simultaneously.

Quoted 3 lines
drivers/pci/hotplug/pciehp_hpc.c +0 -0
 	}
 
 	/* Check Power Fault Detected */
author
#4 lore raw
Quoted 5 lines · Bjorn Helgaas

On Thu, Feb 16, 2017 at 10:12:47PM -0800, Yinghai Lu wrote:

I don't think it really makes sense to tie PDC handling with the attention button. It might happen to avoid the problem on your system, but I don't see the logical connection between them.

but in pcie_enable_notification() we don't enable PDCE when ATTN is not there.

        cmd = PCI_EXP_SLTCTL_DLLSCE;
        if (ATTN_BUTTN(ctrl))
                cmd |= PCI_EXP_SLTCTL_ABPE;
        else
                cmd |= PCI_EXP_SLTCTL_PDCE;
        if (!pciehp_poll_mode)
                cmd |= PCI_EXP_SLTCTL_HPIE | PCI_EXP_SLTCTL_CCIE;

        mask = (PCI_EXP_SLTCTL_PDCE | PCI_EXP_SLTCTL_ABPE |
                PCI_EXP_SLTCTL_PFDE |
                PCI_EXP_SLTCTL_HPIE | PCI_EXP_SLTCTL_CCIE |
                PCI_EXP_SLTCTL_DLLSCE);

        pcie_write_cmd_nowait(ctrl, cmd, mask);

should we remove that check there ?

Quoted 6 lines

Can you reproduce this by disabling pciehp and driving this sequence manually with setpci? I suspect that we are tripping over our own feet because we read PCI_EXP_SLTSTA once, clear it (probably too early), then queue multiple events, then process those events possibly simultaneously.

sca15-2243-0a818158:~ # echo 1 > /sys/bus/pci/devices/0000\:3b\:00.0/remove

[  171.769322] pci 0000:3b:00.0: PME# disabled
[  171.774459] iommu: Removing device 0000:3b:00.0 from group 36
[  171.780984] pci 0000:3b:00.0: freeing pci_dev info

sca15-2243-0a818158:~ # setpci -s 0000:3a:00.0 0xa8.w 01cb
sca15-2243-0a818158:~ # setpci -s 0000:3a:00.0 0xaa.w 0050
sca15-2243-0a818158:~ # setpci -s 0000:3a:00.0 0xa8.w=0x05cb sca15-2243-0a818158:~ # setpci -s 0000:3a:00.0 0xaa.w 0158

so after power off, status bit 3 the PDC get set.