Skip to contentSkip to conversation

PCI: pciehp: Simplify Command Completion checking

PATCH In review Bjorn Helgaas 2 participants 2 messages Feb 10, 2022 – May 8, 2022 PCI

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 PCI: pciehp: Simplify Command Completion checking

From: Bjorn Helgaas <[email protected]>

If a device sets the No Command Completed Support bit in the Slot Capabilities register (PCIe r6.0, sec 7.5.3.9), it does not generate software notifications when a command is completed, and software can write all fields of the Slot Control register without any delays.

Since we only need to wait for command completion on devices that do *not* set the No Command Completed Support bit, there's no need to even set the ctrl->cmd_busy bit that tracks when the device is busy handling a command.

Set the ctrl->cmd_busy bit only when we need to wait for a command to complete. This shouldn't make much functional difference, but does avoid an smp_mb() for controllers that set PCI_EXP_SLTCAP_NCCS.

---
 drivers/pci/hotplug/pciehp_hpc.c | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)
drivers/pci/hotplug/pciehp_hpc.c +11 -9
index 1c1ebf3dad43..0ff928693a13 100644
@@ -114,13 +114,6 @@ static void pcie_wait_cmd(struct controller *ctrl)
 	unsigned long now, timeout;
 	int rc;
 
-	/*
-	 * If the controller does not generate notifications for command
-	 * completions, we never need to wait between writes.
-	 */
-	if (NO_CMD_CMPL(ctrl))
-		return;
-
 	if (!ctrl->cmd_busy)
 		return;
 
@@ -173,8 +166,17 @@ static void pcie_do_write_cmd(struct controller *ctrl, u16 cmd,
 	slot_ctrl_orig = slot_ctrl;
 	slot_ctrl &= ~mask;
 	slot_ctrl |= (cmd & mask);
-	ctrl->cmd_busy = 1;
-	smp_mb();
+
+	/*
+	 * If controller generates Command Completed events, we must wait
+	 * for it to finish one command before sending another, so we need
+	 * to keep track of when the controller is busy.
+	 */
+	if (!NO_CMD_CMPL(ctrl)) {
+		ctrl->cmd_busy = 1;
+		smp_mb();
+	}
+
 	ctrl->slot_ctrl = slot_ctrl;
 	pcie_capability_write_word(pdev, PCI_EXP_SLTCTL, slot_ctrl);
 	ctrl->cmd_started = jiffies;
Signature · 1 line
2.25.1
#1 lore raw
Quoted 12 lines · Bjorn Helgaas

If a device sets the No Command Completed Support bit in the Slot Capabilities register (PCIe r6.0, sec 7.5.3.9), it does not generate software notifications when a command is completed, and software can write all fields of the Slot Control register without any delays.

Since we only need to wait for command completion on devices that do *not* set the No Command Completed Support bit, there's no need to even set the ctrl->cmd_busy bit that tracks when the device is busy handling a command.

Set the ctrl->cmd_busy bit only when we need to wait for a command to complete. This shouldn't make much functional difference, but does avoid an smp_mb() for controllers that set PCI_EXP_SLTCAP_NCCS.

[...]

Quoted 26 lines
drivers/pci/hotplug/pciehp_hpc.c +4 -9
@@ -114,13 +114,6 @@ static void pcie_wait_cmd(struct controller *ctrl)
 	unsigned long now, timeout;
 	int rc;
 
-	/*
-	 * If the controller does not generate notifications for command
-	 * completions, we never need to wait between writes.
-	 */
-	if (NO_CMD_CMPL(ctrl))
-		return;
-
 	if (!ctrl->cmd_busy)
 		return;
 
@@ -173,8 +166,17 @@ static void pcie_do_write_cmd(struct controller *ctrl, u16 cmd,
 	slot_ctrl_orig = slot_ctrl;
 	slot_ctrl &= ~mask;
 	slot_ctrl |= (cmd & mask);
-	ctrl->cmd_busy = 1;
-	smp_mb();
+
+	/*
+	 * If controller generates Command Completed events, we must wait
+	 * for it to finish one command before sending another, so we need

drivers/pci/hotplug/pciehp_hpc.c static void pcie_do_write_cmd(struct controller *ctrl, u16 cmd, · line 182

Quoted 10 lines
drivers/pci/hotplug/pciehp_hpc.c +7 -0
+	 * to keep track of when the controller is busy.
+	 */
+	if (!NO_CMD_CMPL(ctrl)) {
+		ctrl->cmd_busy = 1;
+		smp_mb();
+	}
+
 	ctrl->slot_ctrl = slot_ctrl;
 	pcie_capability_write_word(pdev, PCI_EXP_SLTCTL, slot_ctrl);
 	ctrl->cmd_started = jiffies;

Looks good in principle, however ctrl->cmd_busy is set to 1 in two other places, namely pciehp_resume_noirq() and pciehp_runtime_resume() in pciehp_core.c. Those two need to be amended as well.

Note that a few lines after the hunk above, the ctrl->cmd_busy flag is reverted back to 0 on certain controllers:

	if (pdev->broken_cmd_compl &&
	    (slot_ctrl_orig & CC_ERRATUM_MASK) == (slot_ctrl & CC_ERRATUM_MASK))
		ctrl->cmd_busy = 0;

That could be combined with your "if (!NO_CMD_CMPL(ctrl))" check:
Basically, "set cmd_busy to 1 if the controller supports Command Completed events and it's not affected by this erratum".

Perhaps it makes sense to put that in a pciehp_has_command_completion() helper (or whatever name you prefer), which could then also be called from pciehp_resume_noirq() / pciehp_runtime_resume().

Thanks (and sorry for the delay),

Lukas