Skip to contentSkip to conversation

VMD subdevice secondary bus resets

PATCH In review Jon Derrick 2 participants 5 messages Sep 28, 2020 – Nov 19, 2020

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover VMD subdevice secondary bus resets

This set adds some resets for VMD. It's very common code but doesn't seem to fit well anywhere that can also be exported if VMD is built as a module.

Jon Derrick (2):
PCI: vmd: Reset the VMD subdevice domain on probe
PCI: Add a reset quirk for VMD

 drivers/pci/controller/vmd.c | 32 ++++++++++++++++++++++++
 drivers/pci/quirks.c         | 48 ++++++++++++++++++++++++++++++++++++
 2 files changed, 80 insertions(+)
Signature · 1 line
2.18.1
#1 lore raw

PATCH 1/2 PCI: vmd: Reset the VMD subdevice domain on probe

The VMD subdevice domain resource requirements may have changed in-between module loads. Generic PCI resource assignment code may rely on existing resource configuration rather than the VMD preference of re-examining the domain. Add a Secondary Bus Reset to the VMD subdevice domain during driver attachment to clear the PCI config space of the subdevices.

---
 drivers/pci/controller/vmd.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
drivers/pci/controller/vmd.c +31 -0
index 50b3520d261d..676acff3622f 100644
@@ -445,6 +445,36 @@ static struct pci_ops vmd_ops = {
 	.write		= vmd_pci_write,
 };
 
+static void vmd_domain_reset_sbr(struct vmd_dev *vmd)
+{
+	char __iomem *base;
+	int rp;
+	u16 ctl;
+
+	/*
+	 * Subdevice config space is mapped linearly using 4k config space
+	 * increments. Use increments of 0x8000 to locate root ports devices.
+	 */
+	for (rp = 0; rp < 4; rp++) {
+		base = vmd->cfgbar + rp * 0x8000;
+		if (readl(base + PCI_COMMAND) == 0xFFFFFFFF)
+			continue;
+
+		/* pci_reset_secondary_bus() */
+		ctl = readw(base + PCI_BRIDGE_CONTROL);
+		ctl |= PCI_BRIDGE_CTL_BUS_RESET;
+		writew(ctl, base + PCI_BRIDGE_CONTROL);
+		readw(base + PCI_BRIDGE_CONTROL);
+		msleep(2);
+
+		ctl &= ~PCI_BRIDGE_CTL_BUS_RESET;
+		writew(ctl, base + PCI_BRIDGE_CONTROL);
+		readw(base + PCI_BRIDGE_CONTROL);
+	}
+
+	ssleep(1);
+}
+
 static void vmd_attach_resources(struct vmd_dev *vmd)
 {
 	vmd->dev->resource[VMD_MEMBAR1].child = &vmd->resources[1];
@@ -784,6 +814,7 @@ static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id)
 	if (!vmd->cfgbar)
 		return -ENOMEM;
 
+	vmd_domain_reset_sbr(dev);
 	pci_set_master(dev);
 	if (dma_set_mask_and_coherent(&dev->dev, DMA_BIT_MASK(64)) &&
 	    dma_set_mask_and_coherent(&dev->dev, DMA_BIT_MASK(32)))
Signature · 1 line
2.18.1
#2 lore raw

PATCH 2/2 PCI: Add a reset quirk for VMD

In reply to Jon Derrick

VMD domains should be reset in-between special attachment such as VFIO users. VMD does not offer a reset however the subdevice domain itself can be reset starting at the Root Bus. Add a Secondary Bus Reset on each of the individual root port devices immediately downstream of the VMD root bus.

---
 drivers/pci/controller/vmd.c |  3 ++-
 drivers/pci/quirks.c         | 48 ++++++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+), 1 deletion(-)
drivers/pci/controller/vmd.c +2 -1
index 676acff3622f..fdc1a206f73e 100644
@@ -14,6 +14,7 @@
 #include <linux/srcu.h>
 #include <linux/rculist.h>
 #include <linux/rcupdate.h>
+#include <linux/delay.h>
 
 #include <asm/irqdomain.h>
 #include <asm/device.h>
@@ -814,7 +815,7 @@ static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id)
 	if (!vmd->cfgbar)
 		return -ENOMEM;
 
-	vmd_domain_reset_sbr(dev);
+	vmd_domain_reset_sbr(vmd);
 	pci_set_master(dev);
 	if (dma_set_mask_and_coherent(&dev->dev, DMA_BIT_MASK(64)) &&
 	    dma_set_mask_and_coherent(&dev->dev, DMA_BIT_MASK(32)))
drivers/pci/quirks.c +48 -0

60 lines not on this page · raw message

Signature · 1 line
2.18.1
#3 lore raw
In reply to Jon Derrick
Quoted 11 lines · Jon Derrick

This set adds some resets for VMD. It's very common code but doesn't seem to fit well anywhere that can also be exported if VMD is built as a module.

Jon Derrick (2):
PCI: vmd: Reset the VMD subdevice domain on probe
PCI: Add a reset quirk for VMD

 drivers/pci/controller/vmd.c | 32 ++++++++++++++++++++++++
 drivers/pci/quirks.c         | 48 ++++++++++++++++++++++++++++++++++++
 2 files changed, 80 insertions(+)

I can queue it up but I need Bjorn's ACK on patch (2).

Lorenzo

#4 lore raw
Quoted 9 lines · Lorenzo Pieralisi
Quoted 8 lines · Jon Derrick

This set adds some resets for VMD. It's very common code but doesn't seem to fit well anywhere that can also be exported if VMD is built as a module.

Jon Derrick (2):
PCI: vmd: Reset the VMD subdevice domain on probe
PCI: Add a reset quirk for VMD

Quoted 7 lines
Quoted 3 lines
 drivers/pci/controller/vmd.c | 32 ++++++++++++++++++++++++
 drivers/pci/quirks.c         | 48 ++++++++++++++++++++++++++++++++++++
 2 files changed, 80 insertions(+)

I can queue it up but I need Bjorn's ACK on patch (2).

Lorenzo

I just noticed 2/2 fixes something in 1/2, so I will send a v2 for this set.