Skip to contentSkip to conversation

Intel Sky Lake-E host root ports check.

PATCH Superseded Shlomo Pongratz 4 participants 9 messages Mar 30, 2022

Subsystems (2)

v4 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v4 cover Intel Sky Lake-E host root ports check.

Changes in v4:

Address Bjorn Helgaas and Jason Gunthorpe comments. Replace the implementation of pci_is_root_port with a simple check pci_pcie_type(root) != PCI_EXP_TYPE_ROOT_PORT and remove the added IS_ROOT_PORT flag.
Update patch text.

Changes in v3:

Use Jason Gunthorpe suggestion, that is add a flag 'IS_ROOT_PORT' instead of 'port' and then just ignore the slot number entirely for root ports.

Changes in v2:

Change comment and description based on Logan Gunthorpe comments.

v1:

In commit 7b94b53db34f ("PCI/P2PDMA: Add Intel Sky Lake-E Root Ports B, C, D to the whitelist")
Andrew Maier added the Sky Lake-E additional devices
2031, 2032 and 2033 root ports to the already existing 2030 device.

The Intel devices 2030, 2031, 2032 and 2033 which are root ports A, B, C and D, respectively and if all exist they will occupy slots 0 till 3 in that order.

The original code handled only the case where the devices in the whiltlist are host bridges and assumed that they will be found on slot 0.

This assumption doesn't hold for root ports so an explicit test was added to cover this case.

Shlomo Pongratz (1):
Intel Sky Lake-E host root ports check.

 drivers/pci/p2pdma.c | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)
Signature · 1 line
2.17.1
#1 lore raw

PATCH v4 1/1 Intel Sky Lake-E host root ports check.

In commit 7b94b53db34f ("PCI/P2PDMA: Add Intel Sky Lake-E Root Ports B, C, D to the whitelist")
Andrew Maier added the Sky Lake-E additional devices
2031, 2032 and 2033 root ports to the already existing 2030 device.

The Intel devices 2030, 2031, 2032 and 2033 which are root ports A, B, C and D, respectively and if all exist they will occupy slots 0 till 3 in that order.

The original code handled only the case where the devices in the whiltlist are host bridges and assumed that they will be found on slot 0.

This assumption doesn't hold for root ports so an explicit test was added to cover this case.

---
 drivers/pci/p2pdma.c | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)
drivers/pci/p2pdma.c +13 -7
index 30b1df3c9d2f..107905dfa229 100644
@@ -327,15 +327,18 @@ static const struct pci_p2pdma_whitelist_entry {
 
 /*
  * This lookup function tries to find the PCI device corresponding to a given
- * host bridge.
+ * host bridge or a root port.
  *
  * It assumes the host bridge device is the first PCI device in the
- * bus->devices list and that the devfn is 00.0. These assumptions should hold
- * for all the devices in the whitelist above.
+ * bus->devices list and that the devfn is 00.0. The first assumption should
+ * hold for all the devices in the whitelist above, however the second one
+ * doesn't always holds for root ports, for example Intel SkyLake-E devices
+ * 2030, 2031, 2032 and 2033 which are root ports (A, B, C and D respectively).
+ * So being a root port is checked explicitly.
  *
- * This function is equivalent to pci_get_slot(host->bus, 0), however it does
- * not take the pci_bus_sem lock seeing __host_bridge_whitelist() must not
- * sleep.
+ * This function is equivalent to pci_get_slot(host->bus, 0) (except for
+ * the root port test), however it does not take the pci_bus_sem lock seeing
+ * __host_bridge_whitelist() must not sleep.
  *
  * For this to be safe, the caller should hold a reference to a device on the
  * bridge, which should ensure the host_bridge device will not be freed
@@ -350,7 +353,10 @@ static struct pci_dev *pci_host_bridge_dev(struct pci_host_bridge *host)
 
 	if (!root)
 		return NULL;
-	if (root->devfn != PCI_DEVFN(0, 0))
+
+	/* Is it a host bridge or a root port? */
+	if (root->devfn != PCI_DEVFN(0, 0) &&
+		pci_pcie_type(root) != PCI_EXP_TYPE_ROOT_PORT)
 		return NULL;
 
 	return root;
Signature · 1 line
2.17.1
maintainer Reviewed-by on v4 1/1
#2 lore raw
Quoted 9 lines · Shlomo Pongratz

In commit 7b94b53db34f ("PCI/P2PDMA: Add Intel Sky Lake-E Root Ports B, C, D to the whitelist")
Andrew Maier added the Sky Lake-E additional devices
2031, 2032 and 2033 root ports to the already existing 2030 device.

The Intel devices 2030, 2031, 2032 and 2033 which are root ports A, B, C and D, respectively and if all exist they will occupy slots 0 till 3 in that order.

The original code handled only the case where the devices in the whiltlist are

s/whiltlist/whitelist/

Quoted 6 lines

host bridges and assumed that they will be found on slot 0.

This assumption doesn't hold for root ports so an explicit test was added to cover this case.

This patch looks good to me. Much cleaner and easier to understand than the previous ones.

Thanks,

Logan

#3 lore raw
In reply to Shlomo Pongratz

drivers/pci/p2pdma.c static struct pci_dev *pci_host_bridge_dev(struct pci_host_bridge *host) · line 357

Quoted 7 lines · Shlomo Pongratz
drivers/pci/p2pdma.c +2 -1
@@ -350,7 +353,10 @@ static struct pci_dev *pci_host_bridge_dev(struct pci_host_bridge *host)
 
 	if (!root)
 		return NULL;
-	if (root->devfn != PCI_DEVFN(0, 0))
+
+	/* Is it a host bridge or a root port? */

This is a better comment:

/* host bridges must have a 0 devfn, but some of the entries in the
whilelist are root ports that can have any devfn */

drivers/pci/p2pdma.c static struct pci_dev *pci_host_bridge_dev(struct pci_host_bridge *host) · line 360

Quoted 3 lines
drivers/pci/p2pdma.c +2 -0
+	if (root->devfn != PCI_DEVFN(0, 0) &&
+		pci_pcie_type(root) != PCI_EXP_TYPE_ROOT_PORT)
 		return NULL;

Indentation is wrong

Jason

#4 lore raw
Quoted 11 lines · Jason Gunthorpe

drivers/pci/p2pdma.c static struct pci_dev *pci_host_bridge_dev(struct pci_host_bridge *host) · line 357

Quoted 7 lines · Shlomo Pongratz
drivers/pci/p2pdma.c +2 -1
@@ -350,7 +353,10 @@ static struct pci_dev *pci_host_bridge_dev(struct pci_host_bridge *host)
 
 	if (!root)
 		return NULL;
-	if (root->devfn != PCI_DEVFN(0, 0))
+
+	/* Is it a host bridge or a root port? */

This is a better comment:

Quoted 2 lines

/* host bridges must have a 0 devfn, but some of the entries in the
whilelist are root ports that can have any devfn */

Is this something in the spec or is it just common practice? The PCIe spec says very little about "host bridges" and I don't remember anything about them having to be devfn 0 or even that they have to be materialized as PCI devices.

Quoted 7 lines

drivers/pci/p2pdma.c static struct pci_dev *pci_host_bridge_dev(struct pci_host_bridge *host) · line 360

Quoted 3 lines
drivers/pci/p2pdma.c +2 -0
+	if (root->devfn != PCI_DEVFN(0, 0) &&
+		pci_pcie_type(root) != PCI_EXP_TYPE_ROOT_PORT)
 		return NULL;

Indentation is wrong

Jason

#5 lore raw
Quoted 12 lines · Bjorn Helgaas
Quoted 11 lines · Jason Gunthorpe

drivers/pci/p2pdma.c static struct pci_dev *pci_host_bridge_dev(struct pci_host_bridge *host) · line 357

Quoted 7 lines · Shlomo Pongratz
drivers/pci/p2pdma.c +2 -1
@@ -350,7 +353,10 @@ static struct pci_dev *pci_host_bridge_dev(struct pci_host_bridge *host)
 
 	if (!root)
 		return NULL;
-	if (root->devfn != PCI_DEVFN(0, 0))
+
+	/* Is it a host bridge or a root port? */

This is a better comment:

Quoted 7 lines
Quoted 2 lines

/* host bridges must have a 0 devfn, but some of the entries in the
whilelist are root ports that can have any devfn */

Is this something in the spec or is it just common practice? The PCIe spec says very little about "host bridges" and I don't remember anything about them having to be devfn 0 or even that they have to be materialized as PCI devices.

I think we are relying on common practice here, but I don't know why this check was added in the first place? Logan?

Jason

maintainer
#6 lore raw
Quoted 16 lines · Jason Gunthorpe
Quoted 15 lines · Bjorn Helgaas
Quoted 13 lines · Jason Gunthorpe

drivers/pci/p2pdma.c static struct pci_dev *pci_host_bridge_dev(struct pci_host_bridge *host) · line 357

Quoted 7 lines · Shlomo Pongratz
drivers/pci/p2pdma.c +2 -1
@@ -350,7 +353,10 @@ static struct pci_dev *pci_host_bridge_dev(struct pci_host_bridge *host)
 
 	if (!root)
 		return NULL;
-	if (root->devfn != PCI_DEVFN(0, 0))
+
+	/* Is it a host bridge or a root port? */

This is a better comment:

/* host bridges must have a 0 devfn, but some of the entries in the
whilelist are root ports that can have any devfn */

Quoted 7 lines
Quoted 4 lines

Is this something in the spec or is it just common practice? The PCIe spec says very little about "host bridges" and I don't remember anything about them having to be devfn 0 or even that they have to be materialized as PCI devices.

I think we are relying on common practice here, but I don't know why this check was added in the first place? Logan?

Yeah, this was more about common practice than spec. Seeing we only support a subset of devices it was expected that those added to our support list would meet this criteria.

The main purpose here is to just get identifying information about the host bridge. I wouldn't say that the host bridge is dev-fn 0 in any sense of the word. What a host bridge is vs a root port is also a bit murky . I'm really not sure what the tree looks like on this SkyLake system, but on my test system, the two host bridges are also root ports.

00:00.0 Host bridge: Intel Corporation Xeon E5/Core i7 DMI2 (rev 07)
Capabilities: [90] Express (v2) Root Port (Slot-), MSI 00

The point was really just finding the VID/PID of one of the PCI devices belonging to the bridge for identifying whether it was safe to use. Originally this function was implemented as simply

    pci_get_slot(host->bus, PCI_DEVFN(0, 0));

That was replaced with the current function solely to avoid needing to take pci_bus_sem in this path. The check for dev-fn 0 was kept to ensure it matched the old method.

Checking simply for PCI_EXP_TYPE_ROOT_PORT instead of a zero devfn is probably a good idea, assuming it works for all existing systems. I'd expect it would be set for all the devices currently allowed.

Logan

#7 lore raw
Quoted 3 lines · Logan Gunthorpe

Checking simply for PCI_EXP_TYPE_ROOT_PORT instead of a zero devfn is probably a good idea, assuming it works for all existing systems. I'd expect it would be set for all the devices currently allowed.

I think if we find a PCI ID in the white list as we go up the PCI hierarchy we should just declare success?

Does it matter if it is the top of the tree or if it is a root port or host bridge? The IDs in this list only exist as part of SOCs, so seeing them at all should confirm the whole SOC..

Jason

maintainer
#8 lore raw
Quoted 8 lines · Jason Gunthorpe
Quoted 3 lines · Logan Gunthorpe

Checking simply for PCI_EXP_TYPE_ROOT_PORT instead of a zero devfn is probably a good idea, assuming it works for all existing systems. I'd expect it would be set for all the devices currently allowed.

I think if we find a PCI ID in the white list as we go up the PCI hierarchy we should just declare success?

Well, in older hardware we also have to ensure that the devices are on the same host bridge. (See REQ_SAME_HOST_BRIDGE). So it's a bit more complicated than that...

Quoted 3 lines

Does it matter if it is the top of the tree or if it is a root port or host bridge? The IDs in this list only exist as part of SOCs, so seeing them at all should confirm the whole SOC..

Not sure if that will always hold with multi-socket systems. The code currently uses pci_find_host_bridge() for each device, then uses pci_host_bridge_dev() simply to find identifying information about the host bridge. It doesn't really matter where in the tree things are, but it would be add for the host bridge not to be at the top, I would have thought.

Logan