PCI: Configure PCIe MPS settings
Subsystems (2)
cover PCI: Configure PCIe MPS settings
[Some of you will get this twice because Gmail and I aren't getting along today; sorry]
The PCI core doesn't configure the PCIe MPS settings by itself. Each host bridge driver has to call pcie_bus_configure_settings() to make this happen.
Jon fixed this already for pcie-iproc.c. I propose these similar patches for other drivers.
HV guys, I included you because create_root_hv_pci_bus() is one place that calls pci_scan_child_bus() but does not call pcie_bus_configure_settings(). I know you probably don't strictly *need* to configure MPS settings in a paravirtual front-end, but the PCI core does other device configuration in this path:
pci_scan_child_bus
pci_scan_slot
pci_scan_single_device
pci_device_add
pci_configure_device
pci_init_capabilities
and I would like to eventually migrate the MPS configuration into that same path. Since we do this other configuration for HV devices already, I think pcie_bus_configure_settings() should also work (even if it ends up not doing anything to real devices).
It would make that eventual migration easier if all the pci_scan_child_bus() callers had the same pattern of calling pcie_bus_configure_settings().
So would it make sense to make a similar patch for HV? It looks easy to add it to create_root_hv_pci_bus(), but I don't know exactly what to do about pci_devices_present_work().
---
Bjorn Helgaas (3):
PCI: xilinx: Configure PCIe MPS settings
PCI: versatile: Configure PCIe MPS settings
PCI: xgene: Configure PCIe MPS settings
drivers/pci/host/pci-versatile.c | 4 +++- drivers/pci/host/pci-xgene.c | 4 +++- drivers/pci/host/pcie-xilinx.c | 4 +++- 3 files changed, 9 insertions(+), 3 deletions(-)
PATCH 1/3 PCI: xilinx: Configure PCIe MPS settings
From: Bjorn Helgaas <[email protected]>
Make sure PCIe MPS settings are valid when we enumerate a new hierarchy.
--- drivers/pci/host/pcie-xilinx.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/pci/host/pcie-xilinx.c +3 -1
@@ -632,7 +632,7 @@ static int xilinx_pcie_probe(struct platform_device *pdev) {struct device *dev = &pdev->dev;struct xilinx_pcie_port *port; -struct pci_bus *bus; +struct pci_bus *bus, *child;int err;resource_size_t iobase =0 ;LIST_HEAD (res);
@@ -686,6 +686,8 @@ static int xilinx_pcie_probe(struct platform_device *pdev)#ifndef CONFIG_MICROBLAZE pci_fixup_irqs (pci_common_swizzle, of_irq_parse_and_map_pci);#endif +list_for_each_entry (child, &bus->children, node) +pcie_bus_configure_settings (child);pci_bus_add_devices (bus);return 0 ;
PATCH 1/3 PCI: xilinx: Configure PCIe MPS settings
drivers/pci/host/pcie-xilinx.c static int xilinx_pcie_probe(struct platform_device *pdev) · line 690
Quoted 2 lines · Bjorn Helgaas
drivers/pci/host/pcie-xilinx.c +2
-0+list_for_each_entry (child, &bus->children, node) +pcie_bus_configure_settings (child);
This loop is duplicated in just about every driver, so it it might be a good idea to provide a littler helper for it.
PATCH 1/3 PCI: xilinx: Configure PCIe MPS settings
Quoted 6 lines · Christoph Hellwig
drivers/pci/host/pcie-xilinx.c static int xilinx_pcie_probe(struct platform_device *pdev) · line 690
Quoted 2 lines · Bjorn Helgaas
drivers/pci/host/pcie-xilinx.c +2
-0+list_for_each_entry (child, &bus->children, node) +pcie_bus_configure_settings (child);This loop is duplicated in just about every driver, so it it might be a good idea to provide a littler helper for it.
Yeah, you're right. One reason I didn't is because I would like to remove these if we can move the MPS setup into the pci_device_add() path.
We ought to be able to do at least the minimal "configure this new device to match the existing hierarchy; if that's impossible, disable the device" sort of thing there.
We probably would still want some sort of bus- or root port- or host bridge-level configuration that can take a broader view, and that might be done where we currently have this loop. But I don't know exactly what that should look like.
Bjorn
PATCH 2/3 PCI: versatile: Configure PCIe MPS settings
In reply to Bjorn HelgaasFrom: Bjorn Helgaas <[email protected]>
Make sure PCIe MPS settings are valid when we enumerate a new hierarchy.
--- drivers/pci/host/pci-versatile.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/pci/host/pci-versatile.c +3 -1
@@ -124,7 +124,7 @@ static int versatile_pci_probe(struct platform_device *pdev)int ret, i, myslot =-1 ;u32 val;void __iomem *local_pci_cfg_base; -struct pci_bus *bus; +struct pci_bus *bus, *child;LIST_HEAD (pci_res); res =platform_get_resource (pdev,IORESOURCE_MEM ,0 );
@@ -204,6 +204,8 @@ static int versatile_pci_probe(struct platform_device *pdev)pci_fixup_irqs (pci_common_swizzle, of_irq_parse_and_map_pci);pci_assign_unassigned_bus_resources (bus); +list_for_each_entry (child, &bus->children, node) +pcie_bus_configure_settings (child);pci_bus_add_devices (bus);return 0 ;
PATCH 3/3 PCI: xgene: Configure PCIe MPS settings
In reply to Bjorn HelgaasFrom: Bjorn Helgaas <[email protected]>
Make sure PCIe MPS settings are valid when we enumerate a new hierarchy.
--- drivers/pci/host/pci-xgene.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/pci/host/pci-xgene.c +3 -1
@@ -638,7 +638,7 @@ static int xgene_pcie_probe_bridge(struct platform_device *pdev)struct device_node *dn = dev->of_node;struct xgene_pcie_port *port;resource_size_t iobase =0 ; -struct pci_bus *bus; +struct pci_bus *bus, *child;int ret;LIST_HEAD (res);
@@ -681,6 +681,8 @@ static int xgene_pcie_probe_bridge(struct platform_device *pdev)pci_scan_child_bus (bus);pci_assign_unassigned_bus_resources (bus); +list_for_each_entry (child, &bus->children, node) +pcie_bus_configure_settings (child);pci_bus_add_devices (bus);return 0 ;
cover PCI: Configure PCIe MPS settings
In reply to Bjorn HelgaasQuoted 42 lines
-----Original Message-----
From: Bjorn Helgaas [mailto:helgaas at kernel.org]
Sent: Wednesday, February 8, 2017 2:49 PM
To: linux-pci at vger.kernel.org
Cc: Rob Herring <[email protected]>; Haiyang Zhang <[email protected]>; Michal Simek <[email protected]>; Jon Mason <[email protected]>; Ray Jui <[email protected]>; S?ren Brinkmann <[email protected]>;
devel at linuxdriverproject.org; KY Srinivasan <[email protected]>; linux- arm-kernel at lists.infradead.org; Tanmay Inamdar <[email protected]>
Subject: [PATCH v1 0/3] PCI: Configure PCIe MPS settings[Some of you will get this twice because Gmail and I aren't getting along today; sorry]
The PCI core doesn't configure the PCIe MPS settings by itself. Each host bridge driver has to call
pcie_bus_configure_settings()to make this happen.Jon fixed this already for
pcie-iproc.c. I propose these similar patches for other drivers.HV guys, I included you because
create_root_hv_pci_bus()is one place that callspci_scan_child_bus()but does not callpcie_bus_configure_settings(). I know you probably don't strictly *need* to configure MPS settings in a paravirtual front-end, but the PCI core does other device configuration in this path:
pci_scan_child_bus
pci_scan_slot
pci_scan_single_device
pci_device_add
pci_configure_device
pci_init_capabilitiesand I would like to eventually migrate the MPS configuration into that same path. Since we do this other configuration for HV devices already, I think
pcie_bus_configure_settings()should also work (even if it ends up not doing anything to real devices).
Quoted 7 lines
It would make that eventual migration easier if all the
pci_scan_child_bus()callers had the same pattern of callingpcie_bus_configure_settings().So would it make sense to make a similar patch for HV? It looks easy to add it to
create_root_hv_pci_bus(), but I don't know exactly what to do aboutpci_devices_present_work().
Bjorn,
We will take a look.
K. Y
Quoted 13 lines
---
Bjorn Helgaas (3):
PCI: xilinx: Configure PCIe MPS settings
PCI: versatile: Configure PCIe MPS settings
PCI: xgene: Configure PCIe MPS settingsdrivers/pci/host/pci-versatile.c | 4 +++- drivers/pci/host/pci-xgene.c | 4 +++- drivers/pci/host/pcie-xilinx.c | 4 +++- 3 files changed, 9 insertions(+), 3 deletions(-)
Bjorn Helgaas
Appliedcover PCI: Configure PCIe MPS settings
In reply to Bjorn HelgaasQuoted 44 lines · Bjorn Helgaas
[Some of you will get this twice because Gmail and I aren't getting along today; sorry]
The PCI core doesn't configure the PCIe MPS settings by itself. Each host bridge driver has to call
pcie_bus_configure_settings()to make this happen.Jon fixed this already for
pcie-iproc.c. I propose these similar patches for other drivers.HV guys, I included you because
create_root_hv_pci_bus()is one place that callspci_scan_child_bus()but does not callpcie_bus_configure_settings(). I know you probably don't strictly *need* to configure MPS settings in a paravirtual front-end, but the PCI core does other device configuration in this path:
pci_scan_child_bus
pci_scan_slot
pci_scan_single_device
pci_device_add
pci_configure_device
pci_init_capabilitiesand I would like to eventually migrate the MPS configuration into that same path. Since we do this other configuration for HV devices already, I think
pcie_bus_configure_settings()should also work (even if it ends up not doing anything to real devices).It would make that eventual migration easier if all the
pci_scan_child_bus()callers had the same pattern of callingpcie_bus_configure_settings().So would it make sense to make a similar patch for HV? It looks easy to add it to
create_root_hv_pci_bus(), but I don't know exactly what to do aboutpci_devices_present_work().---
Bjorn Helgaas (3):
PCI: xilinx: Configure PCIe MPS settings
PCI: versatile: Configure PCIe MPS settings
PCI: xgene: Configure PCIe MPS settings
Quoted 4 lines
drivers/pci/host/pci-versatile.c | 4 +++- drivers/pci/host/pci-xgene.c | 4 +++- drivers/pci/host/pcie-xilinx.c | 4 +++- 3 files changed, 9 insertions(+), 3 deletions(-)
I applied these to pci/host-xilinx, pci/host-versatile, and pci/host-xgene, respectively, for v4.11.
cover PCI: Configure PCIe MPS settings
Quoted 36 lines
From: devel [mailto:driverdev-devel.. at ...] On Behalf Of Bjorn Helgaas
Sent: Saturday, February 11, 2017 06:00
...Quoted 32 lines · Bjorn Helgaas
The PCI core doesn't configure the PCIe MPS settings by itself. Each host bridge driver has to call
pcie_bus_configure_settings()to make this happen.Jon fixed this already for
pcie-iproc.c. I propose these similar patches for other drivers.HV guys, I included you because
create_root_hv_pci_bus()is one place that callspci_scan_child_bus()but does not callpcie_bus_configure_settings(). I know you probably don't strictly *need* to configure MPS settings in a paravirtual front-end, but the PCI core does other device configuration in this path:
pci_scan_child_bus
pci_scan_slot
pci_scan_single_device
pci_device_add
pci_configure_device
pci_init_capabilitiesand I would like to eventually migrate the MPS configuration into that same path. Since we do this other configuration for HV devices already, I think
pcie_bus_configure_settings()should also work (even if it ends up not doing anything to real devices).It would make that eventual migration easier if all the
pci_scan_child_bus()callers had the same pattern of callingpcie_bus_configure_settings().So would it make sense to make a similar patch for HV? It looks easy to add it to
create_root_hv_pci_bus(), but I don't know exactly what to do aboutpci_devices_present_work().
(I'm sorry for the really late response. I neglected the thread somehow...)
Hi Bjorn,
IMO we don't need to make any change to pci-hyperv.c: when a PCIe device is passed through to a guest on Hyper-V, in the guest the device only directly appears on a new root bus, and the new root bus doesn't have any child bus, and the new root bus itself is actually only a logical concept instead of a concrete device (i.e. it doesn't have its own BDF at all).
So, in the path pci_scan_child_bus -> ... -> pci_device_add -> pci_configure_device -> pci_configure_mps, we're sure pci_upstream_bridge() is NULL, so pci_configure_mps does nothing.
And since the root bus has no child bus, we don't need to call pcie_bus_configure_settings(child).
Then you may ask how the MPS is set.
I suppose the para-virtual PCI back-end driver on the host should take care of this properly.
FWIW, after passing through a Mellanox VF to Linux guest, in the guest I get this for the device by lspci:
DevCap: MaxPayload 512 bytes
DevCtl: MaxPayload 128 bytes, MaxReadReq 128 bytes
Thanks
-- Dexuan
cover PCI: Configure PCIe MPS settings
Quoted 33 lines · Dexuan Cui
Quoted 33 lines
From: devel [mailto:driverdev-devel.. at ...] On Behalf Of Bjorn Helgaas
Sent: Saturday, February 11, 2017 06:00
...Quoted 29 lines · Bjorn Helgaas
The PCI core doesn't configure the PCIe MPS settings by itself. Each host bridge driver has to call
pcie_bus_configure_settings()to make this happen.Jon fixed this already for
pcie-iproc.c. I propose these similar patches for other drivers.HV guys, I included you because
create_root_hv_pci_bus()is one place that callspci_scan_child_bus()but does not callpcie_bus_configure_settings(). I know you probably don't strictly *need* to configure MPS settings in a paravirtual front-end, but the PCI core does other device configuration in this path:
pci_scan_child_bus
pci_scan_slot
pci_scan_single_device
pci_device_add
pci_configure_device
pci_init_capabilitiesand I would like to eventually migrate the MPS configuration into that same path. Since we do this other configuration for HV devices already, I think
pcie_bus_configure_settings()should also work (even if it ends up not doing anything to real devices).It would make that eventual migration easier if all the
pci_scan_child_bus()callers had the same pattern of callingpcie_bus_configure_settings().
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
So would it make sense to make a similar patch for HV? It looks easy to add it to
create_root_hv_pci_bus(), but I don't know exactly what to do aboutpci_devices_present_work().(I'm sorry for the really late response. I neglected the thread somehow...)
No problem, I'm kind of bad about that, too.
Quoted 15 lines
Hi Bjorn,
IMO we don't need to make any change topci-hyperv.c: when a PCIe device is passed through to a guest on Hyper-V, in the guest the device only directly appears on a new root bus, and the new root bus doesn't have any child bus, and the new root bus itself is actually only a logical concept instead of a concrete device (i.e. it doesn't have its own BDF at all).So, in the path
pci_scan_child_bus-> ... ->pci_device_add->pci_configure_device->pci_configure_mps, we're surepci_upstream_bridge()is NULL, sopci_configure_mpsdoes nothing.And since the root bus has no child bus, we don't need to call
pcie_bus_configure_settings(child).
Quoted 8 lines
Then you may ask how the MPS is set.
I suppose the para-virtual PCI back-end driver on the host should take care of this properly.FWIW, after passing through a Mellanox VF to Linux guest, in the guest I get this for the device by lspci:
DevCap: MaxPayload 512 bytes
DevCtl: MaxPayload 128 bytes, MaxReadReq 128 bytes
My question wasn't so much whether pci-hyperv.c *needs* to set MPS; I don't expect a guest to need to or even be able to configure things like that.
My question is whether it would do any harm if we did eventually do MPS configuration in the generic pci_scan_child_bus() path. If/when we do that, the Hyper-V guest will attempt to do MPS configuration. I think this would be a no-op. And I think it would also be a no-op if we made pci-hyperv.c call pcie_bus_configure_settings() today.
I'm really just looking for confirmation of that. If it is true that these are no-ops, I would consider adding a pci-hyperv.c call to pcie_bus_configure_settings() simply to make it conform to the pattern that "everything that calls pci_scan_child_bus() should also call pcie_bus_configure_settings()".
If every caller of pci_scan_child_bus() also calls pcie_bus_configure_settings(), it will make it easier in the future to fold pcie_bus_configure_settings() into pci_scan_child_bus().
Bjorn
cover PCI: Configure PCIe MPS settings
Quoted 5 lines
From: Bjorn Helgaas
...
My question wasn't so much whetherpci-hyperv.c*needs* to set MPS; I don't expect a guest to need to or even be able to configure things like that.
I agree.
Quoted 4 lines
My question is whether it would do any harm if we did eventually do MPS configuration in the generic
pci_scan_child_bus()path. If/when we do that, the Hyper-V guest will attempt to do MPS configuration. I think this would be a no-op.
Yes. It's a no-op, as I explained in my previous mail.
Quoted 2 lines
And I think it would also be a no-op if
we madepci-hyperv.ccallpcie_bus_configure_settings()today.
Yes, I can confirm it's a no-op too.
Quoted 5 lines
I'm really just looking for confirmation of that. If it is true that these are no-ops, I would consider adding a
pci-hyperv.ccall topcie_bus_configure_settings()simply to make it conform to the pattern that "everything that callspci_scan_child_bus()should also callpcie_bus_configure_settings()".
Please help to add that. Thanks!
Quoted 5 lines
If every caller of
pci_scan_child_bus()also callspcie_bus_configure_settings(), it will make it easier in the future to foldpcie_bus_configure_settings()intopci_scan_child_bus().Bjorn
Thanks! I see.
Thanks,
-- Dexuan