VMD: Configuring bus settings
Subsystems (3)
PATCH 1/1 VMD: Configuring bus settings
Hi Bjorn, Lorenzo, Keith,
This patch supercedes the previous MPS on rescan patch:
https://patchwork.kernel.org/patch/10671355/
The previous patch shouldn't have negatively effected already-added devices, but it became clear that some devices required special MPS handling provided by quirks, and thus wasn't safe to perform generically.
This patch instead open-codes pci_rescan_bus() in VMD and performs bus settings prior to driver attach
Jon Derrick (1):
PCI/VMD: Configure MPS settings before adding devices
drivers/pci/controller/vmd.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)
Signature · 1 line
1.8.3.1
PATCH 1/1 PCI/VMD: Configure MPS settings before adding devices
In order to provide the most performance and/or compatible settings, ensure VMD root buses observe the pcie bus tuning settings by configuring those settings prior to adding the devices to the pcie tree.
This patch open-codes pci_rescan_bus() and configures the buses prior to adding devices and attaching drivers.
--- drivers/pci/controller/vmd.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)
drivers/pci/controller/vmd.c +14 -1
@@ -585,6 +585,7 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)LIST_HEAD (resources);resource_size_t offset[2 ] = {0 };resource_size_t membar2_offset =0x2000 , busn_start =0 ; +struct pci_bus *child;/* * Shadow registers may exist in certain VMD device ids which allow
@@ -712,7 +713,19 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)vmd_attach_resources (vmd);vmd_setup_dma_ops (vmd);dev_set_msi_domain (&vmd->bus->dev, vmd->irq_domain); -pci_rescan_bus (vmd->bus); + +pci_scan_child_bus (vmd->bus); +pci_assign_unassigned_bus_resources (vmd->bus); + +/* +* VMD root buses are virtual and don't return true on pci_is_pcie() +* and will fail pcie_bus_configure_settings() early. It can instead be +* run on each of the real root ports. +*/ +list_for_each_entry (child, &vmd->bus->children, node) +pcie_bus_configure_settings (child); + +pci_bus_add_devices (vmd->bus);WARN (sysfs_create_link (&vmd->dev->dev.kobj, &vmd->bus->dev.kobj,"domain" ),"Can't create symlink to domain\n" );
Signature · 1 line
1.8.3.1
Re: [PATCH] PCI/VMD: Configure MPS settings before adding devices
Quoted 8 lines · Jon Derrick
In order to provide the most performance and/or compatible settings, ensure VMD root buses observe the pcie bus tuning settings by configuring those settings prior to adding the devices to the pcie tree.
This patch open-codes
pci_rescan_bus()and configures the buses prior to adding devices and attaching drivers.Signed-off-by Jon Derrick <[email protected]>
Makes sense.
Quoted 40 lines
--- drivers/pci/controller/vmd.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)drivers/pci/controller/vmd.c +14
-1@@ -585,6 +585,7 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)LIST_HEAD (resources);resource_size_t offset[2 ] = {0 };resource_size_t membar2_offset =0x2000 , busn_start =0 ; +struct pci_bus *child;/* * Shadow registers may exist in certain VMD device ids which allow @@ -712,7 +713,19 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)vmd_attach_resources (vmd);vmd_setup_dma_ops (vmd);dev_set_msi_domain (&vmd->bus->dev, vmd->irq_domain); -pci_rescan_bus (vmd->bus); + +pci_scan_child_bus (vmd->bus); +pci_assign_unassigned_bus_resources (vmd->bus); + +/* +* VMD root buses are virtual and don't return true on pci_is_pcie() +* and will fail pcie_bus_configure_settings() early. It can instead be +* run on each of the real root ports. +*/ +list_for_each_entry (child, &vmd->bus->children, node) +pcie_bus_configure_settings (child); + +pci_bus_add_devices (vmd->bus);WARN (sysfs_create_link (&vmd->dev->dev.kobj, &vmd->bus->dev.kobj,"domain" ),"Can't create symlink to domain\n" );Signature · 1 line
1.8.3.1
Re: [PATCH] PCI/VMD: Configure MPS settings before adding devices
Ping?
Quoted 65 lines · Keith Busch
Quoted 11 lines · Jon Derrick
In order to provide the most performance and/or compatible settings,
ensure VMD root buses observe the pcie bus tuning settings by configuring those settings prior to adding the devices to the pcie tree.This patch open-codes
pci_rescan_bus()and configures the buses prior to
adding devices and attaching drivers.Signed-off-by Jon Derrick <[email protected]>Makes sense.
Reviewed-by Keith Busch <[email protected]>Quoted 48 lines
--- drivers/pci/controller/vmd.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)drivers/pci/controller/vmd.c +1
-0@@ -585,6 +585,7 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)LIST_HEAD (resources );resource_size_t offset[2 ] = {0 };resource_size_t membar2_offset =0x2000 , busn_start =0 ; +struct pci_bus *child;/* * Shadow registers may exist in certain VMD device ids
which allowdrivers/pci/controller/vmd.c +7
-1@@ -712,7 +713,19 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)vmd_attach_resources (vmd );vmd_setup_dma_ops (vmd);dev_set_msi_domain (&vmd->bus->dev, vmd->irq_domain); -pci_rescan_bus (vmd->bus); + +pci_scan_child_bus (vmd->bus); +pci_assign_unassigned_bus_resources (vmd->bus); + +/* +* VMD root buses are virtual and don't return true on pci_is_pcie() +* and will fail pcie_bus_configure_settings() early. It can instead be
drivers/pci/controller/vmd.c +6
-0+* run on each of the real root ports. +*/ +list_for_each_entry (child, &vmd->bus->children, node) +pcie_bus_configure_settings (child); + +pci_bus_add_devices (vmd->bus); WARN(sysfs_create_link(&vmd->dev->dev.kobj, &vmd->bus-Quoted 1 line
dev.kobj,
"domain"), "Can't create symlink to
domain\n");Signature · 1 line
1.8.3.1
Re: [PATCH] PCI/VMD: Configure MPS settings before adding devices
In reply to Jon DerrickQuoted 8 lines · Jon Derrick
In order to provide the most performance and/or compatible settings, ensure VMD root buses observe the pcie bus tuning settings by configuring those settings prior to adding the devices to the pcie tree.
This patch open-codes
pci_rescan_bus()and configures the buses prior to adding devices and attaching drivers.Signed-off-by Jon Derrick <[email protected]>
Sorry, Lorenzo takes care of drivers/pci/controller/*, but I had mistakenly delegated this to myself, so it probably wasn't on his radar.
My dream is that someday pcie_bus_configure_settings() will somehow be done as part of enumeration and host bridge drivers won't need to care about it, but that's obviously fanciful, so this looks fine to me.
Quoted 40 lines
--- drivers/pci/controller/vmd.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)drivers/pci/controller/vmd.c +14
-1@@ -585,6 +585,7 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)LIST_HEAD (resources);resource_size_t offset[2 ] = {0 };resource_size_t membar2_offset =0x2000 , busn_start =0 ; +struct pci_bus *child;/* * Shadow registers may exist in certain VMD device ids which allow @@ -712,7 +713,19 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)vmd_attach_resources (vmd);vmd_setup_dma_ops (vmd);dev_set_msi_domain (&vmd->bus->dev, vmd->irq_domain); -pci_rescan_bus (vmd->bus); + +pci_scan_child_bus (vmd->bus); +pci_assign_unassigned_bus_resources (vmd->bus); + +/* +* VMD root buses are virtual and don't return true on pci_is_pcie() +* and will fail pcie_bus_configure_settings() early. It can instead be +* run on each of the real root ports. +*/ +list_for_each_entry (child, &vmd->bus->children, node) +pcie_bus_configure_settings (child); + +pci_bus_add_devices (vmd->bus);WARN (sysfs_create_link (&vmd->dev->dev.kobj, &vmd->bus->dev.kobj,"domain" ),"Can't create symlink to domain\n" );Signature · 1 line
1.8.3.1
Lorenzo Pieralisi
AppliedRe: [PATCH] PCI/VMD: Configure MPS settings before adding devices
In reply to Jon DerrickQuoted 11 lines · Jon Derrick
In order to provide the most performance and/or compatible settings, ensure VMD root buses observe the pcie bus tuning settings by configuring those settings prior to adding the devices to the pcie tree.
This patch open-codes
pci_rescan_bus()and configures the buses prior to adding devices and attaching drivers.Signed-off-by Jon Derrick <[email protected]>--- drivers/pci/controller/vmd.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)
Applied to pci/vmd for v5.1, thanks and apologies for the delay.
Lorenzo
Quoted 36 lines
drivers/pci/controller/vmd.c +14
-1@@ -585,6 +585,7 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)LIST_HEAD (resources);resource_size_t offset[2 ] = {0 };resource_size_t membar2_offset =0x2000 , busn_start =0 ; +struct pci_bus *child;/* * Shadow registers may exist in certain VMD device ids which allow @@ -712,7 +713,19 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)vmd_attach_resources (vmd);vmd_setup_dma_ops (vmd);dev_set_msi_domain (&vmd->bus->dev, vmd->irq_domain); -pci_rescan_bus (vmd->bus); + +pci_scan_child_bus (vmd->bus); +pci_assign_unassigned_bus_resources (vmd->bus); + +/* +* VMD root buses are virtual and don't return true on pci_is_pcie() +* and will fail pcie_bus_configure_settings() early. It can instead be +* run on each of the real root ports. +*/ +list_for_each_entry (child, &vmd->bus->children, node) +pcie_bus_configure_settings (child); + +pci_bus_add_devices (vmd->bus);WARN (sysfs_create_link (&vmd->dev->dev.kobj, &vmd->bus->dev.kobj,"domain" ),"Can't create symlink to domain\n" );Signature · 1 line
1.8.3.1