PCI: vmd: Fix secondary bus reset for Intel bridges
Subsystems (3)
PATCH v3 1/1 PCI: vmd: Fix secondary bus reset for Intel bridges
From: Francisco Munoz <[email protected]>
The reset was never applied in the current implementation because Intel Bridges owned by VMD are parentless. Internally, pci_reset_bus() applies a reset to the parent of the PCI device supplied as argument, but in this case it failed because there wasn't a parent.
In more detail, this change allows the VMD driver to enumerate NVMe devices in pass-through configurations when guest reboots are performed. Commit id 6aab5622296b ("PCI: vmd: Clean up domain before enumeration") attempted to fix this, but later we discovered that the code inside pci_reset_bus() wasn’t triggering secondary bus resets. Therefore, we updated the parameters passed to it, and now NVMe SSDs attached to VMD bridges are properly enumerated in VT-d pass-through scenarios.
---
V3:
- Add WARN_ON
- Include Jonathan as reviewer
- Update commit message
V2:
- Update commit message
drivers/pci/controller/vmd.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)
drivers/pci/controller/vmd.c +11 -2
@@ -859,8 +859,17 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)pci_scan_child_bus (vmd->bus);vmd_domain_reset (vmd); -list_for_each_entry (child, &vmd->bus->children, node) -pci_reset_bus (child->self); + +list_for_each_entry (child, &vmd->bus->children, node) { +if (!list_empty (&child->devices)) { + ret =pci_reset_bus (list_first_entry (&child->devices, + struct pci_dev, + bus_list)); +WARN_ON (ret); +break ; + } + } +pci_assign_unassigned_bus_resources (vmd->bus);/*
Signature · 1 line
2.25.1
Quoted 43 lines · On 11/3/2022 1:14 PM, [email protected] wrote:
From: Francisco Munoz <[email protected]>
The reset was never applied in the current implementation because Intel Bridges owned by VMD are parentless. Internally,
pci_reset_bus()applies a reset to the parent of the PCI device supplied as argument, but in this case it failed because there wasn't a parent.In more detail, this change allows the VMD driver to enumerate NVMe devices in pass-through configurations when guest reboots are performed. Commit id
6aab5622296b("PCI: vmd: Clean up domain before enumeration") attempted to fix this, but later we discovered that the code insidepci_reset_bus()wasn’t triggering secondary bus resets. Therefore, we updated the parameters passed to it, and now NVMe SSDs attached to VMD bridges are properly enumerated in VT-d pass-through scenarios.Signed-off-by Francisco Munoz <[email protected]>Reviewed-by Nirmal Patel <[email protected]>Reviewed-by Jonathan Derrick <[email protected]>---
V3:
- AddWARN_ON
- Include Jonathan as reviewer
- Update commit message
V2:
- Update commit messagedrivers/pci/controller/vmd.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)drivers/pci/controller/vmd.c +4
-2@@ -859,8 +859,17 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)pci_scan_child_bus (vmd->bus);vmd_domain_reset (vmd); -list_for_each_entry (child, &vmd->bus->children, node) -pci_reset_bus (child->self); + +list_for_each_entry (child, &vmd->bus->children, node) { +if (!list_empty (&child->devices)) { + ret = pci_reset_bus(list_first_entry(&child->devices,
drivers/pci/controller/vmd.c static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features) · line 875
Quoted 10 lines
drivers/pci/controller/vmd.c +7
-0+ struct pci_dev, + bus_list)); +WARN_ON (ret); +break ; + } + } +pci_assign_unassigned_bus_resources (vmd->bus);/*
Hi,
Just a gentle reminder for this one
Best wishes,
Francisco.
Lorenzo Pieralisi
maintainerQuoted 7 lines · Francisco
Quoted 6 lines · On 11/3/2022 1:14 PM, [email protected] wrote:
From: Francisco Munoz <[email protected]>
The reset was never applied in the current implementation because Intel Bridges owned by VMD are parentless. Internally,
pci_reset_bus()applies a reset to the parent of the PCI device supplied as argument, but in this case it failed because there wasn't a parent.
Please add a comment in the *code* as well and explain why it is so.
Quoted 7 lines
Quoted 7 lines
In more detail, this change allows the VMD driver to enumerate NVMe devices in pass-through configurations when guest reboots are performed. Commit id
6aab5622296b("PCI: vmd: Clean up domain before enumeration") attempted to fix this, but later we discovered that the code insidepci_reset_bus()wasn’t triggering secondary bus resets. Therefore, we updated the parameters passed to it, and now NVMe SSDs attached to VMD bridges are properly enumerated in VT-d pass-through scenarios.
I am not sure this helps much - I'd rather explain why the device hierarchy in VMD is as it is - when that's explained is rather clear why the current reset is NOP and this patch is needed.
Quoted 20 lines
Quoted 20 lines
Signed-off-by Francisco Munoz <[email protected]>Reviewed-by Nirmal Patel <[email protected]>Reviewed-by Jonathan Derrick <[email protected]>---
V3:
- AddWARN_ON
- Include Jonathan as reviewer
- Update commit message
V2:
- Update commit messagedrivers/pci/controller/vmd.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)drivers/pci/controller/vmd.c +0
-0@@ -859,8 +859,17 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
Quoted 11 lines
drivers/pci/controller/vmd.c static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features) · line 868
Quoted 11 lines
drivers/pci/controller/vmd.c +7
-2pci_scan_child_bus (vmd->bus);vmd_domain_reset (vmd); -list_for_each_entry (child, &vmd->bus->children, node) -pci_reset_bus (child->self); + +list_for_each_entry (child, &vmd->bus->children, node) { +if (!list_empty (&child->devices)) { + ret =pci_reset_bus (list_first_entry (&child->devices, + struct pci_dev, + bus_list)); +WARN_ON (ret);
Technically you are adding a WARN_ON() here to make sure that a failed reset is detected - I am not sure a backtrace is really required, a pci_warn() maybe ?
Lorenzo
Quoted 15 lines
drivers/pci/controller/vmd.c static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features) · line 875
Quoted 7 lines
drivers/pci/controller/vmd.c +4
-0+break ; + } + } +pci_assign_unassigned_bus_resources (vmd->bus);/* Hi,
Just a gentle reminder for this one
Best wishes,
Francisco.