PCI: acpiphp: Reassign resources on bridge if necessary
PATCH v2 1/1 PCI: acpiphp: Reassign resources on bridge if necessary
When using ACPI PCI hotplug, hotplugging a device with large BARs may fail if bridge windows programmed by firmware are not large enough.
Reproducer:
$ qemu-kvm -monitor stdio -M q35 -m 4G \
-global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on \
-device id=rp1,pcie-root-port,bus=pcie.0,chassis=4 \
disk_image wait till linux guest boots, then hotplug device
(qemu) device_add qxl,bus=rp1
hotplug on guest side fails with:
pci 0000:01:00.0: [1b36:0100] type 00 class 0x038000 pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00001fff] pci 0000:01:00.0: reg 0x1c: [io 0x0000-0x001f]
pci 0000:01:00.0: BAR 0: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 1: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 1: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 2: assigned [mem 0xfe800000-0xfe801fff] pci 0000:01:00.0: BAR 3: assigned [io 0x1000-0x101f] qxl 0000:01:00.0: enabling device (0000 -> 0003) Unable to create vram_mapping
qxl: probe of 0000:01:00.0 failed with error -12
However when using native PCIe hotplug
'-global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off'
it works fine, since kernel attempts to reassign unused resources. Use the same machinery as native PCIe hotplug to (re)assign resources.
---
tested in QEMU with Q35 machine on PCIE root port and also with nested conventional bridge attached to root port.
v2:
* fixup subject to match expected style
* drop no longer needed __pci_bus_size_bridges() to avoid memory leak (Bjorn Helgaas <[email protected]>)
--- drivers/pci/hotplug/acpiphp_glue.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)
drivers/pci/hotplug/acpiphp_glue.c +1 -4
@@ -498,7 +498,6 @@ static void enable_slot(struct acpiphp_slot *slot, bool bridge)acpiphp_native_scan_bridge (dev); } }else { -LIST_HEAD (add_list);int max, pass;acpiphp_rescan_slot (slot);
@@ -512,12 +511,10 @@ static void enable_slot(struct acpiphp_slot *slot, bool bridge)if (pass && dev->subordinate) {check_hotplug_bridge (slot, dev);pcibios_resource_survey_bus (dev->subordinate); -__pci_bus_size_bridges (dev->subordinate, - &add_list); } } } -__pci_bus_assign_resources (bus, &add_list,NULL ); +pci_assign_unassigned_bridge_resources (bus->self); }acpiphp_sanitize_bus (bus);
Signature · 1 line
2.39.1
Quoted 29 lines · Igor Mammedov
When using ACPI PCI hotplug, hotplugging a device with large BARs may fail if bridge windows programmed by firmware are not large enough.
Reproducer:
$ qemu-kvm -monitor stdio -M q35 -m 4G \ -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on \ -device id=rp1,pcie-root-port,bus=pcie.0,chassis=4 \ disk_imagewait till linux guest boots, then hotplug device
(qemu)device_addqxl,bus=rp1hotplug on guest side fails with:
pci 0000:01:00.0: [1b36:0100] type 00 class 0x038000 pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00001fff] pci 0000:01:00.0: reg 0x1c: [io 0x0000-0x001f] pci 0000:01:00.0: BAR 0: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 1: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 1: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 2: assigned [mem 0xfe800000-0xfe801fff] pci 0000:01:00.0: BAR 3: assigned [io 0x1000-0x101f] qxl 0000:01:00.0: enabling device (0000 -> 0003) Unable to create vram_mapping qxl: probe of 0000:01:00.0 failed with error -12
Quoted 6 lines
However when using native PCIe hotplug
'-global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off'
it works fine, since kernel attempts to reassign unused resources. Use the same machinery as native PCIe hotplug to (re)assign resources.Signed-off-by Igor Mammedov <[email protected]>
Quoted 40 lines
---
tested in QEMU with Q35 machine on PCIE root port and also with nested conventional bridge attached to root port.v2:
* fixup subject to match expected style
* drop no longer needed__pci_bus_size_bridges()to avoid memory leak (Bjorn Helgaas <[email protected]>)--- drivers/pci/hotplug/acpiphp_glue.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)drivers/pci/hotplug/acpiphp_glue.c +1
-4@@ -498,7 +498,6 @@ static void enable_slot(struct acpiphp_slot *slot, bool bridge)acpiphp_native_scan_bridge (dev); } }else { -LIST_HEAD (add_list);int max, pass;acpiphp_rescan_slot (slot);@@ -512,12 +511,10 @@ static void enable_slot(struct acpiphp_slot *slot, bool bridge)if (pass && dev->subordinate) {check_hotplug_bridge (slot, dev);pcibios_resource_survey_bus (dev->subordinate); -__pci_bus_size_bridges (dev->subordinate, - &add_list); } } } -__pci_bus_assign_resources (bus, &add_list,NULL ); +pci_assign_unassigned_bridge_resources (bus->self); }acpiphp_sanitize_bus (bus);Signature · 1 line
2.39.1
On Mon, 24 Apr 2023 21:15:57 +0200
Igor Mammedov <[email protected]> wrote:
Bjorn,
I've fixed up issues you've mentioned on v1 review, is it good enough to take it in now?
Quoted 73 lines
When using ACPI PCI hotplug, hotplugging a device with large BARs may fail if bridge windows programmed by firmware are not large enough.
Reproducer:
$ qemu-kvm -monitor stdio -M q35 -m 4G \ -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on \ -device id=rp1,pcie-root-port,bus=pcie.0,chassis=4 \ disk_imagewait till linux guest boots, then hotplug device
(qemu)device_addqxl,bus=rp1hotplug on guest side fails with:
pci 0000:01:00.0: [1b36:0100] type 00 class 0x038000 pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00001fff] pci 0000:01:00.0: reg 0x1c: [io 0x0000-0x001f] pci 0000:01:00.0: BAR 0: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 1: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 1: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 2: assigned [mem 0xfe800000-0xfe801fff] pci 0000:01:00.0: BAR 3: assigned [io 0x1000-0x101f] qxl 0000:01:00.0: enabling device (0000 -> 0003) Unable to create vram_mapping qxl: probe of 0000:01:00.0 failed with error -12However when using native PCIe hotplug
'-global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off'
it works fine, since kernel attempts to reassign unused resources. Use the same machinery as native PCIe hotplug to (re)assign resources.Signed-off-by Igor Mammedov <[email protected]>---
tested in QEMU with Q35 machine on PCIE root port and also with nested conventional bridge attached to root port.v2:
* fixup subject to match expected style
* drop no longer needed__pci_bus_size_bridges()to avoid memory leak (Bjorn Helgaas <[email protected]>)--- drivers/pci/hotplug/acpiphp_glue.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)drivers/pci/hotplug/acpiphp_glue.c +1
-4@@ -498,7 +498,6 @@ static void enable_slot(struct acpiphp_slot *slot, bool bridge)acpiphp_native_scan_bridge (dev); } }else { -LIST_HEAD (add_list);int max, pass;acpiphp_rescan_slot (slot);@@ -512,12 +511,10 @@ static void enable_slot(struct acpiphp_slot *slot, bool bridge)if (pass && dev->subordinate) {check_hotplug_bridge (slot, dev);pcibios_resource_survey_bus (dev->subordinate); -__pci_bus_size_bridges (dev->subordinate, - &add_list); } } } -__pci_bus_assign_resources (bus, &add_list,NULL ); +pci_assign_unassigned_bridge_resources (bus->self); }acpiphp_sanitize_bus (bus);
Quoted 3 lines · On Mon, 24 Apr 2023 21:15:57 +0200 Igor Mammedov <[email protected]> wrote:
When using ACPI PCI hotplug, hotplugging a device with large BARs may fail if bridge windows programmed by firmware are not large enough.
Rafael,
Since there was no more comments for a while,
can you merge this patch through your tree?
Quoted 69 lines
Reproducer:
$ qemu-kvm -monitor stdio -M q35 -m 4G \ -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on \ -device id=rp1,pcie-root-port,bus=pcie.0,chassis=4 \ disk_imagewait till linux guest boots, then hotplug device
(qemu)device_addqxl,bus=rp1hotplug on guest side fails with:
pci 0000:01:00.0: [1b36:0100] type 00 class 0x038000 pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00001fff] pci 0000:01:00.0: reg 0x1c: [io 0x0000-0x001f] pci 0000:01:00.0: BAR 0: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 1: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 1: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 2: assigned [mem 0xfe800000-0xfe801fff] pci 0000:01:00.0: BAR 3: assigned [io 0x1000-0x101f] qxl 0000:01:00.0: enabling device (0000 -> 0003) Unable to create vram_mapping qxl: probe of 0000:01:00.0 failed with error -12However when using native PCIe hotplug
'-global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off'
it works fine, since kernel attempts to reassign unused resources. Use the same machinery as native PCIe hotplug to (re)assign resources.Signed-off-by Igor Mammedov <[email protected]>---
tested in QEMU with Q35 machine on PCIE root port and also with nested conventional bridge attached to root port.v2:
* fixup subject to match expected style
* drop no longer needed__pci_bus_size_bridges()to avoid memory leak (Bjorn Helgaas <[email protected]>)--- drivers/pci/hotplug/acpiphp_glue.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)drivers/pci/hotplug/acpiphp_glue.c +1
-4@@ -498,7 +498,6 @@ static void enable_slot(struct acpiphp_slot *slot, bool bridge)acpiphp_native_scan_bridge (dev); } }else { -LIST_HEAD (add_list);int max, pass;acpiphp_rescan_slot (slot);@@ -512,12 +511,10 @@ static void enable_slot(struct acpiphp_slot *slot, bool bridge)if (pass && dev->subordinate) {check_hotplug_bridge (slot, dev);pcibios_resource_survey_bus (dev->subordinate); -__pci_bus_size_bridges (dev->subordinate, - &add_list); } } } -__pci_bus_assign_resources (bus, &add_list,NULL ); +pci_assign_unassigned_bridge_resources (bus->self); }acpiphp_sanitize_bus (bus);
Rafael J. Wysocki
maintainerQuoted 4 lines · Igor Mammedov
On Mon, 24 Apr 2023 21:15:57 +0200
Igor Mammedov <[email protected]> wrote:
Quoted 8 lines
Quoted 3 lines
When using ACPI PCI hotplug, hotplugging a device with large BARs may fail if bridge windows programmed by firmware are not large enough.
Rafael,
Since there was no more comments for a while,
can you merge this patch through your tree?
I need an ACK from Bjorn on this, as the PCI maintainer, or it can go in via the PCI tree as far as I'm concerned.
Thanks!
Quoted 70 lines
Quoted 69 lines
Reproducer:
$ qemu-kvm -monitor stdio -M q35 -m 4G \ -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on \ -device id=rp1,pcie-root-port,bus=pcie.0,chassis=4 \ disk_imagewait till linux guest boots, then hotplug device
(qemu)device_addqxl,bus=rp1hotplug on guest side fails with:
pci 0000:01:00.0: [1b36:0100] type 00 class 0x038000 pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00001fff] pci 0000:01:00.0: reg 0x1c: [io 0x0000-0x001f] pci 0000:01:00.0: BAR 0: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 1: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 1: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 2: assigned [mem 0xfe800000-0xfe801fff] pci 0000:01:00.0: BAR 3: assigned [io 0x1000-0x101f] qxl 0000:01:00.0: enabling device (0000 -> 0003) Unable to create vram_mapping qxl: probe of 0000:01:00.0 failed with error -12However when using native PCIe hotplug
'-global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off'
it works fine, since kernel attempts to reassign unused resources. Use the same machinery as native PCIe hotplug to (re)assign resources.Signed-off-by Igor Mammedov <[email protected]>---
tested in QEMU with Q35 machine on PCIE root port and also with nested conventional bridge attached to root port.v2:
* fixup subject to match expected style
* drop no longer needed__pci_bus_size_bridges()to avoid memory leak (Bjorn Helgaas <[email protected]>)--- drivers/pci/hotplug/acpiphp_glue.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)drivers/pci/hotplug/acpiphp_glue.c +1
-4@@ -498,7 +498,6 @@ static void enable_slot(struct acpiphp_slot *slot, bool bridge)acpiphp_native_scan_bridge (dev); } }else { -LIST_HEAD (add_list);int max, pass;acpiphp_rescan_slot (slot);@@ -512,12 +511,10 @@ static void enable_slot(struct acpiphp_slot *slot, bool bridge)if (pass && dev->subordinate) {check_hotplug_bridge (slot, dev);pcibios_resource_survey_bus (dev->subordinate); -__pci_bus_size_bridges (dev->subordinate, - &add_list); } } } -__pci_bus_assign_resources (bus, &add_list,NULL ); +pci_assign_unassigned_bridge_resources (bus->self); }acpiphp_sanitize_bus (bus);
Quoted 11 lines · Rafael J. Wysocki
Quoted 10 lines · Igor Mammedov
Quoted 3 lines · On Mon, 24 Apr 2023 21:15:57 +0200 Igor Mammedov <[email protected]> wrote:
When using ACPI PCI hotplug, hotplugging a device with large BARs may fail if bridge windows programmed by firmware are not large enough.
Rafael,
Quoted 7 lines
Quoted 2 lines
Since there was no more comments for a while,
can you merge this patch through your tree?I need an ACK from Bjorn on this, as the PCI maintainer, or it can go in via the PCI tree as far as I'm concerned.
Thanks!
I'd also like an ACK for:
Cc: [email protected]
Quoted 70 lines
Quoted 70 lines
Quoted 69 lines
Reproducer:
$ qemu-kvm -monitor stdio -M q35 -m 4G \ -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on \ -device id=rp1,pcie-root-port,bus=pcie.0,chassis=4 \ disk_imagewait till linux guest boots, then hotplug device
(qemu)device_addqxl,bus=rp1hotplug on guest side fails with:
pci 0000:01:00.0: [1b36:0100] type 00 class 0x038000 pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00001fff] pci 0000:01:00.0: reg 0x1c: [io 0x0000-0x001f] pci 0000:01:00.0: BAR 0: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 1: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 1: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 2: assigned [mem 0xfe800000-0xfe801fff] pci 0000:01:00.0: BAR 3: assigned [io 0x1000-0x101f] qxl 0000:01:00.0: enabling device (0000 -> 0003) Unable to create vram_mapping qxl: probe of 0000:01:00.0 failed with error -12However when using native PCIe hotplug
'-global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off'
it works fine, since kernel attempts to reassign unused resources. Use the same machinery as native PCIe hotplug to (re)assign resources.Signed-off-by Igor Mammedov <[email protected]>---
tested in QEMU with Q35 machine on PCIE root port and also with nested conventional bridge attached to root port.v2:
* fixup subject to match expected style
* drop no longer needed__pci_bus_size_bridges()to avoid memory leak (Bjorn Helgaas <[email protected]>)--- drivers/pci/hotplug/acpiphp_glue.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)drivers/pci/hotplug/acpiphp_glue.c +1
-4@@ -498,7 +498,6 @@ static void enable_slot(struct acpiphp_slot *slot, bool bridge)acpiphp_native_scan_bridge (dev); } }else { -LIST_HEAD (add_list);int max, pass;acpiphp_rescan_slot (slot);@@ -512,12 +511,10 @@ static void enable_slot(struct acpiphp_slot *slot, bool bridge)if (pass && dev->subordinate) {check_hotplug_bridge (slot, dev);pcibios_resource_survey_bus (dev->subordinate); -__pci_bus_size_bridges (dev->subordinate, - &add_list); } } } -__pci_bus_assign_resources (bus, &add_list,NULL ); +pci_assign_unassigned_bridge_resources (bus->self); }acpiphp_sanitize_bus (bus);
Quoted 29 lines · Igor Mammedov
When using ACPI PCI hotplug, hotplugging a device with large BARs may fail if bridge windows programmed by firmware are not large enough.
Reproducer:
$ qemu-kvm -monitor stdio -M q35 -m 4G \ -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on \ -device id=rp1,pcie-root-port,bus=pcie.0,chassis=4 \ disk_imagewait till linux guest boots, then hotplug device
(qemu)device_addqxl,bus=rp1hotplug on guest side fails with:
pci 0000:01:00.0: [1b36:0100] type 00 class 0x038000 pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00001fff] pci 0000:01:00.0: reg 0x1c: [io 0x0000-0x001f] pci 0000:01:00.0: BAR 0: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 1: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 1: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 2: assigned [mem 0xfe800000-0xfe801fff] pci 0000:01:00.0: BAR 3: assigned [io 0x1000-0x101f] qxl 0000:01:00.0: enabling device (0000 -> 0003) Unable to create vram_mapping qxl: probe of 0000:01:00.0 failed with error -12
Quoted 6 lines
However when using native PCIe hotplug
'-global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off'
it works fine, since kernel attempts to reassign unused resources. Use the same machinery as native PCIe hotplug to (re)assign resources.Signed-off-by Igor Mammedov <[email protected]>
Tentatively applied to pci/hotplug for v6.5.
Tentative only because I'm hoping for your ack or review, Rafael (I see you acked v1, but this one is a little different).
And I think you'd like a stable tag, Michael? I'm happy to add one, but wasn't sure if you wanted a separate ack for that.
Quoted 41 lines
---
tested in QEMU with Q35 machine on PCIE root port and also with nested conventional bridge attached to root port.v2:
* fixup subject to match expected style
* drop no longer needed__pci_bus_size_bridges()to avoid memory leak (Bjorn Helgaas <[email protected]>)--- drivers/pci/hotplug/acpiphp_glue.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)drivers/pci/hotplug/acpiphp_glue.c +1
-4@@ -498,7 +498,6 @@ static void enable_slot(struct acpiphp_slot *slot, bool bridge)acpiphp_native_scan_bridge (dev); } }else { -LIST_HEAD (add_list);int max, pass;acpiphp_rescan_slot (slot);@@ -512,12 +511,10 @@ static void enable_slot(struct acpiphp_slot *slot, bool bridge)if (pass && dev->subordinate) {check_hotplug_bridge (slot, dev);pcibios_resource_survey_bus (dev->subordinate); -__pci_bus_size_bridges (dev->subordinate, - &add_list); } } } -__pci_bus_assign_resources (bus, &add_list,NULL ); +pci_assign_unassigned_bridge_resources (bus->self); }acpiphp_sanitize_bus (bus);Signature · 1 line
2.39.1
Quoted 36 lines · Bjorn Helgaas
Quoted 34 lines · Igor Mammedov
When using ACPI PCI hotplug, hotplugging a device with large BARs may fail if bridge windows programmed by firmware are not large enough.
Reproducer:
$ qemu-kvm -monitor stdio -M q35 -m 4G \ -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on \ -device id=rp1,pcie-root-port,bus=pcie.0,chassis=4 \ disk_imagewait till linux guest boots, then hotplug device
(qemu)device_addqxl,bus=rp1hotplug on guest side fails with:
pci 0000:01:00.0: [1b36:0100] type 00 class 0x038000 pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00001fff] pci 0000:01:00.0: reg 0x1c: [io 0x0000-0x001f] pci 0000:01:00.0: BAR 0: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 1: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 1: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 2: assigned [mem 0xfe800000-0xfe801fff] pci 0000:01:00.0: BAR 3: assigned [io 0x1000-0x101f] qxl 0000:01:00.0: enabling device (0000 -> 0003) Unable to create vram_mapping qxl: probe of 0000:01:00.0 failed with error -12However when using native PCIe hotplug
'-global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off'
it works fine, since kernel attempts to reassign unused resources. Use the same machinery as native PCIe hotplug to (re)assign resources.
Quoted 6 lines
Quoted 1 line
Signed-off-by Igor Mammedov <[email protected]>Tentatively applied to pci/hotplug for v6.5.
Tentative only because I'm hoping for your ack or review, Rafael (I see you acked v1, but this one is a little different).
This one is fine with me too, so please feel free to add
to it.
Quoted 44 lines
And I think you'd like a stable tag, Michael? I'm happy to add one, but wasn't sure if you wanted a separate ack for that.
Quoted 41 lines
---
tested in QEMU with Q35 machine on PCIE root port and also with nested conventional bridge attached to root port.v2:
* fixup subject to match expected style
* drop no longer needed__pci_bus_size_bridges()to avoid memory leak (Bjorn Helgaas <[email protected]>)--- drivers/pci/hotplug/acpiphp_glue.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)drivers/pci/hotplug/acpiphp_glue.c +1
-5@@ -498,7 +498,6 @@ static void enable_slot(struct acpiphp_slot *slot, bool bridge)acpiphp_native_scan_bridge (dev); } }else { -LIST_HEAD (add_list);int max, pass;acpiphp_rescan_slot (slot);@@ -512,12 +511,10 @@ static void enable_slot(struct acpiphp_slot *slot, bool bridge)if (pass && dev->subordinate) {check_hotplug_bridge (slot, dev);pcibios_resource_survey_bus (dev->subordinate); -__pci_bus_size_bridges (dev->subordinate, - &add_list); } } } -__pci_bus_assign_resources (bus, &add_list,NULL ); +pci_assign_unassigned_bridge_resources (bus->self); }acpiphp_sanitize_bus (bus); --2.39.1
Quoted 37 lines · Bjorn Helgaas
Quoted 29 lines · Igor Mammedov
When using ACPI PCI hotplug, hotplugging a device with large BARs may fail if bridge windows programmed by firmware are not large enough.
Reproducer:
$ qemu-kvm -monitor stdio -M q35 -m 4G \ -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on \ -device id=rp1,pcie-root-port,bus=pcie.0,chassis=4 \ disk_imagewait till linux guest boots, then hotplug device
(qemu)device_addqxl,bus=rp1hotplug on guest side fails with:
pci 0000:01:00.0: [1b36:0100] type 00 class 0x038000 pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00001fff] pci 0000:01:00.0: reg 0x1c: [io 0x0000-0x001f] pci 0000:01:00.0: BAR 0: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 1: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 1: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 2: assigned [mem 0xfe800000-0xfe801fff] pci 0000:01:00.0: BAR 3: assigned [io 0x1000-0x101f] qxl 0000:01:00.0: enabling device (0000 -> 0003) Unable to create vram_mapping qxl: probe of 0000:01:00.0 failed with error -12Quoted 6 lines
However when using native PCIe hotplug
'-global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off'
it works fine, since kernel attempts to reassign unused resources. Use the same machinery as native PCIe hotplug to (re)assign resources.Signed-off-by Igor Mammedov <[email protected]>
Quoted 7 lines
Tentatively applied to pci/hotplug for v6.5.
Tentative only because I'm hoping for your ack or review, Rafael (I see you acked v1, but this one is a little different).
And I think you'd like a stable tag, Michael? I'm happy to add one, but wasn't sure if you wanted a separate ack for that.
I'd like a stable tag, yes. Just was not sure whom best to ask.
Quoted 41 lines
Quoted 41 lines
---
tested in QEMU with Q35 machine on PCIE root port and also with nested conventional bridge attached to root port.v2:
* fixup subject to match expected style
* drop no longer needed__pci_bus_size_bridges()to avoid memory leak (Bjorn Helgaas <[email protected]>)--- drivers/pci/hotplug/acpiphp_glue.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)drivers/pci/hotplug/acpiphp_glue.c +1
-4@@ -498,7 +498,6 @@ static void enable_slot(struct acpiphp_slot *slot, bool bridge)acpiphp_native_scan_bridge (dev); } }else { -LIST_HEAD (add_list);int max, pass;acpiphp_rescan_slot (slot);@@ -512,12 +511,10 @@ static void enable_slot(struct acpiphp_slot *slot, bool bridge)if (pass && dev->subordinate) {check_hotplug_bridge (slot, dev);pcibios_resource_survey_bus (dev->subordinate); -__pci_bus_size_bridges (dev->subordinate, - &add_list); } } } -__pci_bus_assign_resources (bus, &add_list,NULL ); +pci_assign_unassigned_bridge_resources (bus->self); }acpiphp_sanitize_bus (bus);Signature · 1 line
2.39.1
Quoted 37 lines · Bjorn Helgaas
Quoted 29 lines · Igor Mammedov
When using ACPI PCI hotplug, hotplugging a device with large BARs may fail if bridge windows programmed by firmware are not large enough.
Reproducer:
$ qemu-kvm -monitor stdio -M q35 -m 4G \ -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on \ -device id=rp1,pcie-root-port,bus=pcie.0,chassis=4 \ disk_imagewait till linux guest boots, then hotplug device
(qemu)device_addqxl,bus=rp1hotplug on guest side fails with:
pci 0000:01:00.0: [1b36:0100] type 00 class 0x038000 pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00001fff] pci 0000:01:00.0: reg 0x1c: [io 0x0000-0x001f] pci 0000:01:00.0: BAR 0: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 1: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 1: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 2: assigned [mem 0xfe800000-0xfe801fff] pci 0000:01:00.0: BAR 3: assigned [io 0x1000-0x101f] qxl 0000:01:00.0: enabling device (0000 -> 0003) Unable to create vram_mapping qxl: probe of 0000:01:00.0 failed with error -12Quoted 6 lines
However when using native PCIe hotplug
'-global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off'
it works fine, since kernel attempts to reassign unused resources. Use the same machinery as native PCIe hotplug to (re)assign resources.Signed-off-by Igor Mammedov <[email protected]>
Quoted 7 lines
Tentatively applied to pci/hotplug for v6.5.
Tentative only because I'm hoping for your ack or review, Rafael (I see you acked v1, but this one is a little different).
And I think you'd like a stable tag, Michael? I'm happy to add one, but wasn't sure if you wanted a separate ack for that.
Added stable tag and Rafael's ack. All set for v6.5, thanks!
Quoted 41 lines
Quoted 41 lines
---
tested in QEMU with Q35 machine on PCIE root port and also with nested conventional bridge attached to root port.v2:
* fixup subject to match expected style
* drop no longer needed__pci_bus_size_bridges()to avoid memory leak (Bjorn Helgaas <[email protected]>)--- drivers/pci/hotplug/acpiphp_glue.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)drivers/pci/hotplug/acpiphp_glue.c +1
-4@@ -498,7 +498,6 @@ static void enable_slot(struct acpiphp_slot *slot, bool bridge)acpiphp_native_scan_bridge (dev); } }else { -LIST_HEAD (add_list);int max, pass;acpiphp_rescan_slot (slot);@@ -512,12 +511,10 @@ static void enable_slot(struct acpiphp_slot *slot, bool bridge)if (pass && dev->subordinate) {check_hotplug_bridge (slot, dev);pcibios_resource_survey_bus (dev->subordinate); -__pci_bus_size_bridges (dev->subordinate, - &add_list); } } } -__pci_bus_assign_resources (bus, &add_list,NULL ); +pci_assign_unassigned_bridge_resources (bus->self); }acpiphp_sanitize_bus (bus);Signature · 1 line
2.39.1
Quoted 14 lines · Igor Mammedov
When using ACPI PCI hotplug, hotplugging a device with large BARs may fail if bridge windows programmed by firmware are not large enough.
Reproducer:
$ qemu-kvm -monitor stdio -M q35 -m 4G \ -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on \ -device id=rp1,pcie-root-port,bus=pcie.0,chassis=4 \ disk_imagewait till linux guest boots, then hotplug device
(qemu)device_addqxl,bus=rp1hotplug on guest side fails with:
Quoted 12 lines
pci 0000:01:00.0: [1b36:0100] type 00 class 0x038000 pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00001fff] pci 0000:01:00.0: reg 0x1c: [io 0x0000-0x001f] pci 0000:01:00.0: BAR 0: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 1: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 1: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 2: assigned [mem 0xfe800000-0xfe801fff] pci 0000:01:00.0: BAR 3: assigned [io 0x1000-0x101f] qxl 0000:01:00.0: enabling device (0000 -> 0003)
Ugh, I just noticed that we turned on PCI_COMMAND_MEMORY even though BARs 0 and 1 haven't been assigned. How did that happen? It looks like pci_enable_resources() checks for that, but there must be a hole somewhere.
Quoted 2 lines
Unable to create
vram_mapping
qxl: probe of 0000:01:00.0 failed with error -12
Quoted 28 lines · Bjorn Helgaas
Quoted 14 lines · Igor Mammedov
When using ACPI PCI hotplug, hotplugging a device with large BARs may fail if bridge windows programmed by firmware are not large enough.
Reproducer:
$ qemu-kvm -monitor stdio -M q35 -m 4G \ -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on \ -device id=rp1,pcie-root-port,bus=pcie.0,chassis=4 \ disk_imagewait till linux guest boots, then hotplug device
(qemu)device_addqxl,bus=rp1hotplug on guest side fails with:
Quoted 12 lines
pci 0000:01:00.0: [1b36:0100] type 00 class 0x038000 pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00001fff] pci 0000:01:00.0: reg 0x1c: [io 0x0000-0x001f] pci 0000:01:00.0: BAR 0: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 1: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 1: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 2: assigned [mem 0xfe800000-0xfe801fff] pci 0000:01:00.0: BAR 3: assigned [io 0x1000-0x101f] qxl 0000:01:00.0: enabling device (0000 -> 0003)
Quoted 4 lines
Ugh, I just noticed that we turned on
PCI_COMMAND_MEMORYeven though BARs 0 and 1 haven't been assigned. How did that happen? It looks likepci_enable_resources()checks for that, but there must be a hole somewhere.
Maybe because BAR2 was assigned? I think pci_enable_resources just does
if (r->flags & IORESOURCE_MEM)
cmd |= PCI_COMMAND_MEMORY;in a loop so if any memory BARs are assigned then PCI_COMMAND_MEMORY is set.
Quoted 2 lines
Quoted 2 lines
Unable to create
vram_mapping
qxl: probe of 0000:01:00.0 failed with error -12
Quoted 34 lines · Michael S. Tsirkin
Quoted 28 lines · Bjorn Helgaas
Quoted 14 lines · Igor Mammedov
When using ACPI PCI hotplug, hotplugging a device with large BARs may fail if bridge windows programmed by firmware are not large enough.
Reproducer:
$ qemu-kvm -monitor stdio -M q35 -m 4G \ -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on \ -device id=rp1,pcie-root-port,bus=pcie.0,chassis=4 \ disk_imagewait till linux guest boots, then hotplug device
(qemu)device_addqxl,bus=rp1hotplug on guest side fails with:
Quoted 12 lines
pci 0000:01:00.0: [1b36:0100] type 00 class 0x038000 pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00001fff] pci 0000:01:00.0: reg 0x1c: [io 0x0000-0x001f] pci 0000:01:00.0: BAR 0: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 1: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 1: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 2: assigned [mem 0xfe800000-0xfe801fff] pci 0000:01:00.0: BAR 3: assigned [io 0x1000-0x101f] qxl 0000:01:00.0: enabling device (0000 -> 0003)Quoted 4 lines
Ugh, I just noticed that we turned on
PCI_COMMAND_MEMORYeven though BARs 0 and 1 haven't been assigned. How did that happen? It looks likepci_enable_resources()checks for that, but there must be a hole somewhere.
Quoted 6 lines
Maybe because BAR2 was assigned? I think
pci_enable_resourcesjust doesif (r->flags & IORESOURCE_MEM) cmd |= PCI_COMMAND_MEMORY;in a loop so if any memory BARs are assigned then
PCI_COMMAND_MEMORYis set.
It does, but it also bails out if it finds IORESOURCE_UNSET:
pci_enable_resources()
{
...
pci_dev_for_each_resource(dev, r, i) {
...
if (r->flags & IORESOURCE_UNSET) {
pci_err(dev, "can't enable device: BAR %d %pR not assigned\n");
return -EINVAL;
}
...
if (r->flags & IORESOURCE_MEM)
cmd |= PCI_COMMAND_MEMORY;
}
...
}I expected that IORESOURCE_UNSET would still be there from pci_assign_resource(), since we saw the "failed to assign" messages, but there must be more going on.
Bjorn
Quoted 60 lines · Bjorn Helgaas
Quoted 34 lines · Michael S. Tsirkin
Quoted 28 lines · Bjorn Helgaas
Quoted 14 lines · Igor Mammedov
When using ACPI PCI hotplug, hotplugging a device with large BARs may fail if bridge windows programmed by firmware are not large enough.
Reproducer:
$ qemu-kvm -monitor stdio -M q35 -m 4G \ -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on \ -device id=rp1,pcie-root-port,bus=pcie.0,chassis=4 \ disk_imagewait till linux guest boots, then hotplug device
(qemu)device_addqxl,bus=rp1hotplug on guest side fails with:
Quoted 12 lines
pci 0000:01:00.0: [1b36:0100] type 00 class 0x038000 pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00001fff] pci 0000:01:00.0: reg 0x1c: [io 0x0000-0x001f] pci 0000:01:00.0: BAR 0: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 1: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 1: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 2: assigned [mem 0xfe800000-0xfe801fff] pci 0000:01:00.0: BAR 3: assigned [io 0x1000-0x101f] qxl 0000:01:00.0: enabling device (0000 -> 0003)Quoted 4 lines
Ugh, I just noticed that we turned on
PCI_COMMAND_MEMORYeven though BARs 0 and 1 haven't been assigned. How did that happen? It looks likepci_enable_resources()checks for that, but there must be a hole somewhere.Quoted 6 lines
Maybe because BAR2 was assigned? I think
pci_enable_resourcesjust doesif (r->flags & IORESOURCE_MEM) cmd |= PCI_COMMAND_MEMORY;in a loop so if any memory BARs are assigned then
PCI_COMMAND_MEMORYis set.It does, but it also bails out if it finds
IORESOURCE_UNSET:pci_enable_resources() { ... pci_dev_for_each_resource(dev, r, i) { ... if (r->flags & IORESOURCE_UNSET) { pci_err(dev, "can't enable device: BAR %d %pR not assigned\n"); return -EINVAL; } ... if (r->flags & IORESOURCE_MEM) cmd |= PCI_COMMAND_MEMORY; } ... }
Quoted 5 lines
I expected that
IORESOURCE_UNSETwould still be there frompci_assign_resource(), since we saw the "failed to assign" messages, but there must be more going on.Bjorn
Oh you are right. I donnu.
Signature · 1 line
MST
Quoted 60 lines · On Tue, 30 May 2023 13:55:12 -0500 Bjorn Helgaas <[email protected]> wrote:
Quoted 34 lines · Michael S. Tsirkin
Quoted 28 lines · Bjorn Helgaas
Quoted 14 lines · Igor Mammedov
When using ACPI PCI hotplug, hotplugging a device with large BARs may fail if bridge windows programmed by firmware are not large enough.
Reproducer:
$ qemu-kvm -monitor stdio -M q35 -m 4G \ -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on \ -device id=rp1,pcie-root-port,bus=pcie.0,chassis=4 \ disk_imagewait till linux guest boots, then hotplug device
(qemu)device_addqxl,bus=rp1hotplug on guest side fails with:
Quoted 12 lines
pci 0000:01:00.0: [1b36:0100] type 00 class 0x038000 pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x03ffffff] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00001fff] pci 0000:01:00.0: reg 0x1c: [io 0x0000-0x001f] pci 0000:01:00.0: BAR 0: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 1: no space for [mem size 0x04000000] pci 0000:01:00.0: BAR 1: failed to assign [mem size 0x04000000] pci 0000:01:00.0: BAR 2: assigned [mem 0xfe800000-0xfe801fff] pci 0000:01:00.0: BAR 3: assigned [io 0x1000-0x101f] qxl 0000:01:00.0: enabling device (0000 -> 0003)Quoted 4 lines
Ugh, I just noticed that we turned on
PCI_COMMAND_MEMORYeven though BARs 0 and 1 haven't been assigned. How did that happen? It looks likepci_enable_resources()checks for that, but there must be a hole somewhere.Quoted 6 lines
Maybe because BAR2 was assigned? I think
pci_enable_resourcesjust doesif (r->flags & IORESOURCE_MEM) cmd |= PCI_COMMAND_MEMORY;in a loop so if any memory BARs are assigned then
PCI_COMMAND_MEMORYis set.It does, but it also bails out if it finds
IORESOURCE_UNSET:pci_enable_resources() { ... pci_dev_for_each_resource(dev, r, i) { ... if (r->flags & IORESOURCE_UNSET) { pci_err(dev, "can't enable device: BAR %d %pR not assigned\n"); return -EINVAL; } ... if (r->flags & IORESOURCE_MEM) cmd |= PCI_COMMAND_MEMORY; } ... }
Quoted 3 lines
I expected that
IORESOURCE_UNSETwould still be there frompci_assign_resource(), since we saw the "failed to assign" messages, but there must be more going on.
with current acpiphp code pci_assign_resource() isn't called, instead it goes __pci_bus_assign_resources() route.
However I an reproduce similar issue with SHPC when using hierarchy deeper than 1 bridge (for which relocation has never worked)
qemu-kvm -monitor stdio -M q35 -cpu host -enable-kvm -m 4G \
-global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off \
-device pcie-root-port,id=pr1,bus=pcie.0,chassis=4 \
-device pcie-pci-bridge,id=br1,bus=pr1
hotplug device
(qemu) device_add qxl,addr=1,bus=br1
shpchp 0000:01:00.0: Latch close on Slot(1)
shpchp 0000:01:00.0: Button pressed on Slot(1)
shpchp 0000:01:00.0: Card present on Slot(1)
shpchp 0000:01:00.0: PCI slot #1 - powering on due to button press pci 0000:02:01.0: [1b36:0100] type 00 class 0x038000 pci 0000:02:01.0: reg 0x10: [mem 0x00000000-0x03ffffff] pci 0000:02:01.0: reg 0x14: [mem 0x00000000-0x03ffffff] pci 0000:02:01.0: reg 0x18: [mem 0x00000000-0x00001fff] pci 0000:02:01.0: reg 0x1c: [io 0x0000-0x001f]
pci 0000:02:01.0: BAR 0: no space for [mem size 0x04000000] pci 0000:02:01.0: BAR 0: failed to assign [mem size 0x04000000] pci 0000:02:01.0: BAR 1: no space for [mem size 0x04000000] pci 0000:02:01.0: BAR 1: failed to assign [mem size 0x04000000] pci 0000:02:01.0: BAR 2: assigned [mem 0xfe600000-0xfe601fff] pci 0000:02:01.0: BAR 3: assigned [io 0xc000-0xc01f]
^^^^^^^^^^^ shpchp 0000:01:00.0: PCI bridge to [bus 02] shpchp 0000:01:00.0: bridge window [io 0xc000-0xcfff] shpchp 0000:01:00.0: bridge window [mem 0xfe600000-0xfe7fffff] shpchp 0000:01:00.0: bridge window [mem 0xfe000000-0xfe1fffff 64bit pref]
PCI: No. 2 try to assign unassigned res
release child resource [mem 0xfe600000-0xfe601fff]
shpchp 0000:01:00.0: resource 8 [mem 0xfe600000-0xfe7fffff] released shpchp 0000:01:00.0: PCI bridge to [bus 02]
shpchp 0000:01:00.0: BAR 8: no space for [mem size 0x0a000000] shpchp 0000:01:00.0: BAR 8: failed to assign [mem size 0x0a000000] pci 0000:02:01.0: BAR 0: no space for [mem size 0x04000000] pci 0000:02:01.0: BAR 0: failed to assign [mem size 0x04000000] pci 0000:02:01.0: BAR 1: no space for [mem size 0x04000000] pci 0000:02:01.0: BAR 1: failed to assign [mem size 0x04000000] pci 0000:02:01.0: BAR 2: no space for [mem size 0x00002000] pci 0000:02:01.0: BAR 2: failed to assign [mem size 0x00002000] shpchp 0000:01:00.0: PCI bridge to [bus 02]
shpchp 0000:01:00.0: bridge window [io 0xc000-0xcfff]
shpchp 0000:01:00.0: bridge window [mem 0xfe000000-0xfe1fffff 64bit pref]
qxl 0000:02:01.0: enabling device (0000 -> 0001)
^^^ IO res only
where:
assign_requested_resources_sorted()
...
if (pci_assign_resource())
reset_resource(res);reset wipes everything pci_assign_resource() has done for failing resources leaving only assigned IO (on the 1st pass).
Then later pci_enable_device_flags() will build mask for available bars
for (i = 0; i <= PCI_ROM_RESOURCE; i++)
if (dev->resource[i].flags & flags)
bars |= (1 << i);
for (i = PCI_BRIDGE_RESOURCES; i < DEVICE_COUNT_RESOURCE; i++)
if (dev->resource[i].flags & flags)
bars |= (1 << i);however since for failed MEM resources reset cleared flags along with everything else, above snippet will ignore MEM bars, leaving only assigned IO resource. Then
do_pci_enable_device() -> pcibios_enable_device() -> pci_enable_resources(,bars)
will happily succeed since mask tells it only to look into IO.
And well even if mask weren't excluding MEM ones, it won't help since the resource was cleared out in assign_requested_resources_sorted().
Perhaps instead of playing with flags, we should somehow mark device with unusable resources as disabled, and fail pci_enable_device() early. (and also make sure its resources aren't accounted anymore on follow up hotplug events to the bridge)