pci: Add subordinate check before pci_add_new_bus()
PATCH 1/1 pci: Add subordinate check before pci_add_new_bus()
For preconfigured PCI bridge, child bus created on the first scan. While for some reasons(e.g register mutation), the secondary, and subordiante register reset to 0 on the second scan, which caused to create PCI bus twice for the same PCI device.
Following is the related log:
[Wed May 28 20:38:36 CST 2025] pci 0000:0b:01.0: PCI bridge to [bus 0d] [Wed May 28 20:38:36 CST 2025] pci 0000:0b:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring [Wed May 28 20:38:36 CST 2025] pci 0000:0b:01.0: PCI bridge to [bus 0e-10] [Wed May 28 20:38:36 CST 2025] pci 0000:0b:05.0: PCI bridge to [bus 0f-10]
Here PCI device 000:0b:01.0 assigend to bus 0d and 0e.
This patch checks if child PCI bus has been created on the second scan of bridge. If yes, return directly instead of create a new one.
--- drivers/pci/probe.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/pci/probe.c +3 -0
@@ -1444,6 +1444,9 @@ static int pci_scan_bridge_extend(struct pci_bus *bus, struct pci_dev *dev, goto out; } +if (pci_has_subordinate (dev)) + goto out; +/* Clear errors */ pci_write_config_word (dev,PCI_STATUS ,0xffff );
Signature · 1 line
2.43.0
Quoted 4 lines · Rui He
For preconfigured PCI bridge, child bus created on the first scan. While for some reasons(e.g register mutation), the secondary, and subordiante register reset to 0 on the second scan, which caused to create PCI bus twice for the same PCI device.
I don't quite follow this. Do you mean something is changing the bridge configuration between the first and second scans?
Quoted 5 lines
Following is the related log:
[Wed May 28 20:38:36 CST 2025] pci 0000:0b:01.0: PCI bridge to [bus 0d] [Wed May 28 20:38:36 CST 2025] pci 0000:0b:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring [Wed May 28 20:38:36 CST 2025] pci 0000:0b:01.0: PCI bridge to [bus 0e-10] [Wed May 28 20:38:36 CST 2025] pci 0000:0b:05.0: PCI bridge to [bus 0f-10]
Drop the timestamps (since they don't contribute to understanding the problem) and indent the logs a couple spaces.
Quoted 1 line
Here PCI device 000:0b:01.0 assigend to bus 0d and 0e.
It looks like the [bus 0f-10] range is assigned to both bridges (0b:01.0 and 0b:05.0), which would definitely be a problem.
I'm surprised that we haven't tripped over this before, and I'm curious about how we got here. Can you set CONFIG_DYNAMIC_DEBUG=y, boot with the dyndbg="file drivers/pci/* +p" kernel parameter, and collect the complete dmesg log?
Quoted 13 lines
This patch checks if child PCI bus has been created on the second scan of bridge. If yes, return directly instead of create a new one.
Signed-off-by Rui He <[email protected]>--- drivers/pci/probe.c | 3 +++ 1 file changed, 3 insertions(+)drivers/pci/probe.c +0
-0
drivers/pci/probe.c static int pci_scan_bridge_extend(struct pci_bus *bus, struct pci_dev *dev, · line 1448
Quoted 5 lines
drivers/pci/probe.c +2
-0@@ -1444,6 +1444,9 @@ static int pci_scan_bridge_extend(struct pci_bus *bus, struct pci_dev *dev, goto out; } +if (pci_has_subordinate (dev)) + goto out;
Follow the coding style, i.e., add a space in "if (pci_..."
Quoted 6 lines
drivers/pci/probe.c +0
-0/* Clear errors */ pci_write_config_word (dev,PCI_STATUS ,0xffff );Signature · 1 line
2.43.0
Quoted 17 lines
-----Original Message-----
From: Bjorn Helgaas <[email protected]>
Sent: 2025年8月15日 4:36
To: He, Rui <[email protected]>
Cc: Bjorn Helgaas <[email protected]>; [email protected]; [email protected]; Chikhalkar, Prashant <[email protected]>; Xiao, Jiguang <[email protected]>
Subject: Re: [PATCH 1/1] pci: Add subordinate check beforepci_add_new_bus()CAUTION: This email comes from a non Wind River email account! Do not click links or open attachments unless you recognize the sender and know the content is safe.
Quoted 8 lines
Quoted 4 lines · Rui He
For preconfigured PCI bridge, child bus created on the first scan. While for some reasons(e.g register mutation), the secondary, and subordiante register reset to 0 on the second scan, which caused to create PCI bus twice for the same PCI device.
I don't quite follow this. Do you mean something is changing the bridge configuration between the first and second scans?
I'm not sure what changed the bridge configuration, but the secondary and subordinate is indeed 0 on the second scan as [bus 0e-10] created for 0000:0b:01.0.
In my opinion, it might be an invalid communication or register mutation in PCI bridge.
Quoted 11 lines
Quoted 7 lines
Following is the related log:
[Wed May 28 20:38:36 CST 2025] pci 0000:0b:01.0: PCI bridge to [bus 0d] [Wed May 28 20:38:36 CST 2025] pci 0000:0b:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring [Wed May 28 20:38:36 CST 2025] pci 0000:0b:01.0: PCI bridge to [bus 0e-10] [Wed May 28 20:38:36 CST 2025] pci 0000:0b:05.0: PCI bridge to [bus 0f-10]Drop the timestamps (since they don't contribute to understanding the problem) and indent the logs a couple spaces.
OK
Quoted 10 lines
Quoted 1 line
Here PCI device 000:0b:01.0 assigend to bus 0d and 0e.
It looks like the [bus 0f-10] range is assigned to both bridges (0b:01.0 and 0b:05.0), which would definitely be a problem.
I'm surprised that we haven't tripped over this before, and I'm curious about how we got here. Can you set
CONFIG_DYNAMIC_DEBUG=y, boot with the dyndbg="filedrivers/pci/*+p" kernel parameter, and collect the complete dmesg log?
Sorry, as this is a individual issue, and cannot be reproduced, I cannot offer more detailed logs.
Quoted 14 lines
drivers/pci/probe.c static int pci_scan_bridge_extend(struct pci_bus
Quoted 13 lines
This patch checks if child PCI bus has been created on the second scan of bridge. If yes, return directly instead of create a new one.
Signed-off-by Rui He <[email protected]>--- drivers/pci/probe.c | 3 +++ 1 file changed, 3 insertions(+)drivers/pci/probe.c index renamed from drivers/pci/probe.c +0
-0
f41128f91ca76..ec67adbf31738100644drivers/pci/probe.c +0
-0@@ -1444,6 +1444,9 @@ static int pci_scan_bridge_extend(struct pci_bus*bus, struct
pci_dev*dev,
Quoted 7 lines
drivers/pci/probe.c static int pci_scan_bridge_extend(struct pci_bus *bus, struct pci_dev *dev, · line 1448
Quoted 5 lines
drivers/pci/probe.c +2
-0goto out; } +if (pci_has_subordinate (dev)) + goto out;Follow the coding style, i.e., add a space in "if (
pci_..."
Will be changed in v2.
Quoted 7 lines
Quoted 7 lines
drivers/pci/probe.c +0
-1/* Clear errors */ pci_write_config_word (dev,PCI_STATUS ,0xffff ); --2.43.0
Quoted 43 lines · Rui
Quoted 15 lines
-----Original Message-----
From: Bjorn Helgaas <[email protected]>
Sent: 2025年8月15日 4:36
To: He, Rui <[email protected]>
Cc: Bjorn Helgaas <[email protected]>; [email protected]; [email protected]; Chikhalkar, Prashant <[email protected]>; Xiao, Jiguang <[email protected]>
Subject: Re: [PATCH 1/1] pci: Add subordinate check beforepci_add_new_bus()CAUTION: This email comes from a non Wind River email account! Do not click links or open attachments unless you recognize the sender and know the content is safe.
Quoted 8 lines
Quoted 4 lines · Rui He
For preconfigured PCI bridge, child bus created on the first scan. While for some reasons(e.g register mutation), the secondary, and subordiante register reset to 0 on the second scan, which caused to create PCI bus twice for the same PCI device.
I don't quite follow this. Do you mean something is changing the bridge configuration between the first and second scans?
I'm not sure what changed the bridge configuration, but the secondary and subordinate is indeed 0 on the second scan as [bus 0e-10] created for 0000:0b:01.0.
In my opinion, it might be an invalid communication or register mutation in PCI bridge.
Quoted 12 lines
Quoted 8 lines
Following is the related log:
[Wed May 28 20:38:36 CST 2025] pci 0000:0b:01.0: PCI bridge to [bus 0d] [Wed May 28 20:38:36 CST 2025] pci 0000:0b:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring [Wed May 28 20:38:36 CST 2025] pci 0000:0b:01.0: PCI bridge to [bus 0e-10] [Wed May 28 20:38:36 CST 2025] pci 0000:0b:05.0: PCI bridge to [bus 0f-10]Here PCI device 000:0b:01.0 assigend to bus 0d and 0e.
It looks like the [bus 0f-10] range is assigned to both bridges (0b:01.0 and 0b:05.0), which would definitely be a problem.
Quoted 7 lines
Quoted 4 lines
I'm surprised that we haven't tripped over this before, and I'm curious about how we got here. Can you set
CONFIG_DYNAMIC_DEBUG=y, boot with the dyndbg="filedrivers/pci/*+p" kernel parameter, and collect the complete dmesg log?Sorry, as this is a individual issue, and cannot be reproduced, I cannot offer more detailed logs.
Do you have the complete dmesg log from this one time you saw the problem?
As-is, I don't think there's quite enough here to move forward with this. I think we need some more detailed analysis to figure out how this happens.
Bjorn
Quoted 69 lines
-----Original Message-----
From: Bjorn Helgaas <[email protected]>
Sent: 2025年8月15日 22:23
To: He, Rui <[email protected]>
Cc: Bjorn Helgaas <[email protected]>; [email protected]; [email protected]; Chikhalkar, Prashant <[email protected]>; Xiao, Jiguang <[email protected]>
Subject: Re: [PATCH 1/1] pci: Add subordinate check beforepci_add_new_bus()CAUTION: This email comes from a non Wind River email account!
Do not click links or open attachments unless you recognize the sender and know the content is safe.Quoted 44 lines · Rui
Quoted 15 lines
-----Original Message-----
From: Bjorn Helgaas <[email protected]>
Sent: 2025年8月15日 4:36
To: He, Rui <[email protected]>
Cc: Bjorn Helgaas <[email protected]>; [email protected]; [email protected]; Chikhalkar, Prashant <[email protected]>; Xiao, Jiguang <[email protected]>
Subject: Re: [PATCH 1/1] pci: Add subordinate check beforepci_add_new_bus()CAUTION: This email comes from a non Wind River email account! Do not click links or open attachments unless you recognize the sender and know the content is safe.
Quoted 8 lines
Quoted 4 lines · Rui He
For preconfigured PCI bridge, child bus created on the first scan. While for some reasons(e.g register mutation), the secondary, and subordiante register reset to 0 on the second scan, which caused to create PCI bus twice for the same PCI device.
I don't quite follow this. Do you mean something is changing the bridge configuration between the first and second scans?
I'm not sure what changed the bridge configuration, but the secondary and subordinate is indeed 0 on the second scan as [bus 0e-10] created for 0000:0b:01.0.
In my opinion, it might be an invalid communication or register mutation in PCI bridge.
Quoted 13 lines
Quoted 9 lines
Following is the related log:
[Wed May 28 20:38:36 CST 2025] pci 0000:0b:01.0: PCI bridge to [bus 0d] [Wed May 28 20:38:36 CST 2025] pci 0000:0b:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring [Wed May 28 20:38:36 CST 2025] pci 0000:0b:01.0: PCI bridge to [bus 0e-10] [Wed May 28 20:38:36 CST 2025] pci 0000:0b:05.0: PCI bridge to[bus 0f-10]
Here PCI device 000:0b:01.0 assigend to bus 0d and 0e.
It looks like the [bus 0f-10] range is assigned to both bridges (0b:01.0 and 0b:05.0), which would definitely be a problem.
Quoted 7 lines
Quoted 4 lines
I'm surprised that we haven't tripped over this before, and I'm curious about how we got here. Can you set
CONFIG_DYNAMIC_DEBUG=y, boot with the dyndbg="filedrivers/pci/*
+p" kernel parameter, and collect the complete dmesg log?Sorry, as this is a individual issue, and cannot be reproduced, I cannot offer more detailed logs.
Quoted 6 lines
Do you have the complete dmesg log from this one time you saw the problem?
As-is, I don't think there's quite enough here to move forward with this. I think we need some more detailed analysis to figure out how this happens.
Bjorn
Attached is the dmesg logs while scan the PCI bus.
As the dmesg is customer sensitive, I have removed irrelevant logs and only kept the PCI enumeration logs that can be obtained.
Among that, "Quirks: Set bus range to 0xAABBCC" is printed by the custom hook declared through DECLARE_PCI_FIXUP_EARLY() in drvivers/pci/quirks.c. 0xAABBCC refers to the predefined PIC bridge Subordinate Bus Number, Secondary Bus Number, and Primary Bus Number.
Through the logs, [bus 0d 0e 0c] is created for pci device 0000:0b:01.0, but when removing PCI device 0000:0b:01.0, only bus 0c is removed, bus 0d 0e is still there.
Following is the related logs of "lspci"
# lspci -tv
......
\-[0000:00]-+-00.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DMI2
+-03.1-[08-73]----00.0-[09-73]--+-00.0 Microsemi / PMC / IDT PES24NT24G2 PCI Express Switch
| +-02.0-[0a-10]----00.0-[0b-10]--+-01.0-[0d]----00.0 Device xxxx
| | +-04.0 PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch
| | +-05.0-[00]--
| | +-07.0-[00]--
| | \-09.0-[00]--
| +-03.0-[11-17]----00.0-[12-17]--+-01.0-[14]----00.0 Device xxxx
| | +-04.0 PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch
| | +-05.0-[00]--
| | +-07.0-[00]--
| | \-09.0-[00]--
......Thanks,
Rui
pci 0000:11:00.0: [10b5:8606] type 01 class 0x060400
pci 0000:11:00.0: Quirks: Set bus range to 0x171211
pci 0000:11:00.0: reg 0x10: [mem 0x00000000-0x0001ffff]
pci 0000:11:00.0: Max Payload Size set to 256 (was 128, max 512)
pci 0000:11:00.0: PME# supported from D0 D3hot D3cold
pci 0000:11:00.0: Adding to iommu group 28
pci 0000:12:01.0: [10b5:8606] type 01 class 0x060400
pci 0000:12:01.0: Quirks: Set bus range to 0x141412
pci 0000:12:01.0: Max Payload Size set to 256 (was 128, max 512)
pci 0000:12:01.0: PME# supported from D0 D3hot D3cold
pci 0000:12:01.0: Adding to iommu group 28
pci 0000:12:04.0: [10b5:8606] type 00 class 0x068000
pci 0000:12:04.0: Max Payload Size set to 256 (was 128, max 512)
pci 0000:12:04.0: Adding to iommu group 28
pci 0000:12:05.0: [10b5:8606] type 01 class 0x060400
pci 0000:12:05.0: Max Payload Size set to 256 (was 128, max 512)
pci 0000:12:05.0: PME# supported from D0 D3hot D3cold
pci 0000:12:05.0: Adding to iommu group 28
pci 0000:12:07.0: [10b5:8606] type 01 class 0x060400
pci 0000:12:07.0: Max Payload Size set to 256 (was 128, max 512)
pci 0000:12:07.0: PME# supported from D0 D3hot D3cold
pci 0000:12:07.0: Adding to iommu group 28
pci 0000:12:09.0: [10b5:8606] type 01 class 0x060400
pci 0000:12:09.0: Max Payload Size set to 256 (was 128, max 512)
pci 0000:12:09.0: PME# supported from D0 D3hot D3cold
pci 0000:12:09.0: Adding to iommu group 28
pci 0000:11:00.0: PCI bridge to [bus 12-17]
pci 0000:11:00.0: bridge window [mem 0xa1000000-0xa1ffffff]
......
(removed the enumeration of 0000:14:00.0 as it's customer sensitive)
......
pci 0000:12:01.0: PCI bridge to [bus 14]
pci 0000:12:01.0: bridge window [mem 0xa1000000-0xa17fffff]
pci 0000:12:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:12:07.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:12:09.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:12:05.0: PCI bridge to [bus 15-17]pci_bus 0000:15: busn_res: [bus 15-17] end is updated to 15
pci 0000:12:07.0: PCI bridge to [bus 16-17]pci_bus 0000:16: busn_res: [bus 16-17] end is updated to 16
pci 0000:12:09.0: PCI bridge to [bus 17]pci_bus 0000:17: busn_res: [bus 17] end is updated to 17
pcieport 0000:09:02.0: bridge window [io 0x1000-0x0fff] to [bus 0a-10] add_size 1000
pcieport 0000:09:03.0: bridge window [io 0x1000-0x0fff] to [bus 11-17] add_size 1000
pci 0000:11:00.0: BAR 14: assigned [mem 0xa1000000-0xa1ffffff]
pci 0000:12:01.0: BAR 14: assigned [mem 0xa1000000-0xa17fffff]
pci 0000:14:00.0: BAR 0: assigned [mem 0xa1000000-0xa17fffff 64bit]
pci 0000:12:01.0: PCI bridge to [bus 14]
pci 0000:12:01.0: bridge window [mem 0xa1000000-0xa17fffff]
pci 0000:12:05.0: PCI bridge to [bus 15]
pci 0000:12:07.0: PCI bridge to [bus 16]
pci 0000:12:09.0: PCI bridge to [bus 17]
pci 0000:11:00.0: PCI bridge to [bus 12-17]
pci 0000:11:00.0: bridge window [mem 0xa1000000-0xa1ffffff]
pcieport 0000:12:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pcieport 0000:12:07.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pcieport 0000:12:09.0: bridge configuration invalid ([bus 00-00]), reconfiguringpci_bus 0000:15: busn_res: [bus 15] end is updated to 15pci_bus 0000:16: busn_res: [bus 16] end is updated to 16pci_bus 0000:17: busn_res: [bus 17] end is updated to 17
pcieport 0000:09:02.0: bridge window [io 0x1000-0x0fff] to [bus 0a-10] add_size 1000
pcieport 0000:09:03.0: bridge window [io 0x1000-0x0fff] to [bus 11-17] add_size 1000
pci 0000:0a:00.0: [10b5:8606] type 01 class 0x060400
pci 0000:0a:00.0: Quirks: Set bus range to 0x100b0a
pci 0000:0a:00.0: reg 0x10: [mem 0x00000000-0x0001ffff]
pci 0000:0a:00.0: Max Payload Size set to 256 (was 128, max 512)
pci 0000:0a:00.0: PME# supported from D0 D3hot D3cold
pci 0000:0a:00.0: Adding to iommu group 27
pci 0000:0b:01.0: [10b5:8606] type 01 class 0x060400
pci 0000:0b:01.0: Quirks: Set bus range to 0x0d0d0b
pci 0000:0b:01.0: Max Payload Size set to 256 (was 128, max 512)
pci 0000:0b:01.0: PME# supported from D0 D3hot D3cold
pci 0000:0b:01.0: Adding to iommu group 27
pci 0000:0b:04.0: [10b5:8606] type 00 class 0x068000
pci 0000:0b:04.0: Max Payload Size set to 256 (was 128, max 512)
pci 0000:0b:04.0: Adding to iommu group 27
pci 0000:0b:05.0: [10b5:8606] type 01 class 0x060400
pci 0000:0b:05.0: Max Payload Size set to 256 (was 128, max 512)
pci 0000:0b:05.0: PME# supported from D0 D3hot D3cold
pci 0000:0b:05.0: Adding to iommu group 27
pci 0000:0b:07.0: [10b5:8606] type 01 class 0x060400
pci 0000:0b:07.0: Max Payload Size set to 256 (was 128, max 512)
pci 0000:0b:07.0: PME# supported from D0 D3hot D3cold
pci 0000:0b:07.0: Adding to iommu group 27
pci 0000:0b:09.0: [10b5:8606] type 01 class 0x060400
pci 0000:0b:09.0: Max Payload Size set to 256 (was 128, max 512)
pci 0000:0b:09.0: PME# supported from D0 D3hot D3cold
pci 0000:0b:09.0: Adding to iommu group 27
pci 0000:0a:00.0: PCI bridge to [bus 0b-10]
pci 0000:0a:00.0: bridge window [mem 0xa0000000-0xa0ffffff]
......
(removed the enumeration of 0000:0d:00.0 as it's customer sensitive)
......
pci 0000:0b:01.0: PCI bridge to [bus 0d]
pci 0000:0b:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:0b:07.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:0b:09.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:0b:01.0: PCI bridge to [bus 0e-10]pci_bus 0000:0e: busn_res: [bus 0e-10] end is updated to 0e
pci 0000:0b:05.0: PCI bridge to [bus 0f-10]pci_bus 0000:0f: busn_res: [bus 0f-10] end is updated to 0f
pci 0000:0b:07.0: PCI bridge to [bus 10]pci_bus 0000:10: busn_res: [bus 10] end is updated to 10
pcieport 0000:12:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pcieport 0000:12:07.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pcieport 0000:12:09.0: bridge configuration invalid ([bus 00-00]), reconfiguringpci_bus 0000:15: busn_res: [bus 15] end is updated to 15pci_bus 0000:16: busn_res: [bus 16] end is updated to 16pci_bus 0000:17: busn_res: [bus 17] end is updated to 17pci_bus 0000:11: busn_res: [bus 11-17] end is updated to 17
pci 0000:0b:09.0: devices behind bridge are unusable because [bus 11-17] cannot be assigned for them pci 0000:0a:00.0: bridge has subordinate 10 but max busn 17
pcieport 0000:12:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pcieport 0000:12:07.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pcieport 0000:12:09.0: bridge configuration invalid ([bus 00-00]), reconfiguringpci_bus 0000:15: busn_res: [bus 15] end is updated to 15pci_bus 0000:16: busn_res: [bus 16] end is updated to 16pci_bus 0000:17: busn_res: [bus 17] end is updated to 17
pcieport 0000:09:02.0: bridge window [io 0x1000-0x0fff] to [bus 0a-10] add_size 1000
pcieport 0000:09:03.0: bridge window [io 0x1000-0x0fff] to [bus 11-17] add_size 1000
pci 0000:0a:00.0: BAR 14: assigned [mem 0xa0000000-0xa0ffffff]
pci 0000:0a:00.0: BAR 0: no space for [mem size 0x00020000]
pci 0000:0a:00.0: BAR 0: failed to assign [mem size 0x00020000]
pci 0000:0b:01.0: PCI bridge to [bus 0e]
pci 0000:0b:05.0: PCI bridge to [bus 0f]
pci 0000:0b:07.0: PCI bridge to [bus 10]
pci 0000:0a:00.0: PCI bridge to [bus 0b-10]
pci 0000:0a:00.0: bridge window [mem 0xa0000000-0xa0ffffff]
pcieport 0000:0b:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pcieport 0000:0b:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pcieport 0000:0b:07.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pcieport 0000:0b:09.0: bridge configuration invalid ([bus 00-00]), reconfiguringpci_bus 0000:0c: busn_res: can not insert [bus 0c-10] under [bus 0b-10] (conflicts with (null) [bus 0d])
......
(removed the enumeration of 0000:0d:00.0 as it's customer sensitive)
......
pcieport 0000:0b:01.0: PCI bridge to [bus 0c-10]pci_bus 0000:0c: busn_res: [bus 0c-10] end is updated to 0cpci_bus 0000:0d: busn_res: [bus 0d] end is updated to 0dpci_bus 0000:0e: busn_res: [bus 0e] end is updated to 0epci_bus 0000:0f: busn_res: [bus 0f] end is updated to 0f
pcieport 0000:12:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pcieport 0000:12:07.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pcieport 0000:12:09.0: bridge configuration invalid ([bus 00-00]), reconfiguringpci_bus 0000:15: busn_res: [bus 15] end is updated to 15pci_bus 0000:16: busn_res: [bus 16] end is updated to 16pci_bus 0000:17: busn_res: [bus 17] end is updated to 17
pcieport 0000:09:02.0: bridge window [io 0x1000-0x0fff] to [bus 0a-10] add_size 1000
pcieport 0000:09:03.0: bridge window [io 0x1000-0x0fff] to [bus 11-17] add_size 1000
pcieport 0000:0b:01.0: BAR 14: assigned [mem 0xa0000000-0xa07fffff]
pci 0000:0c:00.0: BAR 0: assigned [mem 0xa0000000-0xa07fffff 64bit]
pcieport 0000:00:03.1: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:0a:00.0 pcieport 0000:0a:00.0: AER: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Requester ID)
pcieport 0000:0a:00.0: AER: device [10b5:8606] error status/mask=00100000/00000000
pcieport 0000:0a:00.0: AER: [20] UnsupReq (First)
pcieport 0000:0a:00.0: AER: TLP Header: 40000001 0000060f a0000204 00000000
pcieport 0000:0a:00.0: AER: Error of this Agent is reported first
pcieport 0000:09:02.0: AER: Device recovery failed
pcieport 0000:00:03.1: AER: Uncorrected (Non-Fatal) error received: 0000:0a:00.0
pcieport 0000:0a:00.0: AER: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Requester ID)
pcieport 0000:0a:00.0: AER: device [10b5:8606] error status/mask=00100000/00000000
pcieport 0000:0a:00.0: AER: [20] UnsupReq (First)
pcieport 0000:0a:00.0: AER: TLP Header: 40000001 0000060f a0000204 00000000
pcieport 0000:09:02.0: AER: Device recovery failed
pci 0000:0c:00.0: Removing from iommu group 27pci_bus 0000:0c: busn_res: [bus 0c] is released
pci 0000:0b:01.0: Removing from iommu group 27
pci 0000:0b:04.0: Removing from iommu group 27pci_bus 0000:0f: busn_res: [bus 0f] is released
pci 0000:0b:05.0: Removing from iommu group 27pci_bus 0000:10: busn_res: [bus 10] is released
pci 0000:0b:07.0: Removing from iommu group 27
pci 0000:0b:09.0: Removing from iommu group 27pci_bus 0000:0b: busn_res: [bus 0b-10] is released
pci 0000:0a:00.0: Removing from iommu group 27
pcieport 0000:00:03.1: AER: Uncorrected (Non-Fatal) error received: 0000:00:03.1
pcieport 0000:00:03.1: AER: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Requester ID)
pcieport 0000:00:03.1: AER: device [8086:2f09] error status/mask=00004000/00000000
pcieport 0000:00:03.1: AER: [14] CmpltTO (First)
pcieport 0000:00:03.1: AER: Device recovery failed
Quoted 10 lines · Rui He
For preconfigured PCI bridge, child bus created on the first scan. While for some reasons(e.g register mutation), the secondary, and subordiante register reset to 0 on the second scan, which caused to create PCI bus twice for the same PCI device.
Following is the related log:
[Wed May 28 20:38:36 CST 2025] pci 0000:0b:01.0: PCI bridge to [bus 0d] [Wed May 28 20:38:36 CST 2025] pci 0000:0b:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring [Wed May 28 20:38:36 CST 2025] pci 0000:0b:01.0: PCI bridge to [bus 0e-10] [Wed May 28 20:38:36 CST 2025] pci 0000:0b:05.0: PCI bridge to [bus 0f-10]
Could you help to attach a 'lspci -t' about the topology ? bridge 0000:0b:01.0 and 0000:0b:05.0 have the same subordinate bus number, that is weird seems they aren't connected as upstream and downstream, but siblings.
Does the device behind the bridge 0000:0b:05.0 work after the second scan (TLP are forwarded) ?>
Quoted 15 lines
Here PCI device 000:0b:01.0 assigend to bus 0d and 0e.
This patch checks if child PCI bus has been created on the second scan of bridge. If yes, return directly instead of create a new one.
Signed-off-by Rui He <[email protected]>--- drivers/pci/probe.c | 3 +++ 1 file changed, 3 insertions(+)drivers/pci/probe.c +0
-0
drivers/pci/probe.c static int pci_scan_bridge_extend(struct pci_bus *bus, struct pci_dev *dev, · line 1446
Quoted 3 lines
drivers/pci/probe.c +0
-0@@ -1444,6 +1444,9 @@ static int pci_scan_bridge_extend(struct pci_bus *bus, struct pci_dev *dev, goto out; }
The bridge should was marked as broken=1 already, bailed out earlier, wouldn't get here with bridge forwarding was disabled. no further configuration anymore. what is your kernel number ?
Thanks,
Ethan> + if(pci_has_subordinate(dev))
Quoted 5 lines
drivers/pci/probe.c +2
-0+ goto out; +/* Clear errors */ pci_write_config_word (dev,PCI_STATUS ,0xffff );
Quoted 28 lines
-----Original Message-----
From: Ethan Zhao <[email protected]>
Sent: 2025年8月17日 10:46
To: He, Rui <[email protected]>; Bjorn Helgaas <[email protected]>
Cc: [email protected]; [email protected]; Chikhalkar, Prashant <[email protected]>; Xiao, Jiguang <[email protected]>
Subject: Re: [PATCH 1/1] pci: Add subordinate check beforepci_add_new_bus()CAUTION: This email comes from a non Wind River email account! Do not click links or open attachments unless you recognize the sender and know the content is safe.
Quoted 11 lines · Rui He
For preconfigured PCI bridge, child bus created on the first scan. While for some reasons(e.g register mutation), the secondary, and subordiante register reset to 0 on the second scan, which caused to create PCI bus twice for the same PCI device.
Following is the related log:
[Wed May 28 20:38:36 CST 2025] pci 0000:0b:01.0: PCI bridge to [bus 0d] [Wed May 28 20:38:36 CST 2025] pci 0000:0b:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring [Wed May 28 20:38:36 CST 2025] pci 0000:0b:01.0: PCI bridge to [bus 0e-10] [Wed May 28 20:38:36 CST 2025] pci 0000:0b:05.0: PCI bridge to [bus 0f-10]
Quoted 4 lines
Could you help to attach a 'lspci -t' about the topology ? bridge 0000:0b:01.0 and 0000:0b:05.0 have the same subordinate bus number, that is weird seems they aren't connected as upstream and downstream, but siblings.
Follwing is the related lspci logs.
# lspci -tv
......
\-[0000:00]-+-00.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DMI2
+-03.1-[08-73]----00.0-[09-73]--+-00.0 Microsemi / PMC / IDT PES24NT24G2 PCI Express Switch
| +-02.0-[0a-10]----00.0-[0b-10]--+-01.0-[0d]----00.0 Device xxxx
| | +-04.0 PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch
| | +-05.0-[00]--
| | +-07.0-[00]--
| | \-09.0-[00]--
| +-03.0-[11-17]----00.0-[12-17]--+-01.0-[14]----00.0 Device xxxx
| | +-04.0 PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch
| | +-05.0-[00]--
| | +-07.0-[00]--
| | \-09.0-[00]--
......Yes, you are right. 0000:0b:01.0 and 0000:0b:05.0 are siblings.
I added 000:0b:05.0 to indicate that [bus 0d] is created during the first scan, while [bus 0e-10] is created during the second scan.
Here, 0000:0b:01.0 is pre-assigned to bus 0d. 0000:05[07, 09].0 is not configured.
Quoted 22 lines
Does the device behind the bridge 0000:0b:05.0 work after the second scan (TLP are forwarded) ?>
drivers/pci/probe.c static int pci_scan_bridge_extend(struct pci_bus
Quoted 15 lines
Here PCI device 000:0b:01.0 assigend to bus 0d and 0e.
This patch checks if child PCI bus has been created on the second scan of bridge. If yes, return directly instead of create a new one.
Signed-off-by Rui He <[email protected]>--- drivers/pci/probe.c | 3 +++ 1 file changed, 3 insertions(+)drivers/pci/probe.c index renamed from drivers/pci/probe.c +0
-0
f41128f91ca76..ec67adbf31738100644drivers/pci/probe.c +0
-0@@ -1444,6 +1444,9 @@ static int pci_scan_bridge_extend(struct pci_bus*bus, struct
pci_dev*dev,drivers/pci/probe.c static int pci_scan_bridge_extend(struct pci_bus *bus, struct pci_dev *dev, · line 1446
Quoted 3 lines
drivers/pci/probe.c +0
-0goto out; }
Quoted 7 lines
The bridge should was marked as broken=1 already, bailed out earlier, wouldn't get here with bridge forwarding was disabled. no further configuration anymore. what is your kernel number ?
Thanks,
Ethan
My kernel version is v5.2.60 (https://git.yoctoproject.org/linux-yocto/tree/Makefile?h=v5.2/standard/base).
The bridge can be marked to broken=1 on the first scan, while this error happens on the second scan, here pass=1, (!pass) always be false, broken is impossible to set to 1.
[bus 0e-10] was created on 0000:0b:01.0 on the second scan, which means that the if condition is false.
-> if ((secondary || subordinate) && !pcibios_assign_all_busses() &&
-> !is_cardbus && !broken) {pcibios_assign_all_busses() always be false as "pci=assign-busses" not added to cmdline.
Is_cardbus always be false as 0000:0b.01.0 is a bridge.
Broken always be false on the second scan as pass=1.
The only possible is that (secondary || subordinate) is false.
Thanks,
Rui
Quoted 6 lines
+ if(pci_has_subordinate(dev))
Quoted 5 lines
drivers/pci/probe.c +2
-0+ goto out; +/* Clear errors */ pci_write_config_word (dev,PCI_STATUS ,0xffff );