PCI: dw-rockchip: Add support to reset Root Port upon link down event
Subsystems (3)
cover PCI: dw-rockchip: Add support to reset Root Port upon link down event
Hello all,
This depends on Mani's v8 series with adds a reset_root_port() callback:
https://lore.kernel.org/linux-pci/[email protected]/
Tested by using two Rock 5B boards and simulating a link down by doing:
echo 0 > /sys/kernel/config/pci_ep/controllers/a40000000.pcie-ep/start && \
sleep 0.1 && echo 1 > /sys/kernel/config/pci_ep/controllers/a40000000.pcie-ep/start on the board running in EP mode.
We can see the root port being recovered on the host side after the link down:
pcieport 0000:00:00.0: Recovering Root Port due to Link Down rockchip-dw-pcie a40000000.pcie: PCIe Gen.3 x4 link up
pcieport 0000:00:00.0: Root Port has been reset
pcieport 0000:00:00.0: AER: device recovery successful
Both before and after the link down, we get 19 / 23 tests passed when running the pci_endpoint selftest.
Without this patch, after the link down, all BAR tests and MSI/MSI-X tests fail.
(Side node: When testing this patch on top of Mani's v7 series, MSI/MSI-X tests still failed. Nice to see that this is no longer the case when testing on top of v8. I did not do a deep dive to figure out exactly which change between v7 and v8 that made all tests pass. It is obviously not:
[PATCH v8 1/5] PCI: dwc: ep: Clear MSI iATU mapping in dw_pcie_ep_cleanup() as pcie-dw-rockchip.c never calls this function.
Thus, it must be some of the other changes between v7 and v8, or some change in PCI core which now saves the config space during enumeration, which made MSI/MSI-X work after a link down.)
Kind regards,
Niklas
Wilfred Mallawa (1):
PCI: dw-rockchip: Add support to reset Root Port upon link down event
drivers/pci/controller/dwc/Kconfig | 1 + drivers/pci/controller/dwc/pcie-dw-rockchip.c | 134 +++++++++++++++++- 2 files changed, 132 insertions(+), 3 deletions(-)
Signature · 1 line
2.55.0
PATCH 1/1 PCI: dw-rockchip: Add support to reset Root Port upon link down event
From: Wilfred Mallawa <[email protected]>
The PCIe link may go down in cases like firmware crashes or unstable connections. When this occurs, the PCIe Root Port must be reset to restore the functionality. However, the current driver lacks link down handling, forcing users to reboot the system to recover.
This patch implements the reset_root_port callback for link down handling for Rockchip DWC PCIe host controller. In which, the RC is reset, reconfigured and link training initiated to recover from the link down event.
This also by extension fixes issues with sysfs initiated bus resets. In that, currently, when a sysfs initiated bus reset is issued, the endpoint device is non-functional after (may link up with downgraded link status). With the link down handling support, a sysfs initiated bus reset works as intended. Testing conducted on a ROCK5B board with an M.2 NVMe drive.
--- drivers/pci/controller/dwc/Kconfig | 1 + drivers/pci/controller/dwc/pcie-dw-rockchip.c | 134 +++++++++++++++++- 2 files changed, 132 insertions(+), 3 deletions(-)
drivers/pci/controller/dwc/Kconfig +1 -0
@@ -373,6 +373,7 @@ config PCIE_ROCKCHIP_DW_HOST depends on OF select PCIE_DW_HOST select PCIE_ROCKCHIP_DW + select PCI_HOST_COMMON help Enables support for the DesignWare PCIe controller in the Rockchip SoC (except RK3399) to work in host mode.
drivers/pci/controller/dwc/pcie-dw-rockchip.c +131 -3
189 lines not on this page · raw message
Signature · 1 line
2.55.0
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider: - [High] The patch calls `pci_host_handle_link_down()` which is neither defined nor declared anywhere in…
Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider:
- [High] The patch calls pci_host_handle_link_down() which is neither defined nor declared anywhere in the kernel tree.
- [High] The code attempts to assign a callback to a non-existent member reset_root_port in struct pci_host_bridge.
- [High] Repeatedly calling probe-time initialization functions in the link-down reset handler causes severe devres memory and resource handle leaks (clocks, PHYs, and IRQ domains).
- [High] Early IRQ registration before structures are initialized causes a race condition leading to NULL pointer dereferences in the 'sys' IRQ handler.
- [High] Iterating over the PCI bus devices list without holding the pci_bus_sem reader lock leads to potential list corruption or use-after-free.
- [High] Disabling APB clocks during reset races with interrupt handlers, leading to synchronous external aborts.
--
--- Patch [1]: [PATCH 1/1] PCI: dw-rockchip: Add support to reset Root Port upon link down event --- commit 026e07d49a335504d0a8afc363c1f59787c29767
Author: Wilfred Mallawa <[email protected]>
PCI: dw-rockchip: Add support to reset Root Port upon link down event
This patch implements a reset_root_port callback for link down handling on the Rockchip DWC PCIe host controller. It also adds a sys IRQ thread to detect link-down events and issue the reset.
Note: The format of this review has been altered to provide a free-form summary of findings, without directly quoting the original patch code, due to recitation restrictions.
[Severity: High]
In the new sys IRQ handler rockchip_pcie_rc_sys_irq_thread(), the code calls pci_host_handle_link_down(). Is this function defined or declared anywhere in the kernel tree? It appears missing, which might cause a compilation failure or linker error when this driver is built.
[Severity: High]
In rockchip_pcie_host_init(), the patch assigns rockchip_pcie_rc_reset_root_port to pp->bridge->reset_root_port. Does struct pci_host_bridge actually contain a reset_root_port member? It looks like this member does not exist in include/linux/pci.h, which could result in a hard compiler error.
[Severity: High]
When rockchip_pcie_rc_reset_root_port() is triggered, it calls probe-time initialization routines such as rockchip_pcie_phy_init(), rockchip_pcie_clk_init(), and pp->ops->init() (which eventually calls rockchip_pcie_init_irq_domain()). Because these functions use devres (like devm_phy_get and devm_clk_bulk_get_all) or create new linear IRQ domains unconditionally, does this repeatedly leak PHY objects, clock arrays, and devres memory on every link-down reset? Recreating the IRQ domain might also break existing mappings.
[Severity: High]
In rockchip_pcie_configure_rc(), devm_request_threaded_irq() for the "sys" IRQ is called before dw_pcie_host_init() completes. If the interrupt fires immediately, the handler rockchip_pcie_rc_sys_irq_thread() will run and iterate over pp->bridge->bus. Since pp->bridge and its bus are not fully allocated and assigned until dw_pcie_host_init() finishes, could this lead to a NULL pointer dereference?
[Severity: High]
The handler rockchip_pcie_rc_sys_irq_thread() iterates over the PCI bus devices list using for_each_pci_bridge(port, pp->bridge->bus). Is it safe to iterate this list without holding the pci_bus_sem read lock? Concurrent PCI hotplug, sysfs rescan, or unbind operations might modify the list, potentially leading to list corruption or a use-after-free.
[Severity: High]
In rockchip_pcie_rc_reset_root_port(), clocks are disabled via clk_bulk_disable_unprepare() without synchronizing against active interrupt handlers. If an interrupt such as the intx handler fires concurrently, it will attempt to read APB registers like PCIE_CLIENT_INTR_STATUS_LEGACY while the APB clock is off. Will this cause a synchronous external abort and system hang?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Manivannan Sadhasivam
AppliedQuoted 11 lines · Niklas Cassel
The PCIe link may go down in cases like firmware crashes or unstable connections. When this occurs, the PCIe Root Port must be reset to restore the functionality. However, the current driver lacks link down handling, forcing users to reboot the system to recover.
This patch implements the
reset_root_portcallback for link down handling for Rockchip DWC PCIe host controller. In which, the RC is reset, reconfigured and link training initiated to recover from the link down event.[...]
Applied, thanks!
[1/1] PCI: dw-rockchip: Add support to reset Root Port upon link down event
commit: 9225afa7a8637df41b8e4705ea7e7e890ce78084
Best regards,
Signature · 1 line
மணிவண்ணன் சதாசிவம்