Skip to contentSkip to conversation

PCI: Add D3cold as general reset method

PATCH Superseded Jose Ignacio Tornos Martinez 4 participants 10 messages May 13 – May 18 PCI

v3 of 15 Superseded Latest is v15 →

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/2 PCI: Add D3cold as general reset method

Add D3cold power cycle as a general PCI reset method, available for single-function devices. This provides a more robust reset mechanism than D3hot for devices where a full power cycle is beneficial.

The implementation uses pci_set_power_state(dev, PCI_D3cold), which automatically handles platform differences:
- Platforms WITH _PR3 ACPI power resources: true D3cold (power cycle)
- Platforms WITHOUT _PR3: automatic fallback to D3hot transition

D3cold reset is placed at the end of the reset hierarchy as a last resort before giving up, since it provides a strong reset when other methods are unavailable or broken.

Reset hierarchy with this change:
1. device_specific
2. acpi
3. flr
4. af_flr
5. pm (D3hot via config space)
6. bus (SBR)
7. cxl_bus
8. d3cold (NEW - power cycle with D3hot fallback)

This benefits devices that:
- Have broken or missing FLR
- Advertise NoSoftRst+ incorrectly (blocking D3hot PM reset)
- Have broken bus reset implementations
- Need power cycle for reliable reset
- Are used in VFIO passthrough scenarios

---
v3: Implement d3cold as a general PCI core reset method instead of
device-specific quirk approach from v2 (Alex Williamson suggestion)
v2: https://lore.kernel.org/all/[email protected]/

 drivers/pci/pci.c   | 41 +++++++++++++++++++++++++++++++++++++++++
 include/linux/pci.h |  2 +-
 2 files changed, 42 insertions(+), 1 deletion(-)
drivers/pci/pci.c +41 -0
index 8f7cfcc00090..6da8feff7ccc 100644
@@ -4491,6 +4491,46 @@ static int pci_pm_reset(struct pci_dev *dev, bool probe)
 	return ret;
 }
 
+/**
+ * pci_d3cold_reset - Put device into D3cold (or D3hot) and back to D0 for reset
+ * @dev: PCI device to reset
+ * @probe: if true, check if D3cold reset is supported; if false, perform reset
+ *
+ * Attempt to reset the device by transitioning through D3cold and back to D0.
+ * On platforms with ACPI _PR3 power resources, this performs a true D3cold
+ * power cycle (actual power removal). On platforms without _PR3 support,
+ * pci_set_power_state() automatically falls back to D3hot, providing a
+ * D3hot->D0 reset transition.
+ *
+ * Only available for single-function devices to avoid affecting other
+ * functions in multi-function devices.
+ *
+ * Returns 0 if device can be/was reset this way, -ENOTTY if not supported,
+ * or other negative error code on failure.
+ */
+static int pci_d3cold_reset(struct pci_dev *dev, bool probe)
+{
+	int ret;
+
+	if (dev->multifunction)
+		return -ENOTTY;
+
+	if (probe) {
+		if (!pci_pr3_present(dev))
+			pci_dbg(dev, "d3cold reset: no _PR3 support, will use D3hot fallback\n");
+		return 0;
+	}
+
+	if (dev->current_state != PCI_D0)
+		return -EINVAL;
+
+	ret = pci_set_power_state(dev, PCI_D3cold);
+	if (ret)
+		return ret;
+
+	return pci_set_power_state(dev, PCI_D0);
+}
+
 /**
  * pcie_wait_for_link_status - Wait for link status change
  * @pdev: Device whose link to wait for.
@@ -5065,6 +5105,7 @@ const struct pci_reset_fn_method pci_reset_fn_methods[] = {
 	{ pci_pm_reset, .name = "pm" },
 	{ pci_reset_bus_function, .name = "bus" },
 	{ cxl_reset_bus_function, .name = "cxl_bus" },
+	{ pci_d3cold_reset, .name = "d3cold" },
 };
 
 /**
include/linux/pci.h +1 -1
index 2c4454583c11..1ca7b880ead7 100644
@@ -51,7 +51,7 @@
 			       PCI_STATUS_PARITY)
 
 /* Number of reset methods used in pci_reset_fn_methods array in pci.c */
-#define PCI_NUM_RESET_METHODS 8
+#define PCI_NUM_RESET_METHODS 9
 
 #define PCI_RESET_PROBE		true
 #define PCI_RESET_DO_RESET	false
Signature · 1 line
2.53.0
#1 lore raw

PATCH v3 2/2 PCI: Disable broken bus reset on Qualcomm devices

Some Qualcomm PCIe devices do not properly support Secondary Bus Reset (SBR). These devices have no FLR capability and advertise NoSoftRst+ (blocking PM reset), leaving bus reset as the only available method. However, bus reset does not work reliably for these devices.

The problem manifests in VFIO passthrough scenarios with these affected devices:

- ath11k WiFi (17cb:1103): Normal VM operation works fine, including clean shutdown/reboot. However, when the VM terminates uncleanly (crash, force-off), VFIO attempts to reset the device before it can be assigned to another VM. Without a working reset method, the device remains in an undefined state, preventing reuse.

- ath12k WiFi (17cb:1107): Same behavior as ath11k.

- SDX62/SDX65 5G modems (17cb:0308): Never successfully initialize even on first VM assignment without proper reset capability.

Disable bus reset for these devices (following the pattern of other Atheros/Qualcomm devices) so the d3cold reset method (added in previous patch) is used instead, which provides working reset via D3cold power cycle or D3hot fallback.

---
v3: Try to address Alex Williamson comments:
- Change from PCI_DEV_FLAGS_FORCE_PM_RESET quirk (v2) to quirk_no_bus_reset
- Rely on d3cold general reset method (patch 1) instead of forcing PM reset
v2: https://lore.kernel.org/all/[email protected]/

 drivers/pci/quirks.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/pci/quirks.c +3 -0
index 000000000000..111111111111 100644
@@ -3789,6 +3789,9 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_ATHEROS, 0x0030, quirk_no_bus_reset);
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_ATHEROS, 0x0032, quirk_no_bus_reset);
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_ATHEROS, 0x003c, quirk_no_bus_reset);
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_ATHEROS, 0x0033, quirk_no_bus_reset);
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_ATHEROS, 0x0034, quirk_no_bus_reset);
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_ATHEROS, 0x003e, quirk_no_bus_reset);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_QCOM, 0x1103, quirk_no_bus_reset); /* ath11k */
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_QCOM, 0x1107, quirk_no_bus_reset); /* ath12k */
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_QCOM, 0x0308, quirk_no_bus_reset); /* SDX62/SDX65 */
 

/*
* Root port on some Cavium CN8xxx chips do not successfully complete a bus
--
2.53.0

#2 lore raw
In reply to Jose Ignacio Tornos Martinez
Quoted 81 lines · Jose Ignacio Tornos Martinez

Add D3cold power cycle as a general PCI reset method, available for single-function devices. This provides a more robust reset mechanism than D3hot for devices where a full power cycle is beneficial.

The implementation uses pci_set_power_state(dev, PCI_D3cold), which automatically handles platform differences:
- Platforms WITH _PR3 ACPI power resources: true D3cold (power cycle)
- Platforms WITHOUT _PR3: automatic fallback to D3hot transition

D3cold reset is placed at the end of the reset hierarchy as a last resort before giving up, since it provides a strong reset when other methods are unavailable or broken.

Reset hierarchy with this change:
1. device_specific
2. acpi
3. flr
4. af_flr
5. pm (D3hot via config space)
6. bus (SBR)
7. cxl_bus
8. d3cold (NEW - power cycle with D3hot fallback)

This benefits devices that:
- Have broken or missing FLR
- Advertise NoSoftRst+ incorrectly (blocking D3hot PM reset)
- Have broken bus reset implementations
- Need power cycle for reliable reset
- Are used in VFIO passthrough scenarios

---
v3: Implement d3cold as a general PCI core reset method instead of
device-specific quirk approach from v2 (Alex Williamson suggestion)
v2: https://lore.kernel.org/all/[email protected]/

 drivers/pci/pci.c   | 41 +++++++++++++++++++++++++++++++++++++++++
 include/linux/pci.h |  2 +-
 2 files changed, 42 insertions(+), 1 deletion(-)
drivers/pci/pci.c +2 -0
index 8f7cfcc00090..6da8feff7ccc 100644
@@ -4491,6 +4491,46 @@ static int pci_pm_reset(struct pci_dev *dev, bool probe)
 	return ret;
 }
 
+/**
+ * pci_d3cold_reset - Put device into D3cold (or D3hot) and back to D0 

for reset
+ * @dev: PCI device to reset
+ * @probe: if true, check if D3cold reset is supported; if false, perform reset
+ *
+ * Attempt to reset the device by transitioning through D3cold and back to D0.
+ * On platforms with ACPI _PR3 power resources, this performs a true D3cold
+ * power cycle (actual power removal). On platforms without _PR3 support,
+ * pci_set_power_state() automatically falls back to D3hot, providing a
+ * D3hot->D0 reset transition.
+ *
+ * Only available for single-function devices to avoid affecting other
+ * functions in multi-function devices.
+ *
+ * Returns 0 if device can be/was reset this way, -ENOTTY if not supported,
+ * or other negative error code on failure.
+ */

drivers/pci/pci.c +10 -0
+static int pci_d3cold_reset(struct pci_dev *dev, bool probe)
+{
+	int ret;
+
+	if (dev->multifunction)
+		return -ENOTTY;
+
+	if (probe) {
+		if (!pci_pr3_present(dev))
+			pci_dbg(dev, "d3cold reset: no _PR3 support, will use D3hot 
Quoted 1 line

fallback\n");

This fall through is invalid, if D3cold can't be reached the reset should be handled by pci_pm_reset() where NoSoftRst is honored. pci_pr3_preset() should be tested in all cases, not just probe. I think we also need to test device flags that would prevent a D3cold transition, like pci_pm_reset(). Thanks,

Alex

Quoted 40 lines
drivers/pci/pci.c +14 -0
+		return 0;
+	}
+
+	if (dev->current_state != PCI_D0)
+		return -EINVAL;
+
+	ret = pci_set_power_state(dev, PCI_D3cold);
+	if (ret)
+		return ret;
+
+	return pci_set_power_state(dev, PCI_D0);
+}
+
 /**
  * pcie_wait_for_link_status - Wait for link status change
  * @pdev: Device whose link to wait for.
@@ -5065,6 +5105,7 @@ const struct pci_reset_fn_method 
 pci_reset_fn_methods[] = {
 	{ pci_pm_reset, .name = "pm" },
 	{ pci_reset_bus_function, .name = "bus" },
 	{ cxl_reset_bus_function, .name = "cxl_bus" },
+	{ pci_d3cold_reset, .name = "d3cold" },
 };
 

/**

include/linux/pci.h +1 -1
index 2c4454583c11..1ca7b880ead7 100644
@@ -51,7 +51,7 @@
 			       PCI_STATUS_PARITY)
 
 /* Number of reset methods used in pci_reset_fn_methods array in pci.c */
-#define PCI_NUM_RESET_METHODS 8
+#define PCI_NUM_RESET_METHODS 9
 
 #define PCI_RESET_PROBE		true
 #define PCI_RESET_DO_RESET	false
Signature · 1 line
2.53.0
#3 lore raw

Hello Alex,

Thank you again for your feedbak.

Quoted 5 lines

This fall through is invalid, if D3cold can't be reached the reset should be handled by pci_pm_reset() where NoSoftRst is honored. pci_pr3_preset() should be tested in all cases, not just probe. I think we also need to test device flags that would prevent a D3cold transition,
like pci_pm_reset(). Thanks,

Ok, I've implemented it as a strict method (only when _PR3 is present) as you suggested for the next version. I think it can be very helpful.

However, I'm facing a challenge with a part of our deployment scenario and if possible I would appreciate your guidance (spoiler: d3hot possibility would be very helpful).

Let me comment on about this setup in our lab:
- Qualcomm WiFi cards (ath11k 17cb:1103, ath12k 17cb:1107) and modems (SDX62/SDX65 17cb:0308)
- M.2 format devices using passive PCIe adapters (not native M.2 slots) Problem:
- No FLR capability on devices
- NoSoftRst+ set (blocks PM reset)
- SBR broken through passive adapters (PERST# signal not properly wired)
- No _PR3 ACPI resources for standard PCIe slots (only for native M.2/ Thunderbolt)
- d3cold_allowed=1 but pci_pr3_present() returns false
Result:
Default kernel has no working reset method (reset_methods shows only "bus" which doesn't work).

I've tested the D3hot transition behavior and this is what I see before/ after:
- Command register: Cleared to 0x0000 (device disabled)
- BARs: Preserved (0x84200004 unchanged)
- Device successfully reinitializes and works in VFIO passthrough, so we can use it.

I don't dare to generalize D3hot in pci_pm_reset() with something like:
if (csr & PCI_PM_CTRL_NO_SOFT_RESET) {
pci_read_config_word(dev, dev->pm_cap + PCI_PM_PMC, &pmc);

/* If device supports D3hot, try reset despite NoSoftRst */ if (!pmc & PCI_PM_CAP_D3HOT_MASK)

    return -ENOTTY;
}

because some devices could truly advertise NoSoftRst+ and d3hot could not be usable as a reset.

I don't know if some device-specific quirk could be acceptable to bypass NoSoftRst check for these Qualcomm devices (similar to PCI_DEV_FLAGS_FORCE_PM_RESET from v2), although d3hot is usable in these cases.

Or maybe the generalization could be to create a user parameter to bypass PCI_PM_CTRL_NO_SOFT_RESET for the configured device. This would be manual but at least it would allow the device to work on VMs.

And finally, what I would choose: move pm to the end of the resets methods ignoring NoSoftRst, prioritizing hardware resets first (FLR, bus, d3cold) and using d3hot as the last resource if none of the others work. But this is an important change and there could be some reasons for pm positioning that I ignore, so if you don't mind I would like to consult you first.

What do you think? Could you help me?
I would like to address these devices on VMs in some way.

Thanks

Best regards
José Ignacio

#4 lore raw
Quoted 31 lines · On Thu, 14 May 2026 14:23:15 +0200 Jose Ignacio Tornos Martinez…

Hello Alex,

Thank you again for your feedbak.

Quoted 5 lines

This fall through is invalid, if D3cold can't be reached the reset should be handled by pci_pm_reset() where NoSoftRst is honored. pci_pr3_preset() should be tested in all cases, not just probe. I think we also need to test device flags that would prevent a D3cold transition,
like pci_pm_reset(). Thanks,

Ok, I've implemented it as a strict method (only when _PR3 is present) as you suggested for the next version. I think it can be very helpful.

However, I'm facing a challenge with a part of our deployment scenario and if possible I would appreciate your guidance (spoiler: d3hot possibility would be very helpful).

Let me comment on about this setup in our lab:
- Qualcomm WiFi cards (ath11k 17cb:1103, ath12k 17cb:1107) and modems (SDX62/SDX65 17cb:0308)
- M.2 format devices using passive PCIe adapters (not native M.2 slots) Problem:
- No FLR capability on devices
- NoSoftRst+ set (blocks PM reset)
- SBR broken through passive adapters (PERST# signal not properly wired)
- No _PR3 ACPI resources for standard PCIe slots (only for native M.2/ Thunderbolt)
- d3cold_allowed=1 but pci_pr3_present() returns false
Result:
Default kernel has no working reset method (reset_methods shows only "bus" which doesn't work).

Quoted 6 lines

I've tested the D3hot transition behavior and this is what I see before/ after:
- Command register: Cleared to 0x0000 (device disabled)
- BARs: Preserved (0x84200004 unchanged)
- Device successfully reinitializes and works in VFIO passthrough, so we can use it.

Are you testing with the patch that marks these devices PCI_DEV_FLAGS_NO_BUS_RESET?

I want to be sure that bouncing the device through D3hot is actually an improvement versus no reset at all, and we're not just performing a no-op reset to avoid the bus reset.

That said, if you're testing before BAR restore and those registers aren't being cleared, this seems at best a device specific reset, but evidence is a bit sketchy. In the general case, a transition through D3hot for a device that does not claim NoSoftRst- cannot be consider to satisfy a reset request.

Quoted 10 lines

I don't dare to generalize D3hot in pci_pm_reset() with something like:
if (csr & PCI_PM_CTRL_NO_SOFT_RESET) {
pci_read_config_word(dev, dev->pm_cap + PCI_PM_PMC, &pmc);

/* If device supports D3hot, try reset despite NoSoftRst */ if (!pmc & PCI_PM_CAP_D3HOT_MASK)

    return -ENOTTY;
}

because some devices could truly advertise NoSoftRst+ and d3hot could not be usable as a reset.

This should be the default, no assumed reset. The device is specifically able to indicate soft reset via D3hot.

Quoted 8 lines

I don't know if some device-specific quirk could be acceptable to bypass NoSoftRst check for these Qualcomm devices (similar to PCI_DEV_FLAGS_FORCE_PM_RESET from v2), although d3hot is usable in these cases.

Or maybe the generalization could be to create a user parameter to bypass PCI_PM_CTRL_NO_SOFT_RESET for the configured device. This would be manual but at least it would allow the device to work on VMs.

vfio-pci will try to put unused device into a low power state when they're unused. If the kernel exposes no function level reset mechanism, wouldn't the device experience D3hot after some idle timeout and then be usable again? Could you otherwise set the device power state via sysfs? It's certainly possible to create a userspace vfio stub driver that uses the low-power ioctl to trigger D3hot.

Unfortunately not all devices are sufficiently spec compliant to work cleanly with vfio, especially consumer grade devices without vendor involvement.

Quoted 5 lines

And finally, what I would choose: move pm to the end of the resets methods ignoring NoSoftRst, prioritizing hardware resets first (FLR, bus, d3cold) and using d3hot as the last resource if none of the others work. But this is an important change and there could be some reasons for pm positioning that I ignore, so if you don't mind I would like to consult you first.

We're getting into policy there, where the purpose of exposing reset_methods through sysfs is to allow userspace to define policy. There's obviously some policy in the default ordering, but we generally take the approach of flagging broken resets rather than enforcing an order. In general I agree with your suspicion of pm reset, there's no hard definition of a soft reset in the spec afaik. In fact, QEMU tries to infer whether the vfio reset ioctl likely implements pm reset by looking at the device features and will prefer a bus reset when the alternative is likely a pm reset.

If you can block the known broken resets and manage to trigger a D3hot transition externally, that may provide a reasonably useful environment. We've also been known to implement the sketchier resets that can't fully be justified as a kernel quirk in QEMU itself. Thanks,

Alex

#5 lore raw
In reply to Jose Ignacio Tornos Martinez
Quoted 8 lines · Jose Ignacio Tornos Martinez

Add D3cold power cycle as a general PCI reset method, available for single-function devices. This provides a more robust reset mechanism than D3hot for devices where a full power cycle is beneficial.

The implementation uses pci_set_power_state(dev, PCI_D3cold), which automatically handles platform differences:
- Platforms WITH _PR3 ACPI power resources: true D3cold (power cycle)
- Platforms WITHOUT _PR3: automatic fallback to D3hot transition

Why fall back to D3hot, even though the name of the reset method suggests it uses D3cold? If D3cold isn't supported, I'd just not expose the new reset method.

Quoted 9 lines

Reset hierarchy with this change:
1. device_specific
2. acpi
3. flr
4. af_flr
5. pm (D3hot via config space)
6. bus (SBR)
7. cxl_bus
8. d3cold (NEW - power cycle with D3hot fallback)

I'd call it "powercycle" for clarity but that's just my preferred bikeshed color.

drivers/pci/pci.c static int pci_pm_reset(struct pci_dev *dev, bool probe) · line 4522

Quoted 12 lines
drivers/pci/pci.c +12 -0
+static int pci_d3cold_reset(struct pci_dev *dev, bool probe)
+{
+	int ret;
+
+	if (dev->multifunction)
+		return -ENOTTY;
+
+	if (probe) {
+		if (!pci_pr3_present(dev))
+			pci_dbg(dev, "d3cold reset: no _PR3 support, will use D3hot fallback\n");
+		return 0;
+	}

So I'd just return -ENOTTY here in the !pci_pr3_present() case and I'd do that for probe == 0 as well.

Thanks,

Lukas

#6 lore raw

Hello Lukas,

Quoted 3 lines

Why fall back to D3hot, even though the name of the reset method suggests it uses D3cold? If D3cold isn't supported, I'd just not expose the new reset method.

Ok, the next version will include D3cold only if supported.

Quoted 2 lines

I'd call it "powercycle" for clarity but that's just my preferred bikeshed color.

I don't have any preference, although we can understand clearer what it is. If there are consensus, I am ok with powercycle.

Quoted 2 lines

So I'd just return -ENOTTY here in the !pci_pr3_present() case and I'd do that for probe == 0 as well.

Ok, I am going to wait for Alex Williamson opinion for the other cases but I will do this as you say in the next version.

Thanks

Best regards
José Ignacio

#7 lore raw
Quoted 5 lines · On Thu, 14 May 2026 14:30:58 +0200 Jose Ignacio Tornos Martinez…
Quoted 3 lines

I'd call it "powercycle" for clarity but that's just my preferred bikeshed color.

I don't have any preference, although we can understand clearer what it is. If there are consensus, I am ok with powercycle.

I think "powercycle" significantly expands the scope vs what is being proposed here. This is specifically only testing one method available for slot power, the one used for D3cold. If targeted more generically as "powercycle", that would logically imply support for other slot power control mechanisms, such as for hotplug slots. Thanks,

Alex

#8 lore raw

Hello Alex,

Thank you again for the detailed feedback.

Quoted 2 lines

Are you testing with the patch that marks these devices PCI_DEV_FLAGS_NO_BUS_RESET?

Yes, I'm testing with quirk_no_bus_reset applied.
My evidence about bus issues with the commented Qualcomm cards is device-specific and not adapter-specific because we are using the same
adapter with the commented MediaTek MT7925e
- Uses quirk_no_flr (https://lore.kernel.org/all/[email protected]/)
- reset_method shows: "bus"
- Bus reset works correctly for VFIO passthrough reuse
This proves PERST# is properly wired through the passive adapters. Since Qualcomm devices fail SBR with the same adapter setup where MediaTek
succeeds, I think the issue is device-specific, not deployment-specific.

Quoted 3 lines

I want to be sure that bouncing the device
through D3hot is actually an improvement versus no reset at all, and we're not just performing a no-op reset to avoid the bus reset.

The D3hot transition is definitely an improvement over no reset at all:
- Without any reset: Device remains in undefined state after VM crash and cannot be reused.
- With D3hot transition: Command register gets cleared (0x0002 → 0x0000), device can be reinitialized and successfully assigned to the VM
Testing methodology: VM crash → check PCI config space → verify device reusable. The D3hot transition clears enough state for VFIO passthrough reuse, even though it's not a full FLR-style reset.

Quoted 5 lines

That said, if you're testing before BAR restore and those registers aren't being cleared, this seems at best a device specific reset, but evidence is a bit sketchy. In the general case, a transition through D3hot for a device that does not claim NoSoftRst- cannot be consider to satisfy a reset request.

Agree, my testing confirms:
- Command register: CLEARED (0x0002 → 0x0000 → 0x0506 after driver init)
- BARs: PRESERVED (0x84200004 maintained throughout)
So, this is a partial reset, not a full spec-compliant reset and only the command register is cleared. However it works.

Quoted 4 lines

If you can block the known broken resets and manage to trigger a D3hot transition externally, that may provide a reasonably useful environment. We've also been known to implement the sketchier resets that can't fully be justified as a kernel quirk in QEMU itself.

I understand the concern about sketchy resets in the kernel.
I have prepared v4 version with a different conservative approach not changing anything in the current behavior, as a last resort and only adding a "soft" method when other reset methods are not available (our current case for Qualcomm cards).
I believe providing it as optional kernel infrastructure (user-controllable via sysfs) has value because:
- Benefits all VMMs (QEMU, cloud-hypervisor, etc.), not just QEMU
- Uses standard PCI PM mechanisms, no device-specific hacks
- Provides measurable value for VFIO passthrough where no reset exists
Happy to adjust based on your feedback

Thanks

Best regards
José Ignacio

#9 lore raw
In reply to Jose Ignacio Tornos Martinez
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider: - [Critical] Silent reset failure and state leak via NoSoftRst+ bypass - [High] Missing IOMMU coordination…

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
- [Critical] Silent reset failure and state leak via NoSoftRst+ bypass
- [High] Missing IOMMU coordination during reset sequence
- [High] Missing device readiness wait (CRS polling) after power transition
- [High] Catastrophic power loss for downstream hierarchy on single-function bridges
- [Medium] False advertisement of reset support in probe mode
--

commit 372c424ac498f1ceef866b966b4539db619980ea
Author: Jose Ignacio Tornos Martinez <[email protected]>

PCI: Add D3cold as general reset method

This commit adds a D3cold power cycle as a general PCI reset method for single-function devices. It implements pci_d3cold_reset() which attempts to transition the device through D3cold and back to D0, falling back to D3hot if ACPI _PR3 is unavailable.

Quoted 4 lines
drivers/pci/pci.c +0 -0
index 8f7cfcc000901..6da8feff7ccc3 100644

[ ... ]

Quoted 17 lines
drivers/pci/pci.c +13 -0
@@ -4491,6 +4491,46 @@ static int pci_pm_reset(struct pci_dev *dev, bool probe)
 	return ret;
 }
 
+/**
+ * pci_d3cold_reset - Put device into D3cold (or D3hot) and back to D0 for reset
+ * @dev: PCI device to reset
+ * @probe: if true, check if D3cold reset is supported; if false, perform reset
+ *
+ * Attempt to reset the device by transitioning through D3cold and back to D0.
+ * On platforms with ACPI _PR3 power resources, this performs a true D3cold
+ * power cycle (actual power removal). On platforms without _PR3 support,
+ * pci_set_power_state() automatically falls back to D3hot, providing a
+ * D3hot->D0 reset transition.
+ *
+ * Only available for single-function devices to avoid affecting other
+ * functions in multi-function devices.

drivers/pci/pci.c static int pci_pm_reset(struct pci_dev *dev, bool probe) · line 4516

Quoted 10 lines
drivers/pci/pci.c +10 -0
+ *
+ * Returns 0 if device can be/was reset this way, -ENOTTY if not supported,
+ * or other negative error code on failure.
+ */
+static int pci_d3cold_reset(struct pci_dev *dev, bool probe)
+{
+	int ret;
+
+	if (dev->multifunction)
+		return -ENOTTY;

Many PCIe bridges (like Root Ports and Downstream Switch Ports) are single-function devices.

If pci_d3cold_reset() is triggered on a single-function bridge, pci_set_power_state() will physically shut off power to the entire downstream hierarchy. Because pci_reset_function() is intended to reset only the target function, does this unexpectedly drop power to subordinate devices without unbinding their drivers or saving their state?

Should there be a check for dev->subordinate to prevent execution on bridges?

drivers/pci/pci.c static int pci_pm_reset(struct pci_dev *dev, bool probe) · line 4521

Quoted 5 lines
drivers/pci/pci.c +5 -0
+
+	if (probe) {
+		if (!pci_pr3_present(dev))
+			pci_dbg(dev, "d3cold reset: no _PR3 support, will use D3hot fallback\n");
+		return 0;

When called with probe = true, this unconditionally returns 0 if the device is single-function, even if it lacks both ACPI _PR3 and native PM capabilities (dev->pm_cap is not checked).

Could this incorrectly register d3cold as a supported reset method for hardware that physically cannot change power states?

drivers/pci/pci.c static int pci_pm_reset(struct pci_dev *dev, bool probe) · line 4527

Quoted 6 lines
drivers/pci/pci.c +6 -0
+	}
+
+	if (dev->current_state != PCI_D0)
+		return -EINVAL;
+
+	ret = pci_set_power_state(dev, PCI_D3cold);

Other PCI reset methods in the kernel bracket their operations with IOMMU coordination. For example:

static int pci_pm_reset(struct pci_dev *dev, bool probe)

{
        ...
        ret = pci_dev_reset_iommu_prepare(dev);
        ...
        pci_dev_reset_iommu_done(dev);
        return ret;
}

By omitting these calls, does pci_d3cold_reset() allow power to be removed while the device might still have in-flight DMA or active IOMMU mappings, potentially triggering spurious DMA faults or fatal IOMMU errors?

drivers/pci/pci.c static int pci_pm_reset(struct pci_dev *dev, bool probe) · line 4531

Quoted 4 lines
drivers/pci/pci.c +4 -0
+	if (ret)
+		return ret;
+
+	return pci_set_power_state(dev, PCI_D0);

When true D3cold is unavailable, pci_set_power_state() transparently falls back to putting the device into D3hot. If the device advertises the PCI_PM_CTRL_NO_SOFT_RESET bit, a D3hot->D0 transition explicitly does not reset the device's internal state.

Other reset methods like pci_pm_reset() explicitly check this:

static int pci_pm_reset(struct pci_dev *dev, bool probe)

{
        ...
        pci_read_config_word(dev, dev->pm_cap + PCI_PM_CTRL, &csr);
        if (csr & PCI_PM_CTRL_NO_SOFT_RESET)
                return -ENOTTY;
        ...
}

Since pci_d3cold_reset() does not check this bit and returns 0 on success, does this falsely report a successful reset to subsystems like VFIO that rely on this to wipe guest state, potentially leaking state across VM boundaries?

Additionally, returning a device from D3cold or D3hot to D0 constitutes a Conventional Reset, where devices may require up to 1 second to reinitialize and return Configuration Request Retry Status (CRS).

Existing methods ensure readiness before returning:

static int pci_pm_reset(struct pci_dev *dev, bool probe)

{
        ...
        ret = pci_dev_wait(dev, "PM D3hot->D0", PCIE_RESET_READY_POLL_MS);
        ...
        return ret;
}

Does returning immediately from pci_set_power_state() mean the caller might instantly invoke pci_dev_restore() while the hardware is still unresponsive?

Quoted 1 line
drivers/pci/pci.c +1 -0
+}

[ ... ]

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1