vfio/pci: Handle PCI error recovery and report state to userspace
Subsystems (3)
v2 cover vfio/pci: Handle PCI error recovery and report state to userspace
Hi,
Currently, vfio-pci signals the error eventfd from error_detected() and returns PCI_ERS_RESULT_CAN_RECOVER, including for permanent failure. It has no slot_reset() or resume() callback, so userspace is not notified when host recovery completes or whether the device was reset.
This series adds opt-in host PCI error recovery for vfio-pci. It blocks device access during recovery, restores device state after a host reset, and reports recovery status to userspace. The kernel runs host recovery and the VMM decides how to recover the guest.
Changes from v1
---------------
v1: https://lore.kernel.org/kvm/[email protected]/
- Thanks to Satya and Alex for reviews/feedback.
- Use SRCU to drain ongoing device accesses and reject new accesses during recovery. A mutex protects recovery-state updates.
- Return SIGBUS for blocked BAR faults instead of waiting in the fault handler. VMM handling still needs target validation.
- Reject userspace SR-IOV changes while access is blocked.
- Exclude s390 and all CXL memory devices for now as s390 and CXL RCH recovery can omit completion callbacks.
Design
------
Userspace opts in through VFIO_DEVICE_FEATURE_PCI_ERROR_RECOVERY. It registers an eventfd and reads status flags and a sequence number. The existing error eventfd is still signaled.
The callbacks handle the host recovery sequence:
- error_detected() blocks new accesses and drains admitted ones, quiesces INTx, and revokes BAR mappings and exported DMA-BUFs. On a normal channel it also clears bus mastering. It votes CAN_RECOVER for a normal channel, NEED_RESET for a frozen channel, and DISCONNECT for permanent failure. Failure to save PCI_COMMAND or disable bus mastering leaves access blocked and returns NONE so other devices can continue recovery.
- slot_reset() restores the configuration saved at open after a host reset, then tears down interrupt configuration. Userspace must set up interrupts again.
- resume() completes restoration, unblocks access and completes INTx handling. A restoration failure keeps the device blocked.
Non-fatal errors also block device access because host recovery may require a reset. Userspace reads the recovery status to determine whether the device was reset or recovery failed.
Without opt-in, the existing error notification behavior is retained. The open check also rejects a previously recorded host transaction when recovery is disabled, so closing the device does not discard that state.
Locking and open questions
--------------------------
Several locks are involved in recovery. PCI core holds device_lock while calling the driver, preventing concurrent driver binding or unbinding. During the bus walk, it also holds pci_bus_sem for reading to keep the device list stable. Adding or removing PCI devices requires the write lock. This semaphore is shared across all PCI buses.
VFIO recovery blocks new accesses, waits for existing SRCU readers to finish, and takes memory_lock. Power transitions also take memory_lock, but entering D0 can then acquire pci_bus_sem to update PCIe link power management (ASPM).
Local Sashiko/Claude review reports a lock-order conflict as below:
1. A guest D0 request holds memory_lock.
2. PCI core holds pci_bus_sem for reading while VFIO's recovery callback waits for memory_lock.
3. Removal of an unrelated PCI device waits for the bus write lock.
4. D0's ASPM update requests the bus read lock and waits behind that writer.
Each task waits for another, so none can proceed.
One possible solution could be(not implemented):
- PCI core guarantees that pci_bus_sem is held for reading before invoking supported recovery callbacks. VFIO uses pci_set_power_state_locked() in those callbacks, avoiding another acquisition of the bus lock.
- A new pci_try_set_power_state() helper tries to acquire pci_bus_sem and returns -EAGAIN on contention. Ordinary VFIO D0 paths use this helper and handle failure instead of waiting while holding VFIO locks
Not sure there are better ways to handle this or not. Feedbacks appreciated on this.
Another issue flagged was(I think this is a pre-existing one):
- Open/close coordination with the whole host recovery transaction was already missing. Recovery can start after the open check, and close can overlap the physical reset. Restoration of closed devices is also not implemented.
Testing
-------
Basic sanity tests performed on a GB200 with an NVIDIA GPU assigned.
Kernel branch:
https://github.com/shamiali2008/linux/tree/vfio-aer-rfc-v2-ext
QEMU test branch is here(This registers the recovery eventfd and uses pcie_aer_inject_error() to report the error to Guest)
https://github.com/shamiali2008/qemu-master/commits/private-master-vfio-aer-test-v2/
Software AER injection was performed using a modified pcieaer_inject module.
./aer-inject nonfatal.conf
qemu-system-aarch64: info: vfio 0018:06:00.0: host PCI error recovery completed (seq 1)
Guest kernel:
[ 59.036988] pcieport 0000:01:00.0: AER: Uncorrectable (Non-Fatal) error message received from 0000:02:00.0 [ 59.038326] nvidia 0000:02:00.0: PCIe Bus Error: severity=Uncorrectable (Non-Fatal), type=Transaction Layer, (Completer ID) [ 59.038440] nvidia 0000:02:00.0: device [10de:2941] error status/mask=00008000/02400000 [ 59.038533] nvidia 0000:02:00.0: [15] CmpltAbrt (First) [ 59.038705] nvidia 0000:02:00.0: AER: TLP Header: 0x00000000 0x00000000 0x00000000 0x00000000
...
./aer-inject fatal.conf
qemu-system-aarch64: info: vfio 0018:06:00.0: host PCI error recovery started (seq 2, channel frozen)
qemu-system-aarch64: info: vfio 0018:06:00.0: host PCI error recovery completed (seq 2, device was reset)
Guest kernel:
[ 150.909306] pcieport 0000:01:00.0: AER: Uncorrectable (Fatal) error message received from 0000:02:00.0 [ 150.909566] nvidia 0000:02:00.0: AER: PCIe Bus Error: severity=Uncorrectable (Fatal), type=Inaccessible, (Unregistered Agent ID) [ 150.909835] nvidia 0000:02:00.0: AER: can't recover (no error_detected callback) [ 150.910062] pcieport 0000:01:00.0: unlocked secondary bus reset via: pciehp_reset_slot+0x54/0x98 [ 156.603467] pcieport 0000:01:00.0: AER: Root Port link has been reset (0)
...
Please take a look and let me know your feedback.
Thanks,
Shameer
Shameer Kolothum (16):
vfio/pci: Add a device access gate
vfio/pci: Gate config space access
vfio/pci: Buffer ROM reads before copying to userspace vfio/pci: Gate BAR and ROM access
vfio/pci: Fail BAR faults while access is blocked vfio/pci: Gate interrupt configuration
vfio/pci: Gate function reset and runtime power management vfio/pci: Gate device information queries and DMA-BUF export vfio/pci: Add PCI error recovery state
vfio/pci: Quiesce INTx while access is blocked vfio/pci: Add INTx recovery start and finish helpers vfio/pci: Restore device state from slot_reset() vfio/pci: Complete recovery in resume()
vfio/pci: Block device access during host recovery vfio/pci: Add VFIO_DEVICE_FEATURE_PCI_ERROR_RECOVERY vfio/pci: Enable host PCI error recovery for vfio-pci
drivers/vfio/pci/vfio_pci_priv.h | 10 + include/linux/vfio_pci_core.h | 38 ++ include/uapi/linux/vfio.h | 44 ++ drivers/vfio/pci/vfio_pci.c | 12 + drivers/vfio/pci/vfio_pci_config.c | 104 ++++- drivers/vfio/pci/vfio_pci_core.c | 636 ++++++++++++++++++++++++++++- drivers/vfio/pci/vfio_pci_dmabuf.c | 21 +- drivers/vfio/pci/vfio_pci_intrs.c | 131 ++++++ drivers/vfio/pci/vfio_pci_rdwr.c | 169 ++++++-- 9 files changed, 1090 insertions(+), 75 deletions(-)
Signature · 1 line
2.43.0
RFC PATCH v2 1/16 vfio/pci: Add a device access gate
PCI error recovery needs to block new device accesses and drain outstanding accesses before changing device state.
Add an SRCU-based access gate, enabled by pci_recovery_supported. Serialize open and close state updates with access_lock.
--- drivers/vfio/pci/vfio_pci_priv.h | 3 ++ include/linux/vfio_pci_core.h | 19 ++++++++++++ drivers/vfio/pci/vfio_pci_core.c | 53 ++++++++++++++++++++++++++++++++ 3 files changed, 75 insertions(+)
drivers/vfio/pci/vfio_pci_priv.h +3 -0
@@ -73,6 +73,9 @@ u16 vfio_pci_memory_lock_and_enable(struct vfio_pci_core_device *vdev);void vfio_pci_memory_unlock_and_restore (struct vfio_pci_core_device *vdev,u16 cmd); +int vfio_pci_core_access_begin (struct vfio_pci_core_device *vdev); +void vfio_pci_core_access_end (struct vfio_pci_core_device *vdev,int idx); +#ifdef CONFIG_VFIO_PCI_IGD bool vfio_pci_is_intel_display (struct pci_dev *pdev);int vfio_pci_igd_init (struct vfio_pci_core_device *vdev);
include/linux/vfio_pci_core.h +19 -0
@@ -13,6 +13,7 @@#include <linux/vfio.h> #include <linux/irqbypass.h> #include <linux/rcupdate.h> +#include <linux/srcu.h> #include <linux/types.h> #include <linux/uuid.h> #include <linux/notifier.h>
@@ -129,6 +130,7 @@ struct vfio_pci_core_device { bool disable_idle_d3:1; bool nointxmask:1; bool disable_vga:1; + bool pci_recovery_supported:1;/* Flags modified at runtime - dedicated storage unit */ bool needs_reset;bool pm_intx_masked;
@@ -148,6 +150,23 @@ struct vfio_pci_core_device {struct vfio_pci_core_device *sriov_pf_core_dev;struct notifier_block nb;struct rw_semaphore memory_lock; +/* +* Device accesses take access_srcu and check device_open and +* access_blocked. To block access, set access_blocked and wait for +* existing readers with synchronize_srcu(). New accesses fail with +* -EIO while blocked. +* +* Close clears access_blocked before device_open. +* +* INTx paths use irqlock instead of access_srcu to coordinate with +* recovery. They read pci_recovery_enabled and access_blocked with +* READ_ONCE(), since writers do not always hold irqlock. +*/ +struct srcu_struct access_srcu; +bool access_blocked; +/* Set after open completes, cleared before close tears down state. */ +bool device_open; +struct mutex access_lock;/* gate flag writers */ struct list_head dmabufs; };
drivers/vfio/pci/vfio_pci_core.c +53 -0
99 lines not on this page · raw message
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [RFC PATCH v2 01/16] vfio/pci: Add a device access gate
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 8 lines
PCI error recovery needs to block new device accesses and drain outstanding accesses before changing device state.
Add an SRCU-based access gate, enabled by
pci_recovery_supported. Serialize open and close state updates withaccess_lock.Assisted-by LLMSigned-off-by Shameer Kolothum <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
RFC PATCH v2 2/16 vfio/pci: Gate config space access
In reply to Shameer KolothumProtect config callbacks with the access gate. Keep user copies outside SRCU because userfaultfd can stall them indefinitely.
Run D0 transitions after releasing SRCU to avoid waiting for pci_bus_sem while AER waits for readers to drain. Record blocked D0 requests for replay in resume(). Return -EIO for blocked D1/D2/D3 requests, which are not queued for replay. Preserve existing handling of hardware power-state errors outside recovery.
Add a power_up output parameter to writefn so the PM callback can request a D0 transition after the caller releases access_srcu.
---
Note: The memory_lock/pci_bus_sem dependency remains unresolved. See the cover letter's "Locking and open questions" section.
--- include/linux/vfio_pci_core.h | 2 + drivers/vfio/pci/vfio_pci_config.c | 104 +++++++++++++++++++++++------ 2 files changed, 84 insertions(+), 22 deletions(-)
include/linux/vfio_pci_core.h +2 -0
@@ -138,6 +138,8 @@ struct vfio_pci_core_device {bool sriov_active;struct pci_saved_state *pci_saved_state;struct pci_saved_state *pm_save; +/* Deferred D0 request, protected by memory_lock. */ +bool power_up_pending;int ioeventfds_nr;struct vfio_pci_eventfd __rcu *err_trigger;struct vfio_pci_eventfd __rcu *req_trigger;
drivers/vfio/pci/vfio_pci_config.c +82 -22
231 lines not on this page · raw message
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [RFC PATCH v2 02/16] vfio/pci: Gate config space access
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 9 lines
Protect config callbacks with the access gate. Keep user copies outside SRCU because userfaultfd can stall them indefinitely.
Run D0 transitions after releasing SRCU to avoid waiting for
pci_bus_semwhile AER waits for readers to drain. Record blocked D0 requests for replay inresume(). Return-EIOfor blocked D1/D2/D3 requests, which are not queued for replay. Preserve existing handling of hardware power-state errors outside recovery.
Quoted 5 lines
Add a
power_upoutput parameter to writefn so the PM callback can request a D0 transition after the caller releasesaccess_srcu.Assisted-by LLMSigned-off-by Shameer Kolothum <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
RFC PATCH v2 3/16 vfio/pci: Buffer ROM reads before copying to userspace
In reply to Shameer KolothumMove ROM reads into a separate helper because vfio_pci_core_do_io_rw() copies directly to userspace. Buffer the requested ROM data and unmap the ROM before copying it to userspace.
The next patch adds recovery gating around ROM mapping, reading and unmapping. Buffering keeps userspace faults outside that gate, so they cannot stall recovery or leave ROM reads to resume after recovery has disabled decoding.
Preserve aligned byte, word and dword reads and 0xff padding. The temporary allocation can fail with -ENOMEM.
--- drivers/vfio/pci/vfio_pci_rdwr.c | 147 ++++++++++++++++++++++--------- 1 file changed, 104 insertions(+), 43 deletions(-)
drivers/vfio/pci/vfio_pci_rdwr.c +104 -43
@@ -10,8 +10,10 @@* Author: Tom Lyon, [email protected] */ +#include <linux/align.h> #include <linux/fs.h> #include <linux/pci.h> +#include <linux/slab.h> #include <linux/uaccess.h> #include <linux/io.h> #include <linux/vfio.h>
@@ -198,6 +200,96 @@ ssize_t vfio_pci_core_do_io_rw(struct vfio_pci_core_device *vdev, bool test_mem, }EXPORT_SYMBOL_GPL (vfio_pci_core_do_io_rw); +static ssize_t vfio_pci_rom_read (struct vfio_pci_core_device *vdev, +char __user *buf,size_t count,loff_t pos) +{ +struct pci_dev *pdev = vdev->pdev; +bool rom_bar =pci_resource_start (pdev,PCI_ROM_RESOURCE ); +size_t size, length =0 , done =0 ; +void *data =NULL ; +void __iomem *io; +ssize_t ret; + +/* Serialize ROM decoding with power-state and other ROM accesses. */ +down_write (&vdev->memory_lock); +if (rom_bar) { + io =pci_map_rom (pdev, &size); + }else { + io =ioremap (pdev->rom, pdev->romlen); + size = pdev->romlen; + } +if (!io) { + ret = -ENOMEM ; + goto out_unlock; + } + +/* Buffer only ROM data, not unused space in a large ROM BAR. */ +if (pos < size) + length =min (count, size - (size_t )pos); +if (length) { +if ((pci_resource_flags (pdev,PCI_ROM_RESOURCE ) &IORESOURCE_MEM ) && + !__vfio_pci_memory_enabled (vdev)) { + ret = -EIO ; + goto out_unmap; + } + data =kvmalloc (length,GFP_KERNEL_ACCOUNT ); +if (!data) { + ret = -ENOMEM ; + goto out_unmap; + } + } + +/* +* Certain devices (e.g. Intel X710) don't support qword +* access to the ROM bar. Otherwise PCI AER errors might be +* triggered. +* +* Disable qword access to the ROM bar universally, which +* worked reliably for years before qword access is enabled. +*/ +while (done < length) { +if (length - done >=4 &&IS_ALIGNED (pos + done,4 )) { +u32 val =vfio_ioread32 (io + pos + done); + +memcpy (data + done, &val,sizeof (val)); + done +=sizeof (val); + }else if (length - done >=2 &&IS_ALIGNED (pos + done,2 )) { +u16 val =vfio_ioread16 (io + pos + done); + +memcpy (data + done, &val,sizeof (val)); + done +=sizeof (val); + }else { + ((u8 *)data)[done] =vfio_ioread8 (io + pos + done); + done++; + } + } + ret = count; + +out_unmap: +if (rom_bar) +pci_unmap_rom (pdev, io); +else +iounmap (io); +out_unlock: +up_write (&vdev->memory_lock); +if (ret <0 ) + goto out_free; + +if (length &©_to_user (buf, data, length)) { + ret = -EFAULT ; + goto out_free; + } +for (done = length; done < count; done++) { +if (put_user ((u8 )0xff , buf + done)) { + ret = -EFAULT ; +break ; + } + } +out_free: +kvfree (data); +return ret; +} +ssize_t vfio_pci_bar_rw (struct vfio_pci_core_device *vdev,char __user *buf,size_t count,loff_t *ppos,bool iswrite) {
@@ -209,7 +301,6 @@ ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *buf,void __iomem *io;struct resource *res = &vdev->pdev->resource[bar];ssize_t done; -enum vfio_pci_io_width max_width =VFIO_PCI_IO_WIDTH_8 ;if (pci_resource_start (pdev, bar)) end =pci_resource_len (pdev, bar);
@@ -224,57 +315,27 @@ ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *buf, count =min (count, (size_t )(end - pos));if (bar ==PCI_ROM_RESOURCE ) { -/* -* The ROM can fill less space than the BAR, so we start the -* excluded range at the end of the actual ROM. This makes -* filling large ROM BARs much faster. -*/ -if (pci_resource_start (pdev, bar)) { - io =pci_map_rom (pdev, &x_start); - }else { - io =ioremap (pdev->rom, pdev->romlen); - x_start = pdev->romlen; - } -if (!io) -return -ENOMEM ; - x_end = end; - -/* -* Certain devices (e.g. Intel X710) don't support qword -* access to the ROM bar. Otherwise PCI AER errors might be -* triggered. -* -* Disable qword access to the ROM bar universally, which -* worked reliably for years before qword access is enabled. -*/ - max_width =VFIO_PCI_IO_WIDTH_4 ; +if (iswrite) +return -EINVAL ; + done =vfio_pci_rom_read (vdev, buf, count, pos); }else { io =vfio_pci_core_get_iomap (vdev, bar); -if (IS_ERR (io)) { - done =PTR_ERR (io); - goto out; +if (IS_ERR (io)) +return PTR_ERR (io); + +if (bar == vdev->msix_bar) { + x_start = vdev->msix_offset; + x_end = vdev->msix_offset + vdev->msix_size; } - } -if (bar == vdev->msix_bar) { - x_start = vdev->msix_offset; - x_end = vdev->msix_offset + vdev->msix_size; + done =vfio_pci_core_do_io_rw (vdev, res->flags &IORESOURCE_MEM , + io, buf, pos, count, x_start, x_end, + iswrite,VFIO_PCI_IO_WIDTH_8 ); } - done =vfio_pci_core_do_io_rw (vdev, res->flags &IORESOURCE_MEM , io, buf, pos, - count, x_start, x_end, iswrite, max_width); -if (done >=0 ) *ppos += done; -if (bar ==PCI_ROM_RESOURCE ) { -if (pci_resource_start (pdev, bar)) -pci_unmap_rom (pdev, io); -else -iounmap (io); - } - -out:return done; }
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [RFC PATCH v2 03/16] vfio/pci: Buffer ROM reads before copying to userspace
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] vfio/pci: soft lockup in vfio_pci_rom_read loops due to missing cond_resched --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] vfio/pci: soft lockup in vfio_pci_rom_read loops due to missing cond_resched --
commit 27d4f4cebe481807304449721e6097b02db9fc50
Author: Shameer Kolothum <[email protected]>
vfio/pci: Buffer ROM reads before copying to userspace
This commit moves ROM reads into a separate helper, buffering the requested ROM data and unmapping the ROM before copying it to userspace. This helps keep userspace faults outside the recovery gate.
Quoted 3 lines
drivers/vfio/pci/vfio_pci_rdwr.c +0
-0
[ ... ]
Quoted 3 lines
drivers/vfio/pci/vfio_pci_rdwr.c +3
-0+static ssize_t vfio_pci_rom_read (struct vfio_pci_core_device *vdev, +char __user *buf,size_t count,loff_t pos) +{
[ ... ]
drivers/vfio/pci/vfio_pci_rdwr.c ssize_t vfio_pci_core_do_io_rw(struct vfio_pci_core_device *vdev, bool test_mem, · line 252
Quoted 11 lines
drivers/vfio/pci/vfio_pci_rdwr.c +11
-0+/* +* Certain devices (e.g. Intel X710) don't support qword +* access to the ROM bar. Otherwise PCI AER errors might be +* triggered. +* +* Disable qword access to the ROM bar universally, which +* worked reliably for years before qword access is enabled. +*/ +while (done < length) { +if (length - done >=4 &&IS_ALIGNED (pos + done,4 )) { +u32 val =vfio_ioread32 (io + pos + done);
[Severity: High]
Can this loop cause a soft lockup on large ROM reads?
If a userspace application or guest issues a large read on the ROM BAR (for example, 16MB), this while loop in vfio_pci_rom_read() will perform millions of synchronous MMIO reads without yielding.
Since each MMIO read can take significant time, this could stall the CPU core for seconds and trigger a soft lockup panic. Should this loop include a cond_resched() to prevent an untrusted guest from causing a local Denial of Service?
Quoted 4 lines
drivers/vfio/pci/vfio_pci_rdwr.c +4
-0+ +memcpy (data + done, &val,sizeof (val)); + done +=sizeof (val); + }else if (length - done >=2 &&IS_ALIGNED (pos + done,2 )) {
[ ... ]
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
RFC PATCH v2 4/16 vfio/pci: Gate BAR and ROM access
In reply to Shameer KolothumProtect the common I/O accessors used for BAR, VGA, port I/O and ioeventfd accesses with access_srcu. Recovery can then block new accesses and wait for admitted accesses to finish. Keep user copies outside SRCU so a userspace fault cannot stall recovery.
Return -EIO before runtime resume if region access is already blocked. This check avoids waking the device unnecessarily; it does not serialize runtime resume against recovery. The I/O accessors check the gate again under SRCU before touching hardware.
Hold one SRCU section across ROM mapping, the buffered hardware read and unmapping. Copy the snapshot to userspace afterward, so recovery cannot disable ROM decoding between read chunks or wait for a userspace fault.
--- drivers/vfio/pci/vfio_pci_core.c | 4 ++++ drivers/vfio/pci/vfio_pci_rdwr.c | 22 ++++++++++++++++++++++ 2 files changed, 26 insertions(+)
drivers/vfio/pci/vfio_pci_core.c +4 -0
@@ -1666,6 +1666,10 @@ static ssize_t vfio_pci_rw(struct vfio_pci_core_device *vdev, char __user *buf,if (index >=VFIO_PCI_NUM_REGIONS + vdev->num_regions)return -EINVAL ; +/* Avoid runtime resume when blocked; region handlers check access again. */ +if (vdev->pci_recovery_supported &&READ_ONCE (vdev->access_blocked)) +return -EIO ; + ret =pm_runtime_resume_and_get (&vdev->pdev->dev);if (ret) { pci_info_ratelimited(vdev->pdev, "runtime resume failed %d\n",
drivers/vfio/pci/vfio_pci_rdwr.c +22 -0
@@ -44,10 +44,17 @@ int vfio_pci_core_iowrite##size(struct vfio_pci_core_device *vdev, \ bool test_mem, u##size val, void __iomem *io) \ { \ + int idx; \ + \ + idx = vfio_pci_core_access_begin(vdev); \ + if (idx < 0) \ + return idx; \ + \ if (test_mem) { \ down_read(&vdev->memory_lock); \ if (!__vfio_pci_memory_enabled(vdev)) { \ up_read(&vdev->memory_lock); \ + vfio_pci_core_access_end(vdev, idx); \ return -EIO; \ } \ } \
@@ -56,6 +63,7 @@ int vfio_pci_core_iowrite##size(struct vfio_pci_core_device *vdev, \ \if (test_mem) \up_read (&vdev->memory_lock); \ +vfio_pci_core_access_end (vdev, idx); \ \return 0 ; \ } \
@@ -70,10 +78,17 @@ VFIO_IOWRITE(64) int vfio_pci_core_ioread##size(struct vfio_pci_core_device *vdev, \ bool test_mem, u##size *val, void __iomem *io) \ { \ + int idx; \ + \ + idx = vfio_pci_core_access_begin(vdev); \ + if (idx < 0) \ + return idx; \ + \ if (test_mem) { \ down_read(&vdev->memory_lock); \ if (!__vfio_pci_memory_enabled(vdev)) { \ up_read(&vdev->memory_lock); \ + vfio_pci_core_access_end(vdev, idx); \ return -EIO; \ } \ } \
@@ -82,6 +97,7 @@ int vfio_pci_core_ioread##size(struct vfio_pci_core_device *vdev, \ \if (test_mem) \up_read (&vdev->memory_lock); \ +vfio_pci_core_access_end (vdev, idx); \ \return 0 ; \ } \
@@ -209,6 +225,11 @@ static ssize_t vfio_pci_rom_read(struct vfio_pci_core_device *vdev,void *data =NULL ;void __iomem *io;ssize_t ret; +int idx; + + idx =vfio_pci_core_access_begin (vdev); +if (idx <0 ) +return idx;/* Serialize ROM decoding with power-state and other ROM accesses. */ down_write (&vdev->memory_lock);
@@ -272,6 +293,7 @@ static ssize_t vfio_pci_rom_read(struct vfio_pci_core_device *vdev,iounmap (io); out_unlock:up_write (&vdev->memory_lock); +vfio_pci_core_access_end (vdev, idx);if (ret <0 ) goto out_free;
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [RFC PATCH v2 04/16] vfio/pci: Gate BAR and ROM access
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
Protect the common I/O accessors used for BAR, VGA, port I/O and ioeventfd accesses with
access_srcu. Recovery can then block new accesses and wait for admitted accesses to finish. Keep user copies outside SRCU so a userspace fault cannot stall recovery.Return
-EIObefore runtime resume if region access is already blocked. This check avoids waking the device unnecessarily; it does not serialize runtime resume against recovery. The I/O accessors check the gate again under SRCU before touching hardware.
Quoted 6 lines
Hold one SRCU section across ROM mapping, the buffered hardware read and unmapping. Copy the snapshot to userspace afterward, so recovery cannot disable ROM decoding between read chunks or wait for a userspace fault.
Assisted-by LLMSigned-off-by Shameer Kolothum <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
RFC PATCH v2 5/16 vfio/pci: Fail BAR faults while access is blocked
In reply to Shameer KolothumHold access_srcu across PFN insertion in the BAR fault handler. This ensures recovery waits for an insertion before revoking the mapping.
Return VM_FAULT_SIGBUS rather than wait for recovery in the fault handler when access is blocked. Whether userspace can handle the resulting failure depends on the architecture and KVM fault-reporting path.
--- drivers/vfio/pci/vfio_pci_core.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
drivers/vfio/pci/vfio_pci_core.c +11 -0
@@ -1842,8 +1842,19 @@ static vm_fault_t vfio_pci_mmap_huge_fault(struct vm_fault *vmf,vm_fault_t ret =VM_FAULT_FALLBACK ;if (is_aligned_for_order (vma, addr, pfn, order)) { +int idx; + +/* +* Hold access_srcu across PFN insertion so recovery cannot +* finish revoking mappings before this insertion completes. +*/ + idx =vfio_pci_core_access_begin (vdev); +if (idx <0 ) +return VM_FAULT_SIGBUS ; +scoped_guard (rwsem_read, &vdev->memory_lock) ret = vfio_pci_vmf_insert_pfn(vdev, vmf, pfn, order); +vfio_pci_core_access_end (vdev, idx); } dev_dbg_ratelimited(&vdev->pdev->dev,
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [RFC PATCH v2 05/16] vfio/pci: Fail BAR faults while access is blocked
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
Hold
access_srcuacross PFN insertion in the BAR fault handler. This ensures recovery waits for an insertion before revoking the mapping.Return
VM_FAULT_SIGBUSrather than wait for recovery in the fault handler when access is blocked. Whether userspace can handle the resulting failure depends on the architecture and KVM fault-reporting path.Assisted-by LLMSigned-off-by Shameer Kolothum <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
RFC PATCH v2 6/16 vfio/pci: Gate interrupt configuration
In reply to Shameer KolothumProtect INTx, MSI and MSI-X hardware accesses in GET_IRQ_INFO and SET_IRQS with access_srcu. ERR and REQ only update eventfds, so leave them available while device access is blocked.
SET_IRQS uses separate read sections for validation and configuration, with the user copy between them. Recovery must also wait for IRQ teardown to finish flushing the global virqfd workqueue.
--- drivers/vfio/pci/vfio_pci_core.c | 54 ++++++++++++++++++++++++++++++-- 1 file changed, 51 insertions(+), 3 deletions(-)
drivers/vfio/pci/vfio_pci_core.c +51 -3
@@ -1266,6 +1266,19 @@ int vfio_pci_ioctl_get_region_info(struct vfio_device *core_vdev, }EXPORT_SYMBOL_GPL (vfio_pci_ioctl_get_region_info); +/* ERR and REQ do not access hardware. */ +static bool vfio_pci_irq_index_is_device (u32 index) +{ +switch (index) { +case VFIO_PCI_INTX_IRQ_INDEX : +case VFIO_PCI_MSI_IRQ_INDEX : +case VFIO_PCI_MSIX_IRQ_INDEX : +return true; +default : +return false; + } +} +static int vfio_pci_ioctl_get_irq_info (struct vfio_pci_core_device *vdev,struct vfio_irq_info __user *arg) {
@@ -1289,7 +1302,16 @@ static int vfio_pci_ioctl_get_irq_info(struct vfio_pci_core_device *vdev, info.flags =VFIO_IRQ_INFO_EVENTFD ; - info.count =vfio_pci_get_irq_count (vdev, info.index); +if (vfio_pci_irq_index_is_device (info.index)) { +int idx =vfio_pci_core_access_begin (vdev); + +if (idx <0 ) +return idx; + info.count =vfio_pci_get_irq_count (vdev, info.index); +vfio_pci_core_access_end (vdev, idx); + }else { + info.count =vfio_pci_get_irq_count (vdev, info.index); + }if (info.index ==VFIO_PCI_INTX_IRQ_INDEX ) info.flags |=
@@ -1306,13 +1328,24 @@ static int vfio_pci_ioctl_set_irqs(struct vfio_pci_core_device *vdev, unsigned long minsz = offsetofend(struct vfio_irq_set, count);struct vfio_irq_set hdr;u8 *data =NULL ; -int max, ret =0 ; +bool device_irq; +int max, ret =0 , idx =0 ;size_t data_size =0 ;if (copy_from_user (&hdr, arg, minsz))return -EFAULT ; - max =vfio_pci_get_irq_count (vdev, hdr.index); +/* Use separate SRCU sections to keep user copies outside access_srcu. */ + device_irq =vfio_pci_irq_index_is_device (hdr.index); +if (device_irq) { + idx =vfio_pci_core_access_begin (vdev); +if (idx <0 ) +return idx; + max =vfio_pci_get_irq_count (vdev, hdr.index); +vfio_pci_core_access_end (vdev, idx); + }else { + max =vfio_pci_get_irq_count (vdev, hdr.index); + } ret =vfio_set_irqs_validate_and_prepare (&hdr, max,VFIO_PCI_NUM_IRQS , &data_size);
@@ -1325,12 +1358,27 @@ static int vfio_pci_ioctl_set_irqs(struct vfio_pci_core_device *vdev,return PTR_ERR (data); } +/* +* IRQ teardown flushes the global virqfd workqueue. Recovery must +* wait for that flush, including pending work for other devices. +*/ +if (device_irq) { + idx =vfio_pci_core_access_begin (vdev); +if (idx <0 ) { + ret = idx; + goto out_free; + } + } +/* SRCU drains accesses for recovery; igate serializes IRQ changes. */ mutex_lock (&vdev->igate); ret =vfio_pci_set_irqs_ioctl (vdev, hdr.flags, hdr.index, hdr.start, hdr.count, data);mutex_unlock (&vdev->igate); +if (device_irq) +vfio_pci_core_access_end (vdev, idx); +out_free:kfree (data);return ret;
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [RFC PATCH v2 06/16] vfio/pci: Gate interrupt configuration
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
Protect INTx, MSI and MSI-X hardware accesses in
GET_IRQ_INFOandSET_IRQSwithaccess_srcu. ERR and REQ only update eventfds, so leave them available while device access is blocked.
SET_IRQSuses separate read sections for validation and configuration, with the user copy between them. Recovery must also wait for IRQ teardown to finish flushing the global virqfd workqueue.Assisted-by LLMSigned-off-by Shameer Kolothum <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
RFC PATCH v2 7/16 vfio/pci: Gate function reset and runtime power management
In reply to Shameer KolothumReject function reset and low-power entry or exit while device access is blocked. Hold access_srcu across reset and runtime power-management updates so recovery waits for admitted operations to finish.
Perform the reset ioctl's D0 transition before taking SRCU, preserving VFIO's pm_save handling. Keep memory_lock held across D0 preparation and reset so a concurrent guest D3 write cannot intervene. Check the blocked state before D0 and check SRCU admission afterward. A rejected reset does not queue a deferred D0 request.
The reset uses pci_try_reset_function(), which returns -EAGAIN if AER holds device_lock.
---
Note: D0 remains outside SRCU, but this series introduces a recovery locking dependency involving memory_lock and pci_bus_sem through ASPM. This remains unresolved. See the cover letter's "Locking and open questions" section.
--- drivers/vfio/pci/vfio_pci_priv.h | 3 +++ drivers/vfio/pci/vfio_pci_config.c | 4 ++-- drivers/vfio/pci/vfio_pci_core.c | 32 ++++++++++++++++++++++++++++-- 3 files changed, 35 insertions(+), 4 deletions(-)
drivers/vfio/pci/vfio_pci_priv.h +3 -0
@@ -41,6 +41,9 @@ ssize_t vfio_pci_config_rw_single(struct vfio_pci_core_device *vdev, char __user *buf, size_t count, loff_t *ppos, bool iswrite); +int vfio_lock_and_set_power_state (struct vfio_pci_core_device *vdev, +pci_power_t state); +ssize_t vfio_pci_bar_rw (struct vfio_pci_core_device *vdev,char __user *buf,size_t count,loff_t *ppos,bool iswrite);
drivers/vfio/pci/vfio_pci_config.c +2 -2
@@ -720,8 +720,8 @@ static int __init init_pci_cap_basic_perm(struct perm_bits *perm)* It takes all the required locks to protect the access of power related * variables and then invokes vfio_pci_set_power_state(). */ -static int vfio_lock_and_set_power_state (struct vfio_pci_core_device *vdev, -pci_power_t state) +int vfio_lock_and_set_power_state (struct vfio_pci_core_device *vdev, +pci_power_t state) {if (state >= PCI_D3hot) {vfio_pci_zap_and_down_write_memory_lock (vdev);
drivers/vfio/pci/vfio_pci_core.c +30 -2
@@ -372,6 +372,12 @@ int vfio_pci_set_power_state(struct vfio_pci_core_device *vdev, pci_power_t statstatic int vfio_pci_runtime_pm_entry (struct vfio_pci_core_device *vdev,struct eventfd_ctx *efdctx) { +int idx; + + idx =vfio_pci_core_access_begin (vdev); +if (idx <0 ) +return idx; +/* * The vdev power related flags are protected with 'memory_lock' * semaphore.
@@ -381,6 +387,7 @@ static int vfio_pci_runtime_pm_entry(struct vfio_pci_core_device *vdev,if (vdev->pm_runtime_engaged) {up_write (&vdev->memory_lock); +vfio_pci_core_access_end (vdev, idx);return -EINVAL ; }
@@ -388,6 +395,7 @@ static int vfio_pci_runtime_pm_entry(struct vfio_pci_core_device *vdev, vdev->pm_wake_eventfd_ctx = efdctx;pm_runtime_put_noidle (&vdev->pdev->dev);up_write (&vdev->memory_lock); +vfio_pci_core_access_end (vdev, idx);return 0 ; }
@@ -470,7 +478,7 @@ static void vfio_pci_runtime_pm_exit(struct vfio_pci_core_device *vdev)static int vfio_pci_core_pm_exit (struct vfio_pci_core_device *vdev,u32 flags,void __user *arg,size_t argsz) { -int ret; +int ret, idx; ret =vfio_check_feature (flags, argsz,VFIO_DEVICE_FEATURE_SET ,0 );if (ret !=1 )
@@ -483,7 +491,11 @@ static int vfio_pci_core_pm_exit(struct vfio_pci_core_device *vdev, u32 flags,* already signaled the eventfd and exited low power mode itself. * pm_runtime_engaged protects the redundant call here. */ + idx =vfio_pci_core_access_begin (vdev); +if (idx <0 ) +return idx;vfio_pci_runtime_pm_exit (vdev); +vfio_pci_core_access_end (vdev, idx);return 0 ; }
@@ -1387,12 +1399,16 @@ static int vfio_pci_ioctl_set_irqs(struct vfio_pci_core_device *vdev,static int vfio_pci_ioctl_reset (struct vfio_pci_core_device *vdev,void __user *arg) { -int ret; +int ret, idx;if (!vdev->reset_works)return -EINVAL ;vfio_pci_zap_and_down_write_memory_lock (vdev); +if (vdev->pci_recovery_supported &&READ_ONCE (vdev->access_blocked)) { +up_write (&vdev->memory_lock); +return -EIO ; + }/* * This function can be invoked while the power state is non-D0. If
@@ -1402,15 +1418,27 @@ static int vfio_pci_ioctl_reset(struct vfio_pci_core_device *vdev,* have NoSoftRst-, the reset function can cause the PCI config space * reset without restoring the original state (saved locally in * 'vdev->pm_save'). +* +* Do the D0 transition outside access_srcu because it can acquire +* pci_bus_sem through ASPM. Keep memory_lock held through reset to +* prevent a concurrent power-state write from putting the device in D3. */ vfio_pci_set_power_state (vdev,PCI_D0 ); + idx =vfio_pci_core_access_begin (vdev); +if (idx <0 ) { +up_write (&vdev->memory_lock); +return idx; + } +vfio_pci_dma_buf_move (vdev, true); ret =pci_try_reset_function (vdev->pdev);if (__vfio_pci_memory_enabled (vdev))vfio_pci_dma_buf_move (vdev, false);up_write (&vdev->memory_lock); +vfio_pci_core_access_end (vdev, idx); +return ret; }
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [RFC PATCH v2 07/16] vfio/pci: Gate function reset and runtime power management
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
Reject function reset and low-power entry or exit while device access is blocked. Hold
access_srcuacross reset and runtime power-management updates so recovery waits for admitted operations to finish.Perform the reset ioctl's D0 transition before taking SRCU, preserving VFIO's
pm_savehandling. Keepmemory_lockheld across D0 preparation and reset so a concurrent guest D3 write cannot intervene. Check the blocked state before D0 and check SRCU admission afterward. A rejected reset does not queue a deferred D0 request.
Quoted 5 lines
The reset uses
pci_try_reset_function(), which returns-EAGAINif AER holdsdevice_lock.Assisted-by LLMSigned-off-by Shameer Kolothum <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
RFC PATCH v2 8/16 vfio/pci: Gate device information queries and DMA-BUF export
In reply to Shameer KolothumGET_INFO accesses PCI configuration to report AtomicOps support, and GET_REGION_INFO maps the ROM to validate its contents. Protect these hardware accesses with access_srcu.
Hold SRCU through DMA-BUF creation and list insertion so recovery cannot finish revoking buffers before a concurrent export becomes visible.
--- drivers/vfio/pci/vfio_pci_core.c | 13 ++++++++++++- drivers/vfio/pci/vfio_pci_dmabuf.c | 16 +++++++++++++--- 2 files changed, 25 insertions(+), 4 deletions(-)
drivers/vfio/pci/vfio_pci_core.c +12 -1
@@ -1112,7 +1112,7 @@ static int vfio_pci_ioctl_get_info(struct vfio_pci_core_device *vdev, unsigned long minsz = offsetofend(struct vfio_device_info, num_irqs);struct vfio_device_info info = {};struct vfio_info_cap caps = { .buf =NULL , .size =0 }; -int ret; +int ret, idx;if (copy_from_user (&info, arg, minsz))return -EFAULT ;
@@ -1137,7 +1137,13 @@ static int vfio_pci_ioctl_get_info(struct vfio_pci_core_device *vdev,return ret; } + idx =vfio_pci_core_access_begin (vdev); +if (idx <0 ) { +kfree (caps.buf); +return idx; + } ret =vfio_pci_info_atomic_cap (vdev, &caps); +vfio_pci_core_access_end (vdev, idx);if (ret && ret != -ENODEV ) {pci_warn (vdev->pdev,"Failed to setup AtomicOps info capability\n" );
@@ -1213,6 +1219,10 @@ int vfio_pci_ioctl_get_region_info(struct vfio_device *core_vdev,* Check ROM content is valid. Need to enable memory * decode for ROM access in pci_map_rom(). */ +int idx =vfio_pci_core_access_begin (vdev); + +if (idx <0 ) +return idx; cmd =vfio_pci_memory_lock_and_enable (vdev); io =pci_map_rom (pdev, &size);if (io) {
@@ -1223,6 +1233,7 @@ int vfio_pci_ioctl_get_region_info(struct vfio_device *core_vdev,pci_unmap_rom (pdev, io); }vfio_pci_memory_unlock_and_restore (vdev, cmd); +vfio_pci_core_access_end (vdev, idx); }else if (pdev->rom && pdev->romlen) { info->flags =VFIO_REGION_INFO_FLAG_READ ;/* Report BAR size as power of two. */
drivers/vfio/pci/vfio_pci_dmabuf.c +13 -3
@@ -227,7 +227,7 @@ int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags,DEFINE_DMA_BUF_EXPORT_INFO (exp_info);struct vfio_pci_dma_buf *priv;size_t length; -int ret; +int ret, idx;if (!vdev->pci_ops || !vdev->pci_ops->get_dmabuf_phys)return -EOPNOTSUPP ;
@@ -274,19 +274,26 @@ int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags, priv->vdev = vdev; priv->nr_ranges = get_dma_buf.nr_ranges; priv->size = length; + + idx =vfio_pci_core_access_begin (vdev); +if (idx <0 ) { + ret = idx; + goto err_free_phys; + } + ret = vdev->pci_ops->get_dmabuf_phys (vdev, &priv->provider, get_dma_buf.region_index, priv->phys_vec, dma_ranges, priv->nr_ranges);if (ret) - goto err_free_phys; + goto err_access;kfree (dma_ranges); dma_ranges =NULL ;if (!vfio_device_try_get_registration (&vdev->vdev)) { ret = -ENODEV ; - goto err_free_phys; + goto err_access; } exp_info.ops = &vfio_pci_dmabuf_ops;
@@ -311,6 +318,7 @@ int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags,list_add_tail (&priv->dmabufs_elm, &vdev->dmabufs);dma_resv_unlock (priv->dmabuf->resv);up_write (&vdev->memory_lock); +vfio_pci_core_access_end (vdev, idx);/* * dma_buf_fd() consumes the reference, when the file closes the dmabuf
@@ -324,6 +332,8 @@ int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags, err_dev_put:vfio_device_put_registration (&vdev->vdev); +err_access: +vfio_pci_core_access_end (vdev, idx); err_free_phys:kfree (priv->phys_vec); err_free_priv:
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [RFC PATCH v2 08/16] vfio/pci: Gate device information queries and DMA-BUF export
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 9 lines
GET_INFOaccesses PCI configuration to report AtomicOps support, andGET_REGION_INFOmaps the ROM to validate its contents. Protect these hardware accesses withaccess_srcu.Hold SRCU through DMA-BUF creation and list insertion so recovery cannot finish revoking buffers before a concurrent export becomes visible.
Assisted-by LLMSigned-off-by Shameer Kolothum <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
RFC PATCH v2 9/16 vfio/pci: Add PCI error recovery state
In reply to Shameer KolothumAdd recovery flags, a sequence number, a saved PCI_COMMAND value and an eventfd for recovery notifications. Initialize the per-open state at open and clear it at close.
Track the host transaction separately from the userspace session. The host-active flag survives close so a later patch can reject reopen until the outstanding recovery completes.
--- include/linux/vfio_pci_core.h | 19 ++++++++++++++++++- drivers/vfio/pci/vfio_pci_core.c | 11 +++++++++++ 2 files changed, 29 insertions(+), 1 deletion(-)
include/linux/vfio_pci_core.h +18 -1
@@ -96,6 +96,11 @@ static inline int vfio_pci_core_get_dmabuf_phys( }#endif +#define VFIO_PCI_RECOVERY_IN_PROGRESS BIT(0) +#define VFIO_PCI_RECOVERY_FROZEN BIT(1) +#define VFIO_PCI_RECOVERY_RESET BIT(2) +#define VFIO_PCI_RECOVERY_FAILED BIT(3) +struct vfio_pci_core_device {struct vfio_device vdev;struct pci_dev *pdev;
@@ -143,6 +148,7 @@ struct vfio_pci_core_device {int ioeventfds_nr;struct vfio_pci_eventfd __rcu *err_trigger;struct vfio_pci_eventfd __rcu *req_trigger; +struct vfio_pci_eventfd __rcu *pci_recovery_trigger;struct eventfd_ctx *pm_wake_eventfd_ctx;struct list_head dummy_resources_list;struct mutex ioeventfds_lock;
@@ -168,7 +174,18 @@ struct vfio_pci_core_device {bool access_blocked;/* Set after open completes, cleared before close tears down state. */ bool device_open; -struct mutex access_lock;/* gate flag writers */ +struct mutex access_lock;/* gate and recovery state writers */ +u32 pci_recovery_flags; +u64 pci_recovery_sequence; +/* Saved PCI_COMMAND, valid when pci_recovery_command_valid is set. */ +u16 pci_recovery_command; +bool pci_recovery_enabled; +bool pci_recovery_command_valid; +/* +* Host recovery in progress, protected by access_lock. Unlike the +* per-open recovery flags, this is preserved across close. +*/ +bool pci_recovery_host_active;struct list_head dmabufs; };
drivers/vfio/pci/vfio_pci_core.c +11 -0
@@ -845,8 +845,11 @@ void vfio_pci_core_close_device(struct vfio_device *core_vdev)if (vdev->pci_recovery_supported) { scoped_guard(mutex, &vdev->access_lock) { +WRITE_ONCE (vdev->pci_recovery_enabled, false); + vdev->pci_recovery_command_valid = false;WRITE_ONCE (vdev->access_blocked, false);WRITE_ONCE (vdev->device_open, false); +WRITE_ONCE (vdev->pci_recovery_flags,0 ); } }
@@ -866,6 +869,10 @@ void vfio_pci_core_close_device(struct vfio_device *core_vdev)mutex_lock (&vdev->igate);vfio_pci_eventfd_replace_locked (vdev, &vdev->err_trigger,NULL );vfio_pci_eventfd_replace_locked (vdev, &vdev->req_trigger,NULL ); +if (vdev->pci_recovery_supported) +vfio_pci_eventfd_replace_locked (vdev, + &vdev->pci_recovery_trigger, +NULL );mutex_unlock (&vdev->igate); }EXPORT_SYMBOL_GPL (vfio_pci_core_close_device);
@@ -885,6 +892,10 @@ void vfio_pci_core_finish_enable(struct vfio_pci_core_device *vdev)if (vdev->pci_recovery_supported) {guard (mutex)(&vdev->access_lock); +WRITE_ONCE (vdev->pci_recovery_flags,0 ); + vdev->pci_recovery_sequence =0 ; + vdev->pci_recovery_command_valid = false; +WRITE_ONCE (vdev->pci_recovery_enabled, false);WRITE_ONCE (vdev->access_blocked, false);WRITE_ONCE (vdev->device_open, true); }
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [RFC PATCH v2 09/16] vfio/pci: Add PCI error recovery state
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 9 lines
Add recovery flags, a sequence number, a saved
PCI_COMMANDvalue and an eventfd for recovery notifications. Initialize the per-open state at open and clear it at close.Track the host transaction separately from the userspace session. The host-active flag survives close so a later patch can reject reopen until the outstanding recovery completes.
Signed-off-by Shameer Kolothum <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
RFC PATCH v2 10/16 vfio/pci: Quiesce INTx while access is blocked
In reply to Shameer KolothumThe INTx handler and unmask virqfd can access PCI_COMMAND without taking access_srcu. Check the recovery state in these paths under irqlock.
Mask pending INTx while access is blocked to prevent an interrupt storm. For shared lines, claim only interrupts identified by pci_check_and_mask_intx(). Record unmask requests for replay when recovery completes.
--- drivers/vfio/pci/vfio_pci_intrs.c | 45 +++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+)
drivers/vfio/pci/vfio_pci_intrs.c +45 -0
@@ -29,6 +29,7 @@ struct vfio_pci_irq_ctx {struct virqfd *mask;char *name;bool masked; +bool unmask_pending;struct irq_bypass_producer producer; };
@@ -49,6 +50,15 @@ static bool is_irq_none(struct vfio_pci_core_device *vdev) vdev->irq_type == VFIO_PCI_MSIX_IRQ_INDEX); } +static bool vfio_pci_recovery_blocks_irq (struct vfio_pci_core_device *vdev) +{ +if (!vdev->pci_recovery_supported) +return false; + +return READ_ONCE (vdev->pci_recovery_enabled) && +READ_ONCE (vdev->access_blocked); +} +static struct vfio_pci_irq_ctx *vfio_irq_ctx_get (struct vfio_pci_core_device *vdev,unsigned long index)
@@ -171,6 +181,12 @@ static int vfio_pci_intx_unmask_handler(void *opaque, void *data)int ret =0 ;spin_lock_irqsave (&vdev->irqlock, flags); +/* Defer unmasking until recovery completes. */ +if (unlikely (vfio_pci_recovery_blocks_irq (vdev))) { +if (is_intx (vdev)) + ctx->unmask_pending = true; + goto out_unlock; + }/* * Unmasking comes from ioctl or config, so again, have the
@@ -182,6 +198,8 @@ static int vfio_pci_intx_unmask_handler(void *opaque, void *data) goto out_unlock; } + ctx->unmask_pending = false; +if (ctx->masked && !vdev->virq_disabled) {/* * A pending interrupt here would immediately trigger,
@@ -220,6 +238,24 @@ void vfio_pci_intx_unmask(struct vfio_pci_core_device *vdev)mutex_unlock (&vdev->igate); } +/* Mask INTx for a blocked device. Returns true if this call masked it. */ +static bool vfio_pci_intx_mask_for_recovery (struct vfio_pci_core_device *vdev, +struct vfio_pci_irq_ctx *ctx) +{ +lockdep_assert_held (&vdev->irqlock); + +if (ctx->masked) +return false; + +if (!vdev->pci_2_3) +disable_irq_nosync (vdev->pdev->irq); +else if (!pci_check_and_mask_intx (vdev->pdev)) +return false; + + ctx->masked = true; +return true; +} +static irqreturn_t vfio_intx_handler (int irq,void *dev_id) {struct vfio_pci_irq_ctx *ctx = dev_id;
@@ -228,6 +264,14 @@ static irqreturn_t vfio_intx_handler(int irq, void *dev_id)int ret =IRQ_NONE ;spin_lock_irqsave (&vdev->irqlock, flags); +if (unlikely (vfio_pci_recovery_blocks_irq (vdev))) { +/* Mask pending INTx to prevent an interrupt storm. */ +if (vfio_pci_intx_mask_for_recovery (vdev, ctx)) + ret =IRQ_HANDLED ; +else if (ctx->masked && !vdev->pci_2_3) + ret =IRQ_HANDLED ; + goto out_unlock; + }if (!vdev->pci_2_3) {disable_irq_nosync (vdev->pdev->irq);
@@ -239,6 +283,7 @@ static irqreturn_t vfio_intx_handler(int irq, void *dev_id) ret =IRQ_HANDLED ; } +out_unlock:spin_unlock_irqrestore (&vdev->irqlock, flags);if (ret ==IRQ_HANDLED )
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [RFC PATCH v2 10/16] vfio/pci: Quiesce INTx while access is blocked
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
The INTx handler and unmask virqfd can access
PCI_COMMANDwithout takingaccess_srcu. Check the recovery state in these paths under irqlock.Mask pending INTx while access is blocked to prevent an interrupt storm. For shared lines, claim only interrupts identified by
pci_check_and_mask_intx(). Record unmask requests for replay when recovery completes.Assisted-by LLMSigned-off-by Shameer Kolothum <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=10
RFC PATCH v2 11/16 vfio/pci: Add INTx recovery start and finish helpers
In reply to Shameer KolothumAdd helpers to mask INTx at recovery entry and restore it on completion. Mask INTx at recovery entry even when no interrupt is pending. Record whether recovery changed the mask so the completion helper can undo it and apply deferred unmask requests.
Preserve the current INTx mask when restoring PCI_COMMAND. Only update INTX_DISABLE for devices supporting that bit; other devices use the IRQ controller mask. Run the completion helper after access is unblocked.
An explicit INTx mask cancels pending recovery unmasking, even if the interrupt is already masked. This preserves a mask request made after access reopens but before the recovery completion helper runs.
--- drivers/vfio/pci/vfio_pci_priv.h | 4 ++ drivers/vfio/pci/vfio_pci_intrs.c | 92 ++++++++++++++++++++++++++++++- 2 files changed, 93 insertions(+), 3 deletions(-)
drivers/vfio/pci/vfio_pci_priv.h +4 -0
@@ -25,6 +25,10 @@ struct vfio_pci_ioeventfd {bool vfio_pci_intx_mask (struct vfio_pci_core_device *vdev);void vfio_pci_intx_unmask (struct vfio_pci_core_device *vdev); +void vfio_pci_intx_recovery_start (struct vfio_pci_core_device *vdev); +void vfio_pci_intx_recovery_finish (struct vfio_pci_core_device *vdev); +u16 vfio_pci_intx_recovery_update_command (struct vfio_pci_core_device *vdev, +u16 command); int vfio_pci_eventfd_replace_locked(struct vfio_pci_core_device *vdev, struct vfio_pci_eventfd __rcu **peventfd,
drivers/vfio/pci/vfio_pci_intrs.c +89 -3
139 lines not on this page · raw message
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [RFC PATCH v2 11/16] vfio/pci: Add INTx recovery start and finish helpers
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 9 lines
Add helpers to mask INTx at recovery entry and restore it on completion. Mask INTx at recovery entry even when no interrupt is pending. Record whether recovery changed the mask so the completion helper can undo it and apply deferred unmask requests.
Preserve the current INTx mask when restoring
PCI_COMMAND. Only updateINTX_DISABLEfor devices supporting that bit; other devices use the IRQ controller mask. Run the completion helper after access is unblocked.
Quoted 6 lines
An explicit INTx mask cancels pending recovery unmasking, even if the interrupt is already masked. This preserves a mask request made after access reopens but before the recovery completion helper runs.
Assisted-by LLMSigned-off-by Shameer Kolothum <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
RFC PATCH v2 12/16 vfio/pci: Restore device state from slot_reset()
In reply to Shameer KolothumA host reset clears device configuration. Add slot_reset() to restore the state saved at open before tearing down interrupts. MSI-X shutdown requires the restored BARs to access its table.
Load vdev->pci_saved_state before entering D0, since the power transition may restore BARs. PCI core's saved copy may have been overwritten by a user reset or D3 entry. Discard the stale pre-reset pm_save and update the power state without restoring it. Keep access blocked and report failure if the snapshot is missing or restoration fails.
Keep active INTx masked in the loaded snapshot so restoration cannot re-enable it before IRQ teardown.
Use pci_set_power_state() as existing recovery callbacks do.
Hold access_lock to exclude close's interrupt teardown. Return NONE on failure to allow recovery of other devices under the bridge to continue.
---
Note: This series adds a VFIO recovery callback that can reacquire pci_bus_sem through the D0 ASPM update while AER already holds it. This can deadlock behind a queued writer. Similar callback patterns in other drivers do not resolve this new VFIO locking issue; topology does not establish lock ownership. See the cover letter's "Locking and open questions" section.
--- drivers/vfio/pci/vfio_pci_core.c | 101 +++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+)
drivers/vfio/pci/vfio_pci_core.c +101 -0
@@ -311,6 +311,15 @@ static void vfio_pci_probe_power_state(struct vfio_pci_core_device *vdev) vdev->needs_pm_restore = !(pmcsr &PCI_PM_CTRL_NO_SOFT_RESET ); } +/* Keep recovery's INTx mask when restoring a loaded PCI state snapshot. */ +static void vfio_pci_recovery_mask_saved_intx (struct vfio_pci_core_device *vdev) +{ +if (READ_ONCE (vdev->access_blocked) && vdev->pci_2_3 && +READ_ONCE (vdev->irq_type) ==VFIO_PCI_INTX_IRQ_INDEX ) + vdev->pdev->saved_config_space[PCI_COMMAND /4 ] |= +PCI_COMMAND_INTX_DISABLE ; +} +/* * pci_set_power_state() wrapper handling devices which perform a soft reset on * D3->D0 transition. Save state prior to D0/1/2->D3, stash it on the vdev,
@@ -2499,6 +2508,18 @@ void vfio_pci_core_unregister_device(struct vfio_pci_core_device *vdev) }EXPORT_SYMBOL_GPL (vfio_pci_core_unregister_device); +static void +vfio_pci_signal_recovery_event (struct vfio_pci_core_device *vdev) +{ +struct vfio_pci_eventfd *eventfd; + +rcu_read_lock (); + eventfd =rcu_dereference (vdev->pci_recovery_trigger); +if (eventfd) +eventfd_signal (eventfd->ctx); +rcu_read_unlock (); +} +pci_ers_result_t vfio_pci_core_aer_err_detected (struct pci_dev *pdev,pci_channel_state_t state) {
@@ -2515,6 +2536,85 @@ pci_ers_result_t vfio_pci_core_aer_err_detected(struct pci_dev *pdev, }EXPORT_SYMBOL_GPL (vfio_pci_core_aer_err_detected); +/* Caller holds memory_lock. Discard the pre-reset PM snapshot. */ +static int vfio_pci_recovery_restore_state (struct vfio_pci_core_device *vdev) +{ +struct pci_dev *pdev = vdev->pdev; +int ret; + +if (!vdev->pci_saved_state) +return -ENODATA ; + ret =pci_load_saved_state (pdev, vdev->pci_saved_state); +if (ret) +return ret; + +kfree (vdev->pm_save); + vdev->pm_save =NULL ; + ret =pci_set_power_state (pdev,PCI_D0 ); +if (ret) +return ret; + +vfio_pci_recovery_mask_saved_intx (vdev); +pci_restore_state (pdev); +return 0 ; +} + +static pci_ers_result_t vfio_pci_core_aer_slot_reset (struct pci_dev *pdev) +{ +struct vfio_pci_core_device *vdev =dev_get_drvdata (&pdev->dev); +int ret =0 ; + +mutex_lock (&vdev->access_lock); +if (!(vdev->pci_recovery_flags &VFIO_PCI_RECOVERY_IN_PROGRESS ) || + !vdev->device_open) { +mutex_unlock (&vdev->access_lock); +return PCI_ERS_RESULT_NONE ; + } + +/* +* Load the open snapshot before D0, which may restore BARs from +* saved state. Restore BARs before IRQ teardown so MSI-X shutdown +* can access the table after the host reset. +*/ +down_write (&vdev->memory_lock); + ret =vfio_pci_recovery_restore_state (vdev); +up_write (&vdev->memory_lock); +if (ret) + goto out_failed; + +/* +* Close clears device_open under access_lock before IRQ teardown, +* excluding this callback from the teardown path. +*/ +mutex_lock (&vdev->igate); +if (vdev->irq_type <VFIO_PCI_NUM_IRQS ) + ret =vfio_pci_set_irqs_ioctl (vdev, +VFIO_IRQ_SET_DATA_NONE | +VFIO_IRQ_SET_ACTION_TRIGGER , + vdev->irq_type,0 ,0 ,NULL ); +mutex_unlock (&vdev->igate); +if (ret) + goto out_failed; + +WRITE_ONCE (vdev->pci_recovery_flags, + vdev->pci_recovery_flags |VFIO_PCI_RECOVERY_RESET ); +mutex_unlock (&vdev->access_lock); + +return PCI_ERS_RESULT_RECOVERED ; + +out_failed: +WRITE_ONCE (vdev->pci_recovery_flags, + (vdev->pci_recovery_flags |VFIO_PCI_RECOVERY_FAILED ) & + ~VFIO_PCI_RECOVERY_IN_PROGRESS ); + vdev->pci_recovery_command_valid = false; +mutex_unlock (&vdev->access_lock); +/* Report failure here; resume() skips completed transactions. */ +vfio_pci_signal_recovery_event (vdev); + +/* Allow recovery of other devices under the bridge to continue. */ +return PCI_ERS_RESULT_NONE ; +} +int vfio_pci_core_sriov_configure (struct vfio_pci_core_device *vdev,int nr_virtfn) {
@@ -2587,6 +2687,7 @@ EXPORT_SYMBOL_GPL(vfio_pci_core_sriov_configure);const struct pci_error_handlers vfio_pci_core_err_handlers = { .error_detected = vfio_pci_core_aer_err_detected, + .slot_reset = vfio_pci_core_aer_slot_reset, };EXPORT_SYMBOL_GPL (vfio_pci_core_err_handlers);
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [RFC PATCH v2 12/16] vfio/pci: Restore device state from slot_reset()
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] pci: rwsem ABBA/starvation deadlock in AER slot reset via pci_set_power_state --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] pci: rwsem ABBA/starvation deadlock in AER slot reset via pci_set_power_state --
commit 6afd5feca9e35f2a1b59b9d3719fa0d9dbdec2ca
Author: Shameer Kolothum <[email protected]>
vfio/pci: Restore device state from slot_reset()
A host reset clears device configuration. Add slot_reset() to restore the state saved at open before tearing down interrupts. MSI-X shutdown requires the restored BARs to access its table.
Quoted 4 lines
drivers/vfio/pci/vfio_pci_core.c +0
-0
[ ... ]
Quoted 5 lines
drivers/vfio/pci/vfio_pci_core.c +5
-0+/* Caller holds memory_lock. Discard the pre-reset PM snapshot. */ +static int vfio_pci_recovery_restore_state (struct vfio_pci_core_device *vdev) +{ +struct pci_dev *pdev = vdev->pdev; +int ret;
drivers/vfio/pci/vfio_pci_core.c pci_ers_result_t vfio_pci_core_aer_err_detected(struct pci_dev *pdev, · line 2553
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c +10
-0+ +if (!vdev->pci_saved_state) +return -ENODATA ; + ret =pci_load_saved_state (pdev, vdev->pci_saved_state); +if (ret) +return ret; + +kfree (vdev->pm_save); + vdev->pm_save =NULL ; + ret =pci_set_power_state (pdev,PCI_D0 );
[Severity: High]
Can this introduce an ABBA starvation deadlock with pci_bus_sem?
During an AER slot reset, pcie_do_recovery() already holds a read lock on pci_bus_sem via pci_walk_bus(). The AER framework then calls vfio_pci_core_aer_slot_reset(), which calls into this function and attempts to restore the device to D0 by calling pci_set_power_state().
Internally, this transitions ASPM via pcie_aspm_pm_state_change(), which unconditionally executes a nested down_read(&pci_bus_sem) because the locked parameter is passed as false:
drivers/pci/pcie/aspm.c:pcie_aspm_pm_state_change() {
...
if (!locked)
down_read(&pci_bus_sem);
...
}If a concurrent thread, such as PCI hotplug or another subsystem, attempts to acquire down_write(&pci_bus_sem) between these two read lock acquisitions, the fair rwsem implementation will queue the writer. The nested down_read() will then block waiting for the writer, creating an unresolvable starvation ABBA deadlock.
As noted in the commit message, the final patches in this series do not appear to address or resolve this bug. Are there plans to address this deadlock before merging?
Quoted 7 lines
drivers/vfio/pci/vfio_pci_core.c +7
-0+if (ret) +return ret; + +vfio_pci_recovery_mask_saved_intx (vdev); +pci_restore_state (pdev); +return 0 ; +}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=12
RFC PATCH v2 13/16 vfio/pci: Complete recovery in resume()
In reply to Shameer KolothumAdd resume() to complete the host transaction and notify userspace. For recovery without reset, apply a deferred D0 request before restoring PCI_COMMAND. After a reset, slot_reset() has already established D0 and restored the open snapshot. Use the existing VFIO power helper to retain pm_save handling. Keep active INTx masked when restoring pm_save while access is blocked, until recovery completes.
After restoring device state, disable ROM decoding unless the ROM resource is marked to remain enabled. Then unblock access, restore DMA-BUF availability and complete INTx unmasking. Protect PCI_COMMAND and the unblock operation with irqlock. A power or command restore failure leaves access blocked and sets FAILED.
Clear the host-active flag even if close or a local failure has already cleared IN_PROGRESS. In that case, skip the per-open completion work.
---
Note: As in slot_reset(), this series introduces a VFIO recovery path that can reacquire pci_bus_sem through the ordinary PCI power helper's ASPM update. This remains unresolved. See the cover letter's "Locking and open questions" section.
--- drivers/vfio/pci/vfio_pci_core.c | 83 ++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+)
drivers/vfio/pci/vfio_pci_core.c +83 -0
@@ -371,6 +371,7 @@ int vfio_pci_set_power_state(struct vfio_pci_core_device *vdev, pci_power_t stat vdev->pm_save =pci_store_saved_state (pdev); }else if (needs_restore) {pci_load_and_free_saved_state (pdev, &vdev->pm_save); +vfio_pci_recovery_mask_saved_intx (vdev);pci_restore_state (pdev); } }
@@ -2615,6 +2616,87 @@ static pci_ers_result_t vfio_pci_core_aer_slot_reset(struct pci_dev *pdev)return PCI_ERS_RESULT_NONE ; } +static void vfio_pci_core_aer_resume (struct pci_dev *pdev) +{ +struct vfio_pci_core_device *vdev =dev_get_drvdata (&pdev->dev); +unsigned long irq_flags; +bool notify_recovery = false; +bool power_up; +u32 flags; +int ret =0 ; + +mutex_lock (&vdev->access_lock); + vdev->pci_recovery_host_active = false; +if (!(vdev->pci_recovery_flags &VFIO_PCI_RECOVERY_IN_PROGRESS )) + goto out_unlock; + + notify_recovery = true; +down_write (&vdev->memory_lock); +/* +* Apply deferred D0 requests before restoring PCI_COMMAND. +* slot_reset() has already established D0 after a reset. +*/ + power_up = vdev->power_up_pending; + vdev->power_up_pending = false; +if (power_up && !(vdev->pci_recovery_flags &VFIO_PCI_RECOVERY_RESET )) { + ret =vfio_pci_set_power_state (vdev,PCI_D0 ); +if (ret) + goto out_memory; + } + +/* +* Serialize with INTx updates to PCI_COMMAND. Keep masked INTx +* disabled until vfio_pci_intx_recovery_finish(). +*/ +spin_lock_irqsave (&vdev->irqlock, irq_flags); +if (!(vdev->pci_recovery_flags &VFIO_PCI_RECOVERY_RESET ) && + vdev->pci_recovery_command_valid) { +u16 cmd = vdev->pci_recovery_command; + + cmd =vfio_pci_intx_recovery_update_command (vdev, cmd); + ret =pci_write_config_word (pdev,PCI_COMMAND , cmd); + } +spin_unlock_irqrestore (&vdev->irqlock, irq_flags); +if (ret) + goto out_memory; + +/* +* A saved-state restore may leave ROM decode enabled. Disable it +* before allowing access to shared decoders. +*/ +if (pci_resource_start (pdev,PCI_ROM_RESOURCE ) && + !(pdev->resource[PCI_ROM_RESOURCE ].flags &IORESOURCE_ROM_ENABLE )) +pci_disable_rom (pdev); + +/* +* Finish restoring hardware state before allowing accesses. +* Serialize the flag update with the INTx handler. +*/ +spin_lock_irqsave (&vdev->irqlock, irq_flags); +WRITE_ONCE (vdev->access_blocked, false); +spin_unlock_irqrestore (&vdev->irqlock, irq_flags); + +if (__vfio_pci_memory_enabled (vdev)) +vfio_pci_dma_buf_move (vdev, false); + +out_memory: +up_write (&vdev->memory_lock); + vdev->pci_recovery_command_valid = false; + flags = vdev->pci_recovery_flags & ~VFIO_PCI_RECOVERY_IN_PROGRESS ; +if (ret) { +WRITE_ONCE (vdev->pci_recovery_flags, + flags |VFIO_PCI_RECOVERY_FAILED ); + goto out_unlock; + } +WRITE_ONCE (vdev->pci_recovery_flags, flags); +vfio_pci_intx_recovery_finish (vdev); + +out_unlock: +mutex_unlock (&vdev->access_lock); +if (notify_recovery) +vfio_pci_signal_recovery_event (vdev); +} +int vfio_pci_core_sriov_configure (struct vfio_pci_core_device *vdev,int nr_virtfn) {
@@ -2688,6 +2770,7 @@ EXPORT_SYMBOL_GPL(vfio_pci_core_sriov_configure);const struct pci_error_handlers vfio_pci_core_err_handlers = { .error_detected = vfio_pci_core_aer_err_detected, .slot_reset = vfio_pci_core_aer_slot_reset, + .resume = vfio_pci_core_aer_resume, };EXPORT_SYMBOL_GPL (vfio_pci_core_err_handlers);
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [RFC PATCH v2 13/16] vfio/pci: Complete recovery in resume()
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] PCI: deadlock in AER resume due to recursive pci_bus_sem read lock --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] PCI: deadlock in AER resume due to recursive pci_bus_sem read lock --
commit ac6b588d8a9a2d9a21c17325b45933f079efe247
Author: Shameer Kolothum <[email protected]>
vfio/pci: Complete recovery in resume()
This commit adds the resume() callback to complete host transactions and notify userspace during VFIO PCI recovery. It restores device state, handles PCI power state transitions, and finishes restoring hardware state before allowing accesses.
Quoted 4 lines
drivers/vfio/pci/vfio_pci_core.c +0
-0
[ ... ]
Quoted 7 lines
drivers/vfio/pci/vfio_pci_core.c +3
-0@@ -2615,6 +2616,87 @@ static pci_ers_result_t vfio_pci_core_aer_slot_reset(struct pci_dev *pdev)return PCI_ERS_RESULT_NONE ; } +static void vfio_pci_core_aer_resume (struct pci_dev *pdev) +{ +struct vfio_pci_core_device *vdev =dev_get_drvdata (&pdev->dev);
[ ... ]
drivers/vfio/pci/vfio_pci_core.c static pci_ers_result_t vfio_pci_core_aer_slot_reset(struct pci_dev *pdev) · line 2642
Quoted 8 lines
drivers/vfio/pci/vfio_pci_core.c +8
-0+/* +* Apply deferred D0 requests before restoring PCI_COMMAND. +* slot_reset() has already established D0 after a reset. +*/ + power_up = vdev->power_up_pending; + vdev->power_up_pending = false; +if (power_up && !(vdev->pci_recovery_flags &VFIO_PCI_RECOVERY_RESET )) { + ret =vfio_pci_set_power_state (vdev,PCI_D0 );
[Severity: High]
Can this lead to a deadlock of the PCI subsystem?
An AER recovery event triggers the AER core, which acquires pci_bus_sem for read via pci_walk_bridge().
The AER resume callback vfio_pci_core_aer_resume() then calls vfio_pci_set_power_state(), which internally attempts to recursively acquire pci_bus_sem for read via pcie_aspm_pm_state_change().
If a writer, such as a PCI hotplug event, requests pci_bus_sem between these two read lock attempts, the second read attempt will deadlock.
While this is noted as an unresolved issue in the series cover letter, is there a plan to mitigate this recursive lock before merging?
Quoted 3 lines
drivers/vfio/pci/vfio_pci_core.c +3
-0+if (ret) + goto out_memory; + }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=13
RFC PATCH v2 14/16 vfio/pci: Block device access during host recovery
In reply to Shameer KolothumFor userspace that enables recovery, extend error_detected() to block device access and drain existing SRCU readers. Mask INTx, revoke BAR mappings and DMA-BUFs, and disable bus mastering on a normal channel.
Return CAN_RECOVER for a normal channel, NEED_RESET for a frozen channel and DISCONNECT for permanent failure. If saving PCI_COMMAND or disabling bus mastering fails, return NONE and leave access blocked. Preserve the original command and sequence number across nested events.
Track host recovery across close and reopen, including without userspace opt-in. Reject open during a recorded transaction or after disconnection. Clear deferred power requests at close and keep DMA-BUFs revoked while access is blocked.
Reject hot reset for devices in ongoing recovery, but allow failed devices so healthy siblings can be reset.
Reject userspace SR-IOV configuration while access is blocked.
---
Note: The existing open/close paths are not serialized against the full host recovery transaction. This series blocks reopening during an already recorded transaction, but recovery can still start after the open check, and close can overlap the physical reset between callbacks. See the cover letter's "Locking and open questions" section.
--- drivers/vfio/pci/vfio_pci.c | 3 + drivers/vfio/pci/vfio_pci_core.c | 166 ++++++++++++++++++++++++++++- drivers/vfio/pci/vfio_pci_dmabuf.c | 5 + 3 files changed, 173 insertions(+), 1 deletion(-)
drivers/vfio/pci/vfio_pci.c +3 -0
@@ -213,6 +213,9 @@ static int vfio_pci_sriov_configure(struct pci_dev *pdev, int nr_virtfn)if (!enable_sriov)return -ENOENT ; +if (vdev->pci_recovery_supported &&READ_ONCE (vdev->access_blocked)) +return -EIO ; +return vfio_pci_core_sriov_configure (vdev, nr_virtfn); }
drivers/vfio/pci/vfio_pci_core.c +165 -1
208 lines not on this page · raw message
drivers/vfio/pci/vfio_pci_dmabuf.c +5 -0
@@ -350,6 +350,11 @@ void vfio_pci_dma_buf_move(struct vfio_pci_core_device *vdev, bool revoked)lockdep_assert_held_write (&vdev->memory_lock); +/* Reset and power-state cleanup must not undo recovery revocation. */ +if (!revoked && vdev->pci_recovery_supported && +READ_ONCE (vdev->access_blocked)) +return ; +list_for_each_entry_safe (priv, tmp, &vdev->dmabufs, dmabufs_elm) {if (!get_file_active (&priv->dmabuf->file))continue ;
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [RFC PATCH v2 14/16] vfio/pci: Block device access during host recovery
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] cleanup: vfio_pci_core_enable() mixes goto and scoped_guard() --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] cleanup: vfio_pci_core_enable() mixes goto and scoped_guard()
--
commit 054f5c9b3c995b5b332595d9db5677b803d39ed9
Author: Shameer Kolothum <[email protected]>
vfio/pci: Block device access during host recovery
This patch extends error_detected() to block device access during host recovery by draining existing SRCU readers. It also masks INTx, revokes BAR mappings, and manages the host transaction state across device open and close.
drivers/vfio/pci/vfio_pci_core.c int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) · line 623
Quoted 12 lines
drivers/vfio/pci/vfio_pci_core.c +4
-0@@ -617,6 +617,15 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev)u16 cmd;u8 msix_pos; +if (vdev->pci_recovery_supported) { +if (pci_dev_is_disconnected (pdev)) +return -ENODEV ; + scoped_guard(mutex, &vdev->access_lock) {
[Severity: Low]
Does this mix scope-based cleanup with goto-based error handling in the same function?
The cleanup subsystem guidelines in include/linux/cleanup.h state that goto and cleanup helpers like scoped_guard() should never be mixed in the same function to prevent ownership confusion. For a given routine, the expectation is to convert all resources that need a goto cleanup to scope-based cleanup, or convert none of them.
Looking at vfio_pci_core_enable(), the function still relies heavily on legacy goto cleanup labels (like out_disable_device: and out_power:) on its error paths.
Should this function be fully converted to scope-based cleanup first, or should this new lock acquisition continue using the traditional lock and unlock pattern to match the rest of the function?
Quoted 8 lines
drivers/vfio/pci/vfio_pci_core.c +5
-0+if (vdev->pci_recovery_host_active) +return -EBUSY ; + } + } +if (!vdev->disable_idle_d3) { ret =pm_runtime_resume_and_get (&pdev->dev);if (ret <0 )
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=14
RFC PATCH v2 15/16 vfio/pci: Add VFIO_DEVICE_FEATURE_PCI_ERROR_RECOVERY
In reply to Shameer KolothumAdd a device feature to enable host PCI error recovery and query its status. SET registers an eventfd for start and completion notifications; eventfd -1 disables recovery. GET returns the status flags and sequence number, including while recovery is active.
Reject SET while access is blocked or recovery has failed. Reset the per-open status and sequence when updating the eventfd. Existing VFIO_PCI_ERR_IRQ_INDEX notifications are unchanged.
--- include/uapi/linux/vfio.h | 44 +++++++++++++ drivers/vfio/pci/vfio_pci_core.c | 108 +++++++++++++++++++++++++++++++ 2 files changed, 152 insertions(+)
include/uapi/linux/vfio.h +44 -0
@@ -1555,6 +1555,50 @@ struct vfio_device_feature_zpci_err {#define VFIO_DEVICE_FEATURE_ZPCI_ERROR 13 +/* +* Report host PCI error recovery state for this device. +* +* VFIO_DEVICE_FEATURE_SET with a valid eventfd enables recovery for this +* open and registers notifications for the start and end of each event. +* SET with eventfd -1 disables recovery. flags and sequence must be zero. +* SET returns -EBUSY during recovery, after recovery failure, or while +* device access is blocked, and -ENODEV after close has started. +* VFIO_PCI_ERR_IRQ_INDEX notifications are unchanged. +* +* VFIO_DEVICE_FEATURE_GET returns the current state and eventfd = -1. +* GET is allowed during recovery, but may wait for a running callback. +* +* The sequence number increments for each event and resets to zero when +* recovery is enabled. Use it to detect coalesced notifications. Device +* accesses return -EIO while IN_PROGRESS is set. Recovery may complete +* before userspace reads the notification, so userspace must check the +* sequence and status rather than wait to observe IN_PROGRESS. +* +* CHANNEL_FROZEN records a frozen channel. DEVICE_RESET records a host +* reset; userspace must reconfigure interrupts with VFIO_DEVICE_SET_IRQS. +* FAILED indicates that recovery failed and access remains blocked until +* close and reopen. Status bits persist until the next event or SET. +* ENABLED indicates that recovery is enabled. +* +* Enabling recovery during an existing event does not provide status or +* notifications for that event. +* +* Open returns -EBUSY while a previously recorded host recovery is active, +* including events reported while recovery was disabled. +*/ +struct vfio_device_pci_error_recovery { +__u32 flags; +#define VFIO_PCI_ERROR_RECOVERY_IN_PROGRESS (1U << 0) +#define VFIO_PCI_ERROR_RECOVERY_CHANNEL_FROZEN (1U << 1) +#define VFIO_PCI_ERROR_RECOVERY_DEVICE_RESET (1U << 2) +#define VFIO_PCI_ERROR_RECOVERY_FAILED (1U << 3) +#define VFIO_PCI_ERROR_RECOVERY_ENABLED (1U << 4) +__s32 eventfd; +__aligned_u64 sequence; +}; + +#define VFIO_DEVICE_FEATURE_PCI_ERROR_RECOVERY 14 +/* -------- API for Type1 VFIO IOMMU -------- */ /**
drivers/vfio/pci/vfio_pci_core.c +108 -0
120 lines not on this page · raw message
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [RFC PATCH v2 15/16] vfio/pci: Add VFIO_DEVICE_FEATURE_PCI_ERROR_RECOVERY
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] vfio/pci: mixing goto-based cleanup and guard() in vfio_pci_recovery_set() --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] vfio/pci: mixing goto-based cleanup and guard() in vfio_pci_recovery_set() --
commit a042b1f42248620e0a4adaa43aee23e3f4d393dc
Author: Shameer Kolothum <[email protected]>
vfio/pci: Add VFIO_DEVICE_FEATURE_PCI_ERROR_RECOVERY
This commit adds a device feature to enable host PCI error recovery and query its status via VFIO.
Quoted 4 lines
drivers/vfio/pci/vfio_pci_core.c +0
-0
[ ... ]
Quoted 5 lines
drivers/vfio/pci/vfio_pci_core.c +1
-0@@ -1752,6 +1752,111 @@ static int vfio_pci_core_feature_token(struct vfio_pci_core_device *vdev,return 0 ; } +/* Consumes the eventfd reference on both success and failure. */
drivers/vfio/pci/vfio_pci_core.c static int vfio_pci_core_feature_token(struct vfio_pci_core_device *vdev, · line 1765
Quoted 10 lines
drivers/vfio/pci/vfio_pci_core.c +10
-0+static int vfio_pci_recovery_set (struct vfio_pci_core_device *vdev, +struct eventfd_ctx *ctx,bool enable) +{ +int ret; + +guard (mutex)(&vdev->access_lock); +if (!vdev->device_open) { + ret = -ENODEV ; + goto out_put; + }
[Severity: Medium]
Does this code mix goto-based error handling with scope-based guard() cleanup?
Subsystem guidelines expect that a given routine should either convert all resources that need cleanup to scope-based cleanup, or convert none of them. Mixing the two in the same function creates confusing ownership semantics that can lead to bugs during future modifications.
[ ... ]
Quoted 6 lines
drivers/vfio/pci/vfio_pci_core.c +6
-0+ +out_put: +if (ctx) +eventfd_ctx_put (ctx); +return ret; +}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=15
RFC PATCH v2 16/16 vfio/pci: Enable host PCI error recovery for vfio-pci
In reply to Shameer KolothumEnable recovery support for generic vfio-pci. Userspace opts in through VFIO_DEVICE_FEATURE_PCI_ERROR_RECOVERY. Open rejects an active host recovery even without opt-in. Leave variant drivers disabled until their access paths are covered.
Exclude CXL memory devices and s390: Restricted CXL Host (RCH) and s390 mediated recovery can omit completion callbacks. On s390, permanent-failure notification also blocks config access, so draining SRCU can deadlock with config readers.
--- drivers/vfio/pci/vfio_pci.c | 9 +++++++++ 1 file changed, 9 insertions(+)
drivers/vfio/pci/vfio_pci.c +9 -0
@@ -129,6 +129,7 @@ static int vfio_pci_init_dev(struct vfio_device *core_vdev) {struct vfio_pci_core_device *vdev = container_of(core_vdev, struct vfio_pci_core_device, vdev); +struct pci_dev *pdev =to_pci_dev (core_vdev->dev);/* * These behaviors originated in vfio-pci and moved into
@@ -139,6 +140,14 @@ static int vfio_pci_init_dev(struct vfio_device *core_vdev)*/ vdev->nointxmask = nointxmask; vdev->disable_idle_d3 = disable_idle_d3; +/* +* CXL RCH and s390 mediated recovery can omit completion callbacks. +* s390 also blocks config access across failure notification, which +* prevents draining SRCU readers waiting for config access. +*/ +if (!IS_ENABLED (CONFIG_S390 ) && + (pdev->class >>8 ) !=PCI_CLASS_MEMORY_CXL ) + vdev->pci_recovery_supported = true;#ifdef CONFIG_VFIO_PCI_VGA vdev->disable_vga = disable_vga;#endif
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [RFC PATCH v2 16/16] vfio/pci: Enable host PCI error recovery for vfio-pci
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 5 lines
Enable recovery support for generic vfio-pci. Userspace opts in through
VFIO_DEVICE_FEATURE_PCI_ERROR_RECOVERY. Open rejects an active host recovery even without opt-in. Leave variant drivers disabled until their access paths are covered.
Quoted 7 lines
Exclude CXL memory devices and s390: Restricted CXL Host (RCH) and s390 mediated recovery can omit completion callbacks. On s390, permanent-failure notification also blocks config access, so draining SRCU can deadlock with config readers.
Assisted-by LLMSigned-off-by Shameer Kolothum <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=16
Alex Williamson
maintainerQuoted 88 lines · On Tue, 29 Sep 2026 18:32:49 +0100 Shameer Kolothum <[email protected]>…
Hi,
Currently, vfio-pci signals the error eventfd from
error_detected()and returnsPCI_ERS_RESULT_CAN_RECOVER, including for permanent failure. It has noslot_reset()orresume()callback, so userspace is not notified when host recovery completes or whether the device was reset.This series adds opt-in host PCI error recovery for vfio-pci. It blocks device access during recovery, restores device state after a host reset, and reports recovery status to userspace. The kernel runs host recovery and the VMM decides how to recover the guest.
Changes from v1
---------------
v1: https://lore.kernel.org/kvm/[email protected]/- Thanks to Satya and Alex for reviews/feedback.
- Use SRCU to drain ongoing device accesses and reject new accesses during recovery. A mutex protects recovery-state updates.
- Return SIGBUS for blocked BAR faults instead of waiting in the fault handler. VMM handling still needs target validation.
- Reject userspace SR-IOV changes while access is blocked.
- Exclude s390 and all CXL memory devices for now as s390 and CXL RCH recovery can omit completion callbacks.Design
------
Userspace opts in throughVFIO_DEVICE_FEATURE_PCI_ERROR_RECOVERY. It registers an eventfd and reads status flags and a sequence number. The existing error eventfd is still signaled.The callbacks handle the host recovery sequence:
-
error_detected()blocks new accesses and drains admitted ones, quiesces INTx, and revokes BAR mappings and exported DMA-BUFs. On a normal channel it also clears bus mastering. It votesCAN_RECOVERfor a normal channel,NEED_RESETfor a frozen channel, and DISCONNECT for permanent failure. Failure to savePCI_COMMANDor disable bus mastering leaves access blocked and returns NONE so other devices can continue recovery.
-slot_reset()restores the configuration saved at open after a host reset, then tears down interrupt configuration. Userspace must set up interrupts again.
-resume()completes restoration, unblocks access and completes INTx handling. A restoration failure keeps the device blocked.Non-fatal errors also block device access because host recovery may require a reset. Userspace reads the recovery status to determine whether the device was reset or recovery failed.
Without opt-in, the existing error notification behavior is retained. The open check also rejects a previously recorded host transaction when recovery is disabled, so closing the device does not discard that state.
Locking and open questions
--------------------------
Several locks are involved in recovery. PCI core holdsdevice_lockwhile calling the driver, preventing concurrent driver binding or unbinding. During the bus walk, it also holdspci_bus_semfor reading to keep the device list stable. Adding or removing PCI devices requires the write lock. This semaphore is shared across all PCI buses.VFIO recovery blocks new accesses, waits for existing SRCU readers to finish, and takes
memory_lock. Power transitions also takememory_lock, but entering D0 can then acquirepci_bus_semto update PCIe link power management (ASPM).Local Sashiko/Claude review reports a lock-order conflict as below:
1. A guest D0 request holds
memory_lock.
2. PCI core holdspci_bus_semfor reading while VFIO's recovery callback waits formemory_lock.
3. Removal of an unrelated PCI device waits for the bus write lock.
4. D0's ASPM update requests the bus read lock and waits behind that writer.Each task waits for another, so none can proceed.
One possible solution could be(not implemented):
- PCI core guarantees that
pci_bus_semis held for reading before invoking supported recovery callbacks. VFIO usespci_set_power_state_locked()in those callbacks, avoiding another acquisition of the bus lock.
- A newpci_try_set_power_state()helper tries to acquirepci_bus_semand returns-EAGAINon contention. Ordinary VFIO D0 paths use this helper and handle failure instead of waiting while holding VFIO locks
Quoted 2 lines
Not sure there are better ways to handle this or not. Feedbacks appreciated on this.
I've been churning tokens with an LLM on this. My original intent was that a disconnect via srcu between a device object and user interface object, or at least ops pointers that act on the device, at the vfio-core level could help to avoid this. While that may be worthwhile for more general use cases of removing user access to the device (surprise unplug, force unbind, etc.), it doesn't seem to resolve the scenario here. In fact I haven't come up with better solutions than you've already suggested above.
Quoted 6 lines
Another issue flagged was(I think this is a pre-existing one):
- Open/close coordination with the whole host recovery transaction was already missing. Recovery can start after the open check, and close can overlap the physical reset. Restoration of closed devices is also not implemented.
Yes, it's pre-existing. A more structured concept of a disconnected device would help make these gaps easier to close as well. Thanks,
Alex
Quoted 11 lines · Shameer Kolothum
Hi,
Currently, vfio-pci signals the error eventfd from
error_detected()and returnsPCI_ERS_RESULT_CAN_RECOVER, including for permanent failure. It has noslot_reset()orresume()callback, so userspace is not notified when host recovery completes or whether the device was reset.This series adds opt-in host PCI error recovery for vfio-pci. It blocks device access during recovery, restores device state after a host reset, and reports recovery status to userspace. The kernel runs host recovery and the VMM decides how to recover the guest.
We've been experiementating with this recently. It turns out that doing a meaningful recovery on the user space side is a bit more difficult than initially thought, but we can at least get more deterministic behavior with these notifications and gracefully stop affected processes. I don't think there is much else the kernel side can do to make this go any better, so this series looks good to us.
Quoted 31 lines
-----Original Message-----
From: Keith Busch <[email protected]>
Sent: 06 November 2026 16:57
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Sonang Patel <[email protected]>; Ankit Agrawal <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>
Subject: Re: [RFC PATCH v2 00/16] vfio/pci: Handle PCI error recovery and report state to userspaceExternal email: Use caution opening links or attachments
Quoted 11 lines · Shameer Kolothum
Hi,
Currently, vfio-pci signals the error eventfd from
error_detected()and returnsPCI_ERS_RESULT_CAN_RECOVER, including for permanent failure. It has noslot_reset()orresume()callback, so userspace is not notified when host recovery completes or whether the device was reset.This series adds opt-in host PCI error recovery for vfio-pci. It blocks device access during recovery, restores device state after a host reset, and reports recovery status to userspace. The kernel runs host recovery and the VMM decides how to recover the guest.
Quoted 8 lines
We've been experiementating with this recently. It turns out that doing a meaningful recovery on the user space side is a bit more difficult than initially thought, but we can at least get more deterministic behavior with these notifications and gracefully stop affected processes. I don't think there is much else the kernel side can do to make this go any better, so this series looks good to us.
Acked-by Keith Busch <[email protected]>
Hi Keith,
Thanks for that. I have posted an RFC v3 here [0]. The overall design and UAPI remain the same, but access serialization and power-state locking have changed enough that I haven't carried your Acked-by forward.
Please take a look at v3 and let me know whether your ack still applies.
Thanks,
Shameer