vfio/pds: Fix and simplify resets
Subsystems (5)
Brett Creeley
maintainerv2 cover vfio/pds: Fix and simplify resets
This small series contains a fix and readability improvements for resets.
v2:
- Split single patch into 2 patches
- Improve commit messages
v1:
https://lore.kernel.org/kvm/[email protected]/
Brett Creeley (2):
vfio/pds: Always clear the save/restore FDs on reset vfio/pds: Refactor/simplify reset logic
drivers/vfio/pci/pds/pci_drv.c | 2 +- drivers/vfio/pci/pds/vfio_dev.c | 14 +++++++------- drivers/vfio/pci/pds/vfio_dev.h | 7 ++++++- 3 files changed, 14 insertions(+), 9 deletions(-)
Signature · 1 line
2.17.1
Brett Creeley
maintainerPATCH v2 1/2 vfio/pds: Always clear the save/restore FDs on reset
After reset the VFIO device state will always be put in VFIO_DEVICE_STATE_RUNNING, but the save/restore files will only be cleared if the previous state was VFIO_DEVICE_STATE_ERROR. This can/will cause the restore/save files to be leaked if/when the migration state machine transitions through the states that re-allocates these files. Fix this by always clearing the restore/save files for resets.
--- drivers/vfio/pci/pds/vfio_dev.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/vfio/pci/pds/vfio_dev.c +2 -2
@@ -32,9 +32,9 @@ void pds_vfio_state_mutex_unlock(struct pds_vfio_pci_device *pds_vfio)mutex_lock (&pds_vfio->reset_mutex);if (pds_vfio->deferred_reset) { pds_vfio->deferred_reset = false; +pds_vfio_put_restore_file (pds_vfio); +pds_vfio_put_save_file (pds_vfio);if (pds_vfio->state ==VFIO_DEVICE_STATE_ERROR ) { -pds_vfio_put_restore_file (pds_vfio); -pds_vfio_put_save_file (pds_vfio);pds_vfio_dirty_disable (pds_vfio, false); } pds_vfio->state = pds_vfio->deferred_reset_state;
Signature · 1 line
2.17.1
RE: [PATCH v2 vfio 1/2] vfio/pds: Always clear the save/restore FDs on reset
Quoted 11 lines
From: Brett Creeley <[email protected]>
Sent: Wednesday, February 28, 2024 8:32 AMAfter reset the VFIO device state will always be put in
VFIO_DEVICE_STATE_RUNNING, but the save/restore files will only be cleared if the previous state wasVFIO_DEVICE_STATE_ERROR. This can/will cause the restore/save files to be leaked if/when the migration state machine transitions through the states that re-allocates these files. Fix this by always clearing the restore/save files for resets.
Quoted 4 lines
Fixes 7dabb1bcd177 ("vfio/pds: Add support for firmware recovery")Signed-off-by Brett Creeley <[email protected]>Reviewed-by Shannon Nelson <[email protected]>
Brett Creeley
maintainerPATCH v2 2/2 vfio/pds: Refactor/simplify reset logic
In reply to Brett CreeleyThe current logic for handling resets is more complicated than it needs to be. The deferred_reset flag is used to indicate a reset is needed and the deferred_reset_state is the requested, post-reset, state. The source of the requested reset isn't immediately obvious. Improve readability by replacing deferred_reset_state with deferred_reset_type, which can be either PDS_VFIO_DEVICE_RESET (initiated/requested by the DSC) or PDS_VFIO_HOST_RESET (initiated/requested by the VMM).
--- drivers/vfio/pci/pds/pci_drv.c | 2 +- drivers/vfio/pci/pds/vfio_dev.c | 10 +++++----- drivers/vfio/pci/pds/vfio_dev.h | 7 ++++++- 3 files changed, 12 insertions(+), 7 deletions(-)
drivers/vfio/pci/pds/pci_drv.c +1 -1
@@ -57,7 +57,7 @@ static void pds_vfio_recovery(struct pds_vfio_pci_device *pds_vfio)if (deferred_reset_needed) {mutex_lock (&pds_vfio->reset_mutex); pds_vfio->deferred_reset = true; - pds_vfio->deferred_reset_state =VFIO_DEVICE_STATE_ERROR ; + pds_vfio->deferred_reset_type =PDS_VFIO_DEVICE_RESET ;mutex_unlock (&pds_vfio->reset_mutex); } }
drivers/vfio/pci/pds/vfio_dev.c +5 -5
@@ -34,11 +34,12 @@ void pds_vfio_state_mutex_unlock(struct pds_vfio_pci_device *pds_vfio) pds_vfio->deferred_reset = false;pds_vfio_put_restore_file (pds_vfio);pds_vfio_put_save_file (pds_vfio); -if (pds_vfio->state ==VFIO_DEVICE_STATE_ERROR ) { +if (pds_vfio->deferred_reset_type ==PDS_VFIO_DEVICE_RESET ) {pds_vfio_dirty_disable (pds_vfio, false); + pds_vfio->state =VFIO_DEVICE_STATE_ERROR ; + }else { + pds_vfio->state =VFIO_DEVICE_STATE_RUNNING ; } - pds_vfio->state = pds_vfio->deferred_reset_state; - pds_vfio->deferred_reset_state =VFIO_DEVICE_STATE_RUNNING ;mutex_unlock (&pds_vfio->reset_mutex); goto again; }
@@ -50,7 +51,7 @@ void pds_vfio_reset(struct pds_vfio_pci_device *pds_vfio) {mutex_lock (&pds_vfio->reset_mutex); pds_vfio->deferred_reset = true; - pds_vfio->deferred_reset_state =VFIO_DEVICE_STATE_RUNNING ; + pds_vfio->deferred_reset_type =PDS_VFIO_HOST_RESET ;if (!mutex_trylock (&pds_vfio->state_mutex)) {mutex_unlock (&pds_vfio->reset_mutex);return ;
@@ -194,7 +195,6 @@ static int pds_vfio_open_device(struct vfio_device *vdev)return err; pds_vfio->state =VFIO_DEVICE_STATE_RUNNING ; - pds_vfio->deferred_reset_state =VFIO_DEVICE_STATE_RUNNING ;vfio_pci_core_finish_enable (&pds_vfio->vfio_coredev);
drivers/vfio/pci/pds/vfio_dev.h +6 -1
@@ -10,6 +10,11 @@#include "dirty.h" #include "lm.h" +enum pds_vfio_reset_type { +PDS_VFIO_HOST_RESET =0 , +PDS_VFIO_DEVICE_RESET =1 , +}; +struct pds_vfio_pci_device {struct vfio_pci_core_device vfio_coredev;
@@ -20,7 +25,7 @@ struct pds_vfio_pci_device {enum vfio_device_mig_state state;struct mutex reset_mutex;/* protect reset_done flow */ u8 deferred_reset; -enum vfio_device_mig_state deferred_reset_state; +enum pds_vfio_reset_type deferred_reset_type;struct notifier_block nb;int vf_id;
Signature · 1 line
2.17.1
RE: [PATCH v2 vfio 2/2] vfio/pds: Refactor/simplify reset logic
Quoted 11 lines
From: Brett Creeley <[email protected]>
Sent: Wednesday, February 28, 2024 8:32 AMThe current logic for handling resets is more complicated than it needs to be. The
deferred_resetflag is used to indicate a reset is needed and thedeferred_reset_stateis the requested, post-reset, state. The source of the requested reset isn't immediately obvious. Improve readability by replacingdeferred_reset_statewithdeferred_reset_type, which can be eitherPDS_VFIO_DEVICE_RESET(initiated/requested by the DSC) orPDS_VFIO_HOST_RESET(initiated/requested by the VMM).
Quoted 2 lines
Signed-off-by Brett Creeley <[email protected]>Reviewed-by Shannon Nelson <[email protected]>
Quoted 12 lines
-----Original Message-----
From: Brett Creeley <[email protected]>
Sent: Wednesday, February 28, 2024 12:32 AM
To: [email protected]; [email protected]; Shameerali Kolothum Thodi <[email protected]>; [email protected]; [email protected]; [email protected]; linux- [email protected]
Cc: [email protected]; [email protected]
Subject: [PATCH v2 vfio 0/2] vfio/pds: Fix and simplify resets
Quoted 6 lines
This small series contains a fix and readability improvements for resets.
v2:
- Split single patch into 2 patches
- Improve commit messages
Just a query on the reset_done handler and the deferred_reset() logic in this driver. From a quick look, it doesn't look like you have a condition where a copy_to/from_user() is under state_mutex. So do you think we can get rid of the deferred_reset logic from this driver? Please see the discussion here,
https://lore.kernel.org/kvm/[email protected]/
For HiSilicon, we do have the lock taken for PRE_COPY, but that needs fixing and then can get rid of the deferred_reset. I will sent out a patch for that soon.
Thanks,
Shameer
Quoted 16 lines
v1:
https://lore.kernel.org/kvm/20240126183225.19193-1-
[email protected]/Brett Creeley (2):
vfio/pds: Always clear the save/restore FDs on reset vfio/pds: Refactor/simplify reset logicdrivers/vfio/pci/pds/pci_drv.c | 2 +- drivers/vfio/pci/pds/vfio_dev.c | 14 +++++++------- drivers/vfio/pci/pds/vfio_dev.h | 7 ++++++- 3 files changed, 14 insertions(+), 9 deletions(-)--
2.17.1
Quoted 27 lines · Shameerali Kolothum Thodi
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Quoted 10 lines
-----Original Message-----
From: Brett Creeley <[email protected]>
Sent: Wednesday, February 28, 2024 12:32 AM
To: [email protected]; [email protected]; Shameerali Kolothum Thodi <[email protected]>; [email protected]; [email protected]; [email protected]; linux- [email protected]
Cc: [email protected]; [email protected]
Subject: [PATCH v2 vfio 0/2] vfio/pds: Fix and simplify resetsQuoted 6 lines
This small series contains a fix and readability improvements for resets.
v2:
- Split single patch into 2 patches
- Improve commit messagesJust a query on the
reset_donehandler and thedeferred_reset()logic in this driver. From a quick look, it doesn't look like you have a condition where a copy_to/from_user() is understate_mutex. So do you think we can get rid of thedeferred_resetlogic from this driver? Please see the discussion here,
https://lore.kernel.org/kvm/[email protected]/
Quoted 6 lines
For HiSilicon, we do have the lock taken for
PRE_COPY, but that needs fixing and then can get rid of thedeferred_reset. I will sent out a patch for that soon.Thanks,
Shameer
Hi Shameer,
You are probably right that we can get rid of this logic, but the current 2 patch series is very simple and I would prefer to keep it that way. If you plan to make changes to the HiSilicon driver in the near future, then I can use that as a reference in enhancing the pds-vfio-pci driver.
Thanks,
Brett
Quoted 18 lines
Quoted 17 lines
v1:
https://lore.kernel.org/kvm/20240126183225.19193-1-
[email protected]/Brett Creeley (2):
vfio/pds: Always clear the save/restore FDs on reset vfio/pds: Refactor/simplify reset logicdrivers/vfio/pci/pds/pci_drv.c | 2 +- drivers/vfio/pci/pds/vfio_dev.c | 14 +++++++------- drivers/vfio/pci/pds/vfio_dev.h | 7 ++++++- 3 files changed, 14 insertions(+), 9 deletions(-)--
2.17.1
Alex Williamson
AppliedQuoted 10 lines · On Tue, 27 Feb 2024 16:32:03 -0800 Brett Creeley <[email protected]> wrote:
This small series contains a fix and readability improvements for resets.
v2:
- Split single patch into 2 patches
- Improve commit messages
Quoted 9 lines
Brett Creeley (2):
vfio/pds: Always clear the save/restore FDs on reset vfio/pds: Refactor/simplify reset logicdrivers/vfio/pci/pds/pci_drv.c | 2 +- drivers/vfio/pci/pds/vfio_dev.c | 14 +++++++------- drivers/vfio/pci/pds/vfio_dev.h | 7 ++++++- 3 files changed, 14 insertions(+), 9 deletions(-)
Applied to vfio next branch for v6.9. Thanks,
Alex