Skip to contentSkip to conversation

Add new VFIO PCI driver for NVMe devices

RFC PATCH In review Chaitanya Kulkarni 5 participants 11 messages Aug 3, 2025 – Aug 4, 2025

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover Add new VFIO PCI driver for NVMe devices

Hi,

Some devices, such as Infrastructure Processing Units (IPUs), Data Processing Units (DPUs), and SSDs expose SR-IOV-capable NVMe devices to the host. These virtual function (VF) devices support live migration via specific NVMe admin commands issued through the parent PF's admin queue.

NVMe TP4159 defines support for basic live migration operations, including Suspend, Resume, Get Controller State, and Set Controller State. While TP4159 standardizes the command interface, it does not yet define a fixed layout for controller state, NVIDIA and others in NVMe TWG is actively working on defining this layout.

This series introduces a vfio-pci driver to enable live migration of SR-IOV NVMe devices. It also adds interface hooks to the core NVMe driver to allow VF command submission through the PF's admin queue. Adding support for migration of non SR-IOV devices can be added incrementally.

This RFC complies with the TP4159 specification and is derived from initial submission of Intel and NVIDIA’s vendor-specific implementation.

Objective for this RFC
----------------------

Our initial submission received feedback encouraging standardization of live migration support for NVMe. In response, NVIDIA and Intel collaborated to merge architectural elements from TP4173 into TP4159.

Now that TP4159 has been ratified with core live migration commands, we aim to resume discussion with the upstream community and solicit feedback on what remains to support NVMe live migration in mainline.

What is implemented in this RFC?
--------------------------------

1. Patch 0001 introduces the core vfio-nvme driver infrastructure including helper routines and basic driver registration.

2. Patch 0002 adds TP4159-specific command definitions and updates existing NVMe data structures, such as nvme_id_ctrl.

3. Patch 0003 exports helpers from pci. (Needs a discussion)

4. Patch 0004 implements the TP4159 commands: Suspend, Resume, Get Controller State, and Set Controller State. It also includes debug helpers and command parsing logic.

Open Issues and Discussion Points
---------------------------------

1. This RFC exposes two new interfaces from the nvme-pci driver to submit admin commands for VF devices through the PF. We welcome input on the correct or preferred upstream approach for this.

2. Are there any gaps between the current VFIO live migration architecture and what is required to fully support NVMe VF migration?

3. TP4193 is under development in NVMe TWG, it will define subsystem state and missing configuration functionality. Are there additional capabilities or architecture changes needed beyond what TP4193 will cover to upstream the VFIO NVMe Live Migration support from spec or from linux kernel point of view ?

NVIDIA and Intel has started the NVMe Live Migration upstreaming work and fully committed to upstreaming NVMe live migration support, we are also eager to align ongoing development with community expectations and bring the feedback to the standards representing the kernel community.

This RFC is compiles and generated on linux-nvme tree branch nvme-6.17 HEAD :-

commit 70d12a283303b1241884b04f77dc1b07fdbbc90e (origin/nvme-6.17)

Author: Maurizio Lombardi <[email protected]>
Date:   Wed Jul 2 16:06:29 2025 +0200

nvme-tcp: log TLS handshake failures at error level

We greatly appreciate your feedback and comments on this work.

-ck

Chaitanya Kulkarni (4):
vfio-nvme: add vfio-nvme lm driver infrastructure
nvme: add live migration TP 4159 definitions
nvme: export helpers to implement vfio-nvme lm
vfio-nvme: implement TP4159 live migration cmds

 drivers/nvme/host/core.c       |    5 +-
 drivers/nvme/host/nvme.h       |    5 +
 drivers/nvme/host/pci.c        |   34 ++
 drivers/vfio/pci/Kconfig       |    2 +
 drivers/vfio/pci/Makefile      |    2 +
 drivers/vfio/pci/nvme/Kconfig  |   10 +
 drivers/vfio/pci/nvme/Makefile |    6 +
 drivers/vfio/pci/nvme/nvme.c   | 1036 ++++++++++++++++++++++++++++++++
 drivers/vfio/pci/nvme/nvme.h   |   39 ++
 include/linux/nvme.h           |  334 +++++++++-
 10 files changed, 1471 insertions(+), 2 deletions(-)
 create mode 100644 drivers/vfio/pci/nvme/Kconfig
 create mode 100644 drivers/vfio/pci/nvme/Makefile
 create mode 100644 drivers/vfio/pci/nvme/nvme.c
 create mode 100644 drivers/vfio/pci/nvme/nvme.h
Signature · 1 line
2.40.0
#1 lore raw

RFC PATCH 1/4 vfio-nvme: add vfio-nvme lm driver infrastructure

Add foundational infrastructure for vfio-nvme, enabling support for live migration of NVMe devices via the VFIO framework. The following components are included:

- Core driver skeleton for vfio-nvme support under drivers/vfio/pci/nvme/
- Definitions of basic data structures used in live migration (e.g., nvmevf_pci_core_device and nvmevf_migration_file)
- Implementation of helper routines for managing migration file state
- Integration of PCI driver callbacks and error handling logic
- Registration with vfio-pci-core through nvmevf_pci_ops
- Initial support for VFIO migration states and device open/close flows

Subsequent patches will build upon this base to implement actual live migration commands and complete the vfio device state handling logic.

---
 drivers/vfio/pci/Kconfig       |   2 +
 drivers/vfio/pci/Makefile      |   2 +
 drivers/vfio/pci/nvme/Kconfig  |  10 ++
 drivers/vfio/pci/nvme/Makefile |   3 +
 drivers/vfio/pci/nvme/nvme.c   | 196 +++++++++++++++++++++++++++++++++
 drivers/vfio/pci/nvme/nvme.h   |  36 ++++++
 6 files changed, 249 insertions(+)
 create mode 100644 drivers/vfio/pci/nvme/Kconfig
 create mode 100644 drivers/vfio/pci/nvme/Makefile
 create mode 100644 drivers/vfio/pci/nvme/nvme.c
 create mode 100644 drivers/vfio/pci/nvme/nvme.h
drivers/vfio/pci/Kconfig +2 -0
index 2b0172f54665..8f94429e7adc 100644
@@ -67,4 +67,6 @@ source "drivers/vfio/pci/nvgrace-gpu/Kconfig"
 
 source "drivers/vfio/pci/qat/Kconfig"
 
+source "drivers/vfio/pci/nvme/Kconfig"
+
 endmenu
drivers/vfio/pci/Makefile +2 -0
index cf00c0a7e55c..be8c4b5ee0ba 100644
@@ -10,6 +10,8 @@ obj-$(CONFIG_VFIO_PCI) += vfio-pci.o
 
 obj-$(CONFIG_MLX5_VFIO_PCI)           += mlx5/
 
+obj-$(CONFIG_NVME_VFIO_PCI) += nvme/
+
 obj-$(CONFIG_HISI_ACC_VFIO_PCI) += hisilicon/
 
 obj-$(CONFIG_PDS_VFIO_PCI) += pds/
drivers/vfio/pci/nvme/Kconfig new file +10 -0
new file mode 100644
index 000000000000..12e0eaba0de1
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0-only
+config NVME_VFIO_PCI
+	tristate "VFIO support for NVMe PCI devices"
+	depends on NVME_CORE
+	depends on VFIO_PCI_CORE
+	help
+	  This provides migration support for NVMe devices using the
+	  VFIO framework.
+
+	  If you don't know what to do here, say N.
drivers/vfio/pci/nvme/Makefile new file +3 -0
new file mode 100644
index 000000000000..2f4a0ad3d9cf
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-$(CONFIG_NVME_VFIO_PCI) += nvme-vfio-pci.o
+nvme-vfio-pci-y := nvme.o
drivers/vfio/pci/nvme/nvme.c new file +196 -0

196 lines not on this page · raw message

drivers/vfio/pci/nvme/nvme.h new file +36 -0
new file mode 100644
index 000000000000..ee602254679e
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022, INTEL CORPORATION. All rights reserved
+ * Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved
+ */
+
+#ifndef NVME_VFIO_PCI_H
+#define NVME_VFIO_PCI_H
+
+#include <linux/kernel.h>
+#include <linux/vfio_pci_core.h>
+#include <linux/nvme.h>
+
+struct nvmevf_migration_file {
+	struct file *filp;
+	struct mutex lock;
+	bool disabled;
+	u8 *vf_data;
+	size_t total_length;
+};
+
+struct nvmevf_pci_core_device {
+	struct vfio_pci_core_device core_device;
+	int vf_id;
+	u8 migrate_cap:1;
+	u8 deferred_reset:1;
+	/* protect migration state */
+	struct mutex state_mutex;
+	enum vfio_device_mig_state mig_state;
+	/* protect the reset_done flow */
+	spinlock_t reset_lock;
+	struct nvmevf_migration_file *resuming_migf;
+	struct nvmevf_migration_file *saving_migf;
+};
+
+#endif /* NVME_VFIO_PCI_H */
Signature · 1 line
2.40.0
#2 lore raw

RE: [RFC PATCH 1/4] vfio-nvme: add vfio-nvme lm driver infrastructure

Quoted 126 lines

-----Original Message-----
From: Chaitanya Kulkarni <[email protected]>
Sent: Sunday, August 3, 2025 3:47 AM
To: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Shameerali Kolothum Thodi <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Chaitanya Kulkarni <[email protected]>; Lei Rao <[email protected]>
Subject: [RFC PATCH 1/4] vfio-nvme: add vfio-nvme lm driver infrastructure

Add foundational infrastructure for vfio-nvme, enabling support for live migration of NVMe devices via the VFIO framework. The following components are included:

- Core driver skeleton for vfio-nvme support under drivers/vfio/pci/nvme/
- Definitions of basic data structures used in live migration (e.g., nvmevf_pci_core_device and nvmevf_migration_file)
- Implementation of helper routines for managing migration file state
- Integration of PCI driver callbacks and error handling logic
- Registration with vfio-pci-core through nvmevf_pci_ops
- Initial support for VFIO migration states and device open/close flows

Subsequent patches will build upon this base to implement actual live migration commands and complete the vfio device state handling logic.

---
 drivers/vfio/pci/Kconfig       |   2 +
 drivers/vfio/pci/Makefile      |   2 +
 drivers/vfio/pci/nvme/Kconfig  |  10 ++
 drivers/vfio/pci/nvme/Makefile |   3 +
 drivers/vfio/pci/nvme/nvme.c   | 196
+++++++++++++++++++++++++++++++++
 drivers/vfio/pci/nvme/nvme.h   |  36 ++++++
 6 files changed, 249 insertions(+)
 create mode 100644 drivers/vfio/pci/nvme/Kconfig
 create mode 100644 drivers/vfio/pci/nvme/Makefile
 create mode 100644 drivers/vfio/pci/nvme/nvme.c
 create mode 100644 drivers/vfio/pci/nvme/nvme.h
drivers/vfio/pci/Kconfig +2 -0
index 2b0172f54665..8f94429e7adc 100644
@@ -67,4 +67,6 @@ source "drivers/vfio/pci/nvgrace-gpu/Kconfig"
 
 source "drivers/vfio/pci/qat/Kconfig"
 
+source "drivers/vfio/pci/nvme/Kconfig"
+
 endmenu
drivers/vfio/pci/Makefile +2 -0
index cf00c0a7e55c..be8c4b5ee0ba 100644
@@ -10,6 +10,8 @@ obj-$(CONFIG_VFIO_PCI) += vfio-pci.o
 
 obj-$(CONFIG_MLX5_VFIO_PCI)           += mlx5/
 
+obj-$(CONFIG_NVME_VFIO_PCI) += nvme/
+
 obj-$(CONFIG_HISI_ACC_VFIO_PCI) += hisilicon/
 
 obj-$(CONFIG_PDS_VFIO_PCI) += pds/
drivers/vfio/pci/nvme/Kconfig new file +10 -0
new file mode 100644
index 000000000000..12e0eaba0de1
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0-only
+config NVME_VFIO_PCI
+	tristate "VFIO support for NVMe PCI devices"
+	depends on NVME_CORE
+	depends on VFIO_PCI_CORE
+	help
+	  This provides migration support for NVMe devices using the
+	  VFIO framework.
+
+	  If you don't know what to do here, say N.
drivers/vfio/pci/nvme/Makefile new file +3 -0
new file mode 100644
index 000000000000..2f4a0ad3d9cf
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-$(CONFIG_NVME_VFIO_PCI) += nvme-vfio-pci.o
+nvme-vfio-pci-y := nvme.o
drivers/vfio/pci/nvme/nvme.c new file +22 -0
new file mode 100644
index 000000000000..08bee3274207
@@ -0,0 +1,196 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022, INTEL CORPORATION. All rights reserved
+ * Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved
+ */
+
+#include <linux/device.h>
+#include <linux/eventfd.h>
+#include <linux/file.h>
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/pci.h>
+#include <linux/types.h>
+#include <linux/vfio.h>
+#include <linux/anon_inodes.h>
+#include <linux/kernel.h>
+#include <linux/vfio_pci_core.h>
+
+#include "nvme.h"
+
+static void nvmevf_disable_fd(struct nvmevf_migration_file *migf)

drivers/vfio/pci/nvme/nvme.c line 32

Quoted 10 lines
drivers/vfio/pci/nvme/nvme.c new file +10 -0
+{
+	mutex_lock(&migf->lock);
+
+	/* release the device states buffer */
+	kvfree(migf->vf_data);
+	migf->vf_data = NULL;
+	migf->disabled = true;
+	migf->total_length = 0;
+	migf->filp->f_pos = 0;
+	mutex_unlock(&migf->lock);

May be we can use guard(mutex) here and elsewhere for this driver now?

Quoted 85 lines
drivers/vfio/pci/nvme/nvme.c new file +3 -0
+}
+
+static void nvmevf_disable_fds(struct nvmevf_pci_core_device

*nvmevf_dev)

drivers/vfio/pci/nvme/nvme.c new file +15 -0
+{
+	if (nvmevf_dev->resuming_migf) {
+		nvmevf_disable_fd(nvmevf_dev->resuming_migf);
+		fput(nvmevf_dev->resuming_migf->filp);
+		nvmevf_dev->resuming_migf = NULL;
+	}
+
+	if (nvmevf_dev->saving_migf) {
+		nvmevf_disable_fd(nvmevf_dev->saving_migf);
+		fput(nvmevf_dev->saving_migf->filp);
+		nvmevf_dev->saving_migf = NULL;
+	}
+}
+
+static void nvmevf_state_mutex_unlock(struct nvmevf_pci_core_device

*nvmevf_dev)

drivers/vfio/pci/nvme/nvme.c new file +16 -0
+{
+	lockdep_assert_held(&nvmevf_dev->state_mutex);
+again:
+	spin_lock(&nvmevf_dev->reset_lock);
+	if (nvmevf_dev->deferred_reset) {
+		nvmevf_dev->deferred_reset = false;
+		spin_unlock(&nvmevf_dev->reset_lock);
+		nvmevf_dev->mig_state = VFIO_DEVICE_STATE_RUNNING;
+		nvmevf_disable_fds(nvmevf_dev);
+		goto again;
+	}
+	mutex_unlock(&nvmevf_dev->state_mutex);
+	spin_unlock(&nvmevf_dev->reset_lock);
+}
+
+static struct nvmevf_pci_core_device *nvmevf_drvdata(struct pci_dev

*pdev)

drivers/vfio/pci/nvme/nvme.c new file +2 -0
+{
+	struct vfio_pci_core_device *core_device = dev_get_drvdata(&pdev-
Quoted 1 line

dev);

drivers/vfio/pci/nvme/nvme.c new file +12 -0
+
+	return container_of(core_device, struct nvmevf_pci_core_device,
+			    core_device);
+}
+
+static int nvmevf_pci_open_device(struct vfio_device *core_vdev)
+{
+	struct nvmevf_pci_core_device *nvmevf_dev;
+	struct vfio_pci_core_device *vdev;
+	int ret;
+
+	nvmevf_dev = container_of(core_vdev, struct

nvmevf_pci_core_device,

drivers/vfio/pci/nvme/nvme.c new file +18 -0
+			core_device.vdev);
+	vdev = &nvmevf_dev->core_device;
+
+	ret = vfio_pci_core_enable(vdev);
+	if (ret)
+		return ret;
+
+	if (nvmevf_dev->migrate_cap)
+		nvmevf_dev->mig_state = VFIO_DEVICE_STATE_RUNNING;
+	vfio_pci_core_finish_enable(vdev);
+	return 0;
+}
+
+static void nvmevf_pci_close_device(struct vfio_device *core_vdev)
+{
+	struct nvmevf_pci_core_device *nvmevf_dev;
+
+	nvmevf_dev = container_of(core_vdev, struct

nvmevf_pci_core_device,

drivers/vfio/pci/nvme/nvme.c new file +13 -0
+			core_device.vdev);
+
+	if (nvmevf_dev->migrate_cap) {
+		mutex_lock(&nvmevf_dev->state_mutex);
+		nvmevf_disable_fds(nvmevf_dev);
+		nvmevf_state_mutex_unlock(nvmevf_dev);
+	}
+
+	vfio_pci_core_close_device(core_vdev);
+}
+
+static const struct vfio_device_ops nvmevf_pci_ops = {
+	.name = "nvme-vfio-pci",

drivers/vfio/pci/nvme/nvme.c line 121

Quoted 10 lines
drivers/vfio/pci/nvme/nvme.c new file +10 -0
+	.release = vfio_pci_core_release_dev,
+	.open_device = nvmevf_pci_open_device,
+	.close_device = nvmevf_pci_close_device,
+	.ioctl = vfio_pci_core_ioctl,
+	.device_feature = vfio_pci_core_ioctl_feature,
+	.read = vfio_pci_core_read,
+	.write = vfio_pci_core_write,
+	.mmap = vfio_pci_core_mmap,
+	.request = vfio_pci_core_request,
+	.match = vfio_pci_core_match,

Any reason why this driver don't need the iommufd related callbacks? bind_iommufd/unbind_iommufd etc?

Quoted 50 lines
drivers/vfio/pci/nvme/nvme.c new file +9 -0
+};
+
+static int nvmevf_pci_probe(struct pci_dev *pdev,
+			    const struct pci_device_id *id)
+{
+	struct nvmevf_pci_core_device *nvmevf_dev;
+	int ret;
+
+	nvmevf_dev = vfio_alloc_device(nvmevf_pci_core_device,

core_device.vdev,

drivers/vfio/pci/nvme/nvme.c new file +19 -0
+				       &pdev->dev, &nvmevf_pci_ops);
+	if (IS_ERR(nvmevf_dev))
+		return PTR_ERR(nvmevf_dev);
+
+	dev_set_drvdata(&pdev->dev, &nvmevf_dev->core_device);
+	ret = vfio_pci_core_register_device(&nvmevf_dev->core_device);
+	if (ret)
+		goto out_put_dev;
+
+	return 0;
+
+out_put_dev:
+	vfio_put_device(&nvmevf_dev->core_device.vdev);
+	return ret;
+}
+
+static void nvmevf_pci_remove(struct pci_dev *pdev)
+{
+	struct nvmevf_pci_core_device *nvmevf_dev =

nvmevf_drvdata(pdev);

drivers/vfio/pci/nvme/nvme.c new file +8 -0
+
+	vfio_pci_core_unregister_device(&nvmevf_dev->core_device);
+	vfio_put_device(&nvmevf_dev->core_device.vdev);
+}
+
+static void nvmevf_pci_aer_reset_done(struct pci_dev *pdev)
+{
+	struct nvmevf_pci_core_device *nvmevf_dev =

nvmevf_drvdata(pdev);

drivers/vfio/pci/nvme/nvme.c new file +7 -0
+
+	if (!nvmevf_dev->migrate_cap)
+		return;
+
+	/*
+	 * As the higher VFIO layers are holding locks across reset and using
+	 * those same locks with the mm_lock we need to prevent ABBA

deadlock

drivers/vfio/pci/nvme/nvme.c new file +2 -0
+	 * with the state_mutex and mm_lock.
+	 * In case the state_mutex was taken already we defer the cleanup

work

drivers/vfio/pci/nvme/nvme.c line 168

Quoted 2 lines
drivers/vfio/pci/nvme/nvme.c new file +2 -0
+	 * to the unlock flow of the other running context.
+	 */

I am not sure this is relevant for this driver. This deferred logic was added to avoid a circular locking issue w.r.t copy_to/from_user() functions holding mm_lock under state mutex.

See this,
https://lore.kernel.org/all/[email protected]/

Looking at patch #4 it doesn't look like this has that issue. May be I missed it. Please double check.

Thanks,
Shameer

Quoted 73 lines
drivers/vfio/pci/nvme/nvme.c new file +28 -0
+	spin_lock(&nvmevf_dev->reset_lock);
+	nvmevf_dev->deferred_reset = true;
+	if (!mutex_trylock(&nvmevf_dev->state_mutex)) {
+		spin_unlock(&nvmevf_dev->reset_lock);
+		return;
+	}
+	spin_unlock(&nvmevf_dev->reset_lock);
+	nvmevf_state_mutex_unlock(nvmevf_dev);
+}
+
+static const struct pci_error_handlers nvmevf_err_handlers = {
+	.reset_done = nvmevf_pci_aer_reset_done,
+	.error_detected = vfio_pci_core_aer_err_detected,
+};
+
+static struct pci_driver nvmevf_pci_driver = {
+	.name = KBUILD_MODNAME,
+	.probe = nvmevf_pci_probe,
+	.remove = nvmevf_pci_remove,
+	.err_handler = &nvmevf_err_handlers,
+	.driver_managed_dma = true,
+};
+
+module_pci_driver(nvmevf_pci_driver);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Chaitanya Kulkarni <[email protected]>");
+MODULE_DESCRIPTION("NVMe VFIO PCI - VFIO PCI driver with live

migration support for NVMe");

drivers/vfio/pci/nvme/nvme.h new file +36 -1
new file mode 100644
index 000000000000..ee602254679e
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022, INTEL CORPORATION. All rights reserved
+ * Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved
+ */
+
+#ifndef NVME_VFIO_PCI_H
+#define NVME_VFIO_PCI_H
+
+#include <linux/kernel.h>
+#include <linux/vfio_pci_core.h>
+#include <linux/nvme.h>
+
+struct nvmevf_migration_file {
+	struct file *filp;
+	struct mutex lock;
+	bool disabled;
+	u8 *vf_data;
+	size_t total_length;
+};
+
+struct nvmevf_pci_core_device {
+	struct vfio_pci_core_device core_device;
+	int vf_id;
+	u8 migrate_cap:1;
+	u8 deferred_reset:1;
+	/* protect migration state */
+	struct mutex state_mutex;
+	enum vfio_device_mig_state mig_state;
+	/* protect the reset_done flow */
+	spinlock_t reset_lock;
+	struct nvmevf_migration_file *resuming_migf;
+	struct nvmevf_migration_file *saving_migf;
+};
+
+#endif /* NVME_VFIO_PCI_H */
--

2.40.0

#3 lore raw

Re: [RFC PATCH 1/4] vfio-nvme: add vfio-nvme lm driver infrastructure

In reply to Chaitanya Kulkarni
Quoted 5 lines · Chaitanya Kulkarni

Add foundational infrastructure for vfio-nvme, enabling support for live migration of NVMe devices via the VFIO framework. The following components are included:

drivers/vfio/pci/nvme/nvme.c new file +1 -0
+static void nvmevf_pci_aer_reset_done(struct pci_dev *pdev)

drivers/vfio/pci/nvme/nvme.c line 165

Quoted 10 lines
drivers/vfio/pci/nvme/nvme.c new file +10 -0
+{
+	struct nvmevf_pci_core_device *nvmevf_dev = nvmevf_drvdata(pdev);
+
+	if (!nvmevf_dev->migrate_cap)
+		return;
+
+	/*
+	 * As the higher VFIO layers are holding locks across reset and using
+	 * those same locks with the mm_lock we need to prevent ABBA deadlock
+	 * with the state_mutex and mm_lock.

Add blank line between paragraphs.

Quoted 12 lines
drivers/vfio/pci/nvme/nvme.c new file +12 -0
+	 * In case the state_mutex was taken already we defer the cleanup work
+	 * to the unlock flow of the other running context.
+	 */
+	spin_lock(&nvmevf_dev->reset_lock);
+	nvmevf_dev->deferred_reset = true;
+	if (!mutex_trylock(&nvmevf_dev->state_mutex)) {
+		spin_unlock(&nvmevf_dev->reset_lock);
+		return;
+	}
+	spin_unlock(&nvmevf_dev->reset_lock);
+	nvmevf_state_mutex_unlock(nvmevf_dev);
+}
#4 lore raw

Re: [RFC PATCH 1/4] vfio-nvme: add vfio-nvme lm driver infrastructure

In reply to Chaitanya Kulkarni
Quoted 272 lines · On Sat, 2 Aug 2025 19:47:02 -0700 Chaitanya Kulkarni <[email protected]> wrote:

Add foundational infrastructure for vfio-nvme, enabling support for live migration of NVMe devices via the VFIO framework. The following components are included:

- Core driver skeleton for vfio-nvme support under drivers/vfio/pci/nvme/
- Definitions of basic data structures used in live migration (e.g., nvmevf_pci_core_device and nvmevf_migration_file)
- Implementation of helper routines for managing migration file state
- Integration of PCI driver callbacks and error handling logic
- Registration with vfio-pci-core through nvmevf_pci_ops
- Initial support for VFIO migration states and device open/close flows

Subsequent patches will build upon this base to implement actual live migration commands and complete the vfio device state handling logic.

---
 drivers/vfio/pci/Kconfig       |   2 +
 drivers/vfio/pci/Makefile      |   2 +
 drivers/vfio/pci/nvme/Kconfig  |  10 ++
 drivers/vfio/pci/nvme/Makefile |   3 +
 drivers/vfio/pci/nvme/nvme.c   | 196 +++++++++++++++++++++++++++++++++
 drivers/vfio/pci/nvme/nvme.h   |  36 ++++++
 6 files changed, 249 insertions(+)
 create mode 100644 drivers/vfio/pci/nvme/Kconfig
 create mode 100644 drivers/vfio/pci/nvme/Makefile
 create mode 100644 drivers/vfio/pci/nvme/nvme.c
 create mode 100644 drivers/vfio/pci/nvme/nvme.h
drivers/vfio/pci/Kconfig +2 -0
index 2b0172f54665..8f94429e7adc 100644
@@ -67,4 +67,6 @@ source "drivers/vfio/pci/nvgrace-gpu/Kconfig"
 
 source "drivers/vfio/pci/qat/Kconfig"
 
+source "drivers/vfio/pci/nvme/Kconfig"
+
 endmenu
drivers/vfio/pci/Makefile +2 -0
index cf00c0a7e55c..be8c4b5ee0ba 100644
@@ -10,6 +10,8 @@ obj-$(CONFIG_VFIO_PCI) += vfio-pci.o
 
 obj-$(CONFIG_MLX5_VFIO_PCI)           += mlx5/
 
+obj-$(CONFIG_NVME_VFIO_PCI) += nvme/
+
 obj-$(CONFIG_HISI_ACC_VFIO_PCI) += hisilicon/
 
 obj-$(CONFIG_PDS_VFIO_PCI) += pds/
drivers/vfio/pci/nvme/Kconfig new file +10 -0
new file mode 100644
index 000000000000..12e0eaba0de1
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0-only
+config NVME_VFIO_PCI
+	tristate "VFIO support for NVMe PCI devices"
+	depends on NVME_CORE
+	depends on VFIO_PCI_CORE
+	help
+	  This provides migration support for NVMe devices using the
+	  VFIO framework.
+
+	  If you don't know what to do here, say N.
drivers/vfio/pci/nvme/Makefile new file +3 -0
new file mode 100644
index 000000000000..2f4a0ad3d9cf
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-$(CONFIG_NVME_VFIO_PCI) += nvme-vfio-pci.o
+nvme-vfio-pci-y := nvme.o
drivers/vfio/pci/nvme/nvme.c new file +186 -0
new file mode 100644
index 000000000000..08bee3274207
@@ -0,0 +1,196 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022, INTEL CORPORATION. All rights reserved
+ * Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved
+ */
+
+#include <linux/device.h>
+#include <linux/eventfd.h>
+#include <linux/file.h>
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/pci.h>
+#include <linux/types.h>
+#include <linux/vfio.h>
+#include <linux/anon_inodes.h>
+#include <linux/kernel.h>
+#include <linux/vfio_pci_core.h>
+
+#include "nvme.h"
+
+static void nvmevf_disable_fd(struct nvmevf_migration_file *migf)
+{
+	mutex_lock(&migf->lock);
+
+	/* release the device states buffer */
+	kvfree(migf->vf_data);
+	migf->vf_data = NULL;
+	migf->disabled = true;
+	migf->total_length = 0;
+	migf->filp->f_pos = 0;
+	mutex_unlock(&migf->lock);
+}
+
+static void nvmevf_disable_fds(struct nvmevf_pci_core_device *nvmevf_dev)
+{
+	if (nvmevf_dev->resuming_migf) {
+		nvmevf_disable_fd(nvmevf_dev->resuming_migf);
+		fput(nvmevf_dev->resuming_migf->filp);
+		nvmevf_dev->resuming_migf = NULL;
+	}
+
+	if (nvmevf_dev->saving_migf) {
+		nvmevf_disable_fd(nvmevf_dev->saving_migf);
+		fput(nvmevf_dev->saving_migf->filp);
+		nvmevf_dev->saving_migf = NULL;
+	}
+}
+
+static void nvmevf_state_mutex_unlock(struct nvmevf_pci_core_device *nvmevf_dev)
+{
+	lockdep_assert_held(&nvmevf_dev->state_mutex);
+again:
+	spin_lock(&nvmevf_dev->reset_lock);
+	if (nvmevf_dev->deferred_reset) {
+		nvmevf_dev->deferred_reset = false;
+		spin_unlock(&nvmevf_dev->reset_lock);
+		nvmevf_dev->mig_state = VFIO_DEVICE_STATE_RUNNING;
+		nvmevf_disable_fds(nvmevf_dev);
+		goto again;
+	}
+	mutex_unlock(&nvmevf_dev->state_mutex);
+	spin_unlock(&nvmevf_dev->reset_lock);
+}
+
+static struct nvmevf_pci_core_device *nvmevf_drvdata(struct pci_dev *pdev)
+{
+	struct vfio_pci_core_device *core_device = dev_get_drvdata(&pdev->dev);
+
+	return container_of(core_device, struct nvmevf_pci_core_device,
+			    core_device);
+}
+
+static int nvmevf_pci_open_device(struct vfio_device *core_vdev)
+{
+	struct nvmevf_pci_core_device *nvmevf_dev;
+	struct vfio_pci_core_device *vdev;
+	int ret;
+
+	nvmevf_dev = container_of(core_vdev, struct nvmevf_pci_core_device,
+			core_device.vdev);
+	vdev = &nvmevf_dev->core_device;
+
+	ret = vfio_pci_core_enable(vdev);
+	if (ret)
+		return ret;
+
+	if (nvmevf_dev->migrate_cap)
+		nvmevf_dev->mig_state = VFIO_DEVICE_STATE_RUNNING;
+	vfio_pci_core_finish_enable(vdev);
+	return 0;
+}
+
+static void nvmevf_pci_close_device(struct vfio_device *core_vdev)
+{
+	struct nvmevf_pci_core_device *nvmevf_dev;
+
+	nvmevf_dev = container_of(core_vdev, struct nvmevf_pci_core_device,
+			core_device.vdev);
+
+	if (nvmevf_dev->migrate_cap) {
+		mutex_lock(&nvmevf_dev->state_mutex);
+		nvmevf_disable_fds(nvmevf_dev);
+		nvmevf_state_mutex_unlock(nvmevf_dev);
+	}
+
+	vfio_pci_core_close_device(core_vdev);
+}
+
+static const struct vfio_device_ops nvmevf_pci_ops = {
+	.name = "nvme-vfio-pci",
+	.release = vfio_pci_core_release_dev,
+	.open_device = nvmevf_pci_open_device,
+	.close_device = nvmevf_pci_close_device,
+	.ioctl = vfio_pci_core_ioctl,
+	.device_feature = vfio_pci_core_ioctl_feature,
+	.read = vfio_pci_core_read,
+	.write = vfio_pci_core_write,
+	.mmap = vfio_pci_core_mmap,
+	.request = vfio_pci_core_request,
+	.match = vfio_pci_core_match,
+};
+
+static int nvmevf_pci_probe(struct pci_dev *pdev,
+			    const struct pci_device_id *id)
+{
+	struct nvmevf_pci_core_device *nvmevf_dev;
+	int ret;
+
+	nvmevf_dev = vfio_alloc_device(nvmevf_pci_core_device, core_device.vdev,
+				       &pdev->dev, &nvmevf_pci_ops);
+	if (IS_ERR(nvmevf_dev))
+		return PTR_ERR(nvmevf_dev);
+
+	dev_set_drvdata(&pdev->dev, &nvmevf_dev->core_device);
+	ret = vfio_pci_core_register_device(&nvmevf_dev->core_device);
+	if (ret)
+		goto out_put_dev;
+
+	return 0;
+
+out_put_dev:
+	vfio_put_device(&nvmevf_dev->core_device.vdev);
+	return ret;
+}
+
+static void nvmevf_pci_remove(struct pci_dev *pdev)
+{
+	struct nvmevf_pci_core_device *nvmevf_dev = nvmevf_drvdata(pdev);
+
+	vfio_pci_core_unregister_device(&nvmevf_dev->core_device);
+	vfio_put_device(&nvmevf_dev->core_device.vdev);
+}
+
+static void nvmevf_pci_aer_reset_done(struct pci_dev *pdev)
+{
+	struct nvmevf_pci_core_device *nvmevf_dev = nvmevf_drvdata(pdev);
+
+	if (!nvmevf_dev->migrate_cap)
+		return;
+
+	/*
+	 * As the higher VFIO layers are holding locks across reset and using
+	 * those same locks with the mm_lock we need to prevent ABBA deadlock
+	 * with the state_mutex and mm_lock.
+	 * In case the state_mutex was taken already we defer the cleanup work
+	 * to the unlock flow of the other running context.
+	 */
+	spin_lock(&nvmevf_dev->reset_lock);
+	nvmevf_dev->deferred_reset = true;
+	if (!mutex_trylock(&nvmevf_dev->state_mutex)) {
+		spin_unlock(&nvmevf_dev->reset_lock);
+		return;
+	}
+	spin_unlock(&nvmevf_dev->reset_lock);
+	nvmevf_state_mutex_unlock(nvmevf_dev);
+}
+
+static const struct pci_error_handlers nvmevf_err_handlers = {
+	.reset_done = nvmevf_pci_aer_reset_done,
+	.error_detected = vfio_pci_core_aer_err_detected,
+};
+
+static struct pci_driver nvmevf_pci_driver = {
+	.name = KBUILD_MODNAME,
+	.probe = nvmevf_pci_probe,

drivers/vfio/pci/nvme/nvme.c line 196

Quoted 10 lines
drivers/vfio/pci/nvme/nvme.c new file +10 -0
+	.remove = nvmevf_pci_remove,
+	.err_handler = &nvmevf_err_handlers,
+	.driver_managed_dma = true,
+};
+
+module_pci_driver(nvmevf_pci_driver);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Chaitanya Kulkarni <[email protected]>");
+MODULE_DESCRIPTION("NVMe VFIO PCI - VFIO PCI driver with live migration support for NVMe");

Without a MODULE_DEVICE_TABLE, what devices are ever going to use this driver? Userspace needs to be given a clue when to use this driver vs vfio-pci. We also don't have a fallback mechanism to try a driver until it fails, so this driver likely needs to take over defacto support for all NVMe devices from vfio-pci, rather that later rejecting those that don't support migration as patch 4/ implements in the .init callback. Thanks,

Alex

Quoted 42 lines
drivers/vfio/pci/nvme/nvme.h new file +36 -0
new file mode 100644
index 000000000000..ee602254679e
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022, INTEL CORPORATION. All rights reserved
+ * Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved
+ */
+
+#ifndef NVME_VFIO_PCI_H
+#define NVME_VFIO_PCI_H
+
+#include <linux/kernel.h>
+#include <linux/vfio_pci_core.h>
+#include <linux/nvme.h>
+
+struct nvmevf_migration_file {
+	struct file *filp;
+	struct mutex lock;
+	bool disabled;
+	u8 *vf_data;
+	size_t total_length;
+};
+
+struct nvmevf_pci_core_device {
+	struct vfio_pci_core_device core_device;
+	int vf_id;
+	u8 migrate_cap:1;
+	u8 deferred_reset:1;
+	/* protect migration state */
+	struct mutex state_mutex;
+	enum vfio_device_mig_state mig_state;
+	/* protect the reset_done flow */
+	spinlock_t reset_lock;
+	struct nvmevf_migration_file *resuming_migf;
+	struct nvmevf_migration_file *saving_migf;
+};
+
+#endif /* NVME_VFIO_PCI_H */
#5 lore raw

RFC PATCH 2/4 nvme: add live migration TP 4159 definitions

In reply to Chaitanya Kulkarni

nvme: add TP4159 live migration definitions and command support

Introduce core definitions and data structures required to support Live Migration operations as described in TP4159. These updates enable controller state extraction, transfer, and restoration.

Key changes:

- Extend nvme_id_ctrl with TP4159 migration capability fields:
- CMMRTD/NMMRTD: Migration tracking granularity
- MCUDMQ/MNSUDMQ: Migration controller suspend queue depth
- TRATTR: Tracking attribute bitfield (e.g., THMCS, TUDCS)

- Define controller state format discovery (CNS=0x20):
- struct nvme_lm_supported_ctrl_state_fmts: layout for reporting NVMe-defined and vendor-defined controller state formats
- struct nvme_lm_ctrl_state_fmts_info: internal parsing view

- Add live migration controller state format v0 support:
- struct nvme_lm_nvme_cs_v0_state: encapsulates suspendable state including I/O submission and completion queues
- struct nvme_lm_iosq_state / iocq_state: PRP, QID, head/tail, etc.

- Introduce Migration Send (Opcode 0x43) and Receive (0x42):
- struct nvme_lm_send_cmd: supports suspend, resume, set state
- struct nvme_lm_recv_cmd: supports get controller state

- Support for sequence indicators (SEQIND) to allow multi-part transfer of controller state buffers during suspend/resume

- Add new admin opcodes to enum nvme_admin_opcode:
- nvme_admin_lm_send = 0x41
- nvme_admin_lm_recv = 0x42

- Extend union nvme_command to include struct nvme_lm_command, enabling transport of send/receive commands through common paths

- Add new status code definitions for migration error handling:
- NVME_SC_NOT_ENOUGH_RESOURCE
- NVME_SC_CTRL_SUSPENDED
- NVME_SC_CTRL_NOT_SUSPENDED
- NVME_SC_CTRL_DATA_QUEUE_FAIL

- Include migration-related size checks in _nvme_check_size() to ensure live migration command structures align to spec (64 bytes)

These additions form the low-level protocol and data contract supporting live migration of NVMe controllers, and are a prerequisite for implementing suspend/resume and controller state transfer flows.

---
 drivers/nvme/host/core.c |   2 +
 include/linux/nvme.h     | 334 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 335 insertions(+), 1 deletion(-)
drivers/nvme/host/core.c +2 -0
index 5d8638086cba..2445862ac7d4 100644
@@ -5314,6 +5314,8 @@ static inline void _nvme_check_size(void)
 	BUILD_BUG_ON(sizeof(struct nvme_rotational_media_log) != 512);
 	BUILD_BUG_ON(sizeof(struct nvme_dbbuf) != 64);
 	BUILD_BUG_ON(sizeof(struct nvme_directive_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct nvme_lm_send_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct nvme_lm_recv_cmd) != 64);
 	BUILD_BUG_ON(sizeof(struct nvme_feat_host_behavior) != 512);
 }
 
include/linux/nvme.h +333 -1

394 lines not on this page · raw message

Signature · 1 line
2.40.0
#6 lore raw

Re: [RFC PATCH 2/4] nvme: add live migration TP 4159 definitions

kernel test robot noticed the following build errors:

Hi Chaitanya,

[This is a private test report for your RFC patch.] kernel test robot noticed the following build errors:

[auto build test ERROR on awilliam-vfio/next]
[also build test ERROR on axboe-block/for-next linus/master v6.16 next-20250801] [cannot apply to linux-nvme/for-next awilliam-vfio/for-linus]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Chaitanya-Kulkarni/vfio-nvme-add-vfio-nvme-lm-driver-infrastructure/20250803-105056
base:   https://github.com/awilliam/linux-vfio.git next
patch link:    https://lore.kernel.org/r/20250803024705.10256-3-kch%40nvidia.com
patch subject: [RFC PATCH 2/4] nvme: add live migration TP 4159 definitions
config: m68k-allmodconfig (https://download.01.org/0day-ci/archive/20250804/[email protected]/config)
compiler: m68k-linux-gcc (GCC) 15.1.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250804/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

In file included from <command-line>:
In function '_nvme_check_size',
inlined from 'nvme_core_init' at drivers/nvme/host/core.c:5342:2:

>> include/linux/compiler_types.h:568:45: error: call to '__compiletime_assert_586' declared with attribute error: BUILD_BUG_ON failed: sizeof(struct nvme_id_ctrl) != NVME_IDENTIFY_DATA_SIZE
     568 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |                                             ^
   include/linux/compiler_types.h:549:25: note: in definition of macro '__compiletime_assert'
     549 |                         prefix ## suffix();                             \
         |                         ^~~~~~
   include/linux/compiler_types.h:568:9: note: in expansion of macro '_compiletime_assert'
     568 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |         ^~~~~~~~~~~~~~~~~~~
   include/linux/build_bug.h:39:37: note: in expansion of macro 'compiletime_assert'
      39 | #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
         |                                     ^~~~~~~~~~~~~~~~~~
   include/linux/build_bug.h:50:9: note: in expansion of macro 'BUILD_BUG_ON_MSG'
      50 |         BUILD_BUG_ON_MSG(condition, "BUILD_BUG_ON failed: " #condition)
         |         ^~~~~~~~~~~~~~~~
   drivers/nvme/host/core.c:5317:9: note: in expansion of macro 'BUILD_BUG_ON'
    5317 |         BUILD_BUG_ON(sizeof(struct nvme_id_ctrl) != NVME_IDENTIFY_DATA_SIZE);
         |         ^~~~~~~~~~~~
--
   In file included from <command-line>:
   In function '_nvme_check_size',
       inlined from 'nvme_core_init' at core.c:5342:2:
>> include/linux/compiler_types.h:568:45: error: call to '__compiletime_assert_586' declared with attribute error: BUILD_BUG_ON failed: sizeof(struct nvme_id_ctrl) != NVME_IDENTIFY_DATA_SIZE
     568 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |                                             ^
   include/linux/compiler_types.h:549:25: note: in definition of macro '__compiletime_assert'
     549 |                         prefix ## suffix();                             \
         |                         ^~~~~~
   include/linux/compiler_types.h:568:9: note: in expansion of macro '_compiletime_assert'
     568 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |         ^~~~~~~~~~~~~~~~~~~
   include/linux/build_bug.h:39:37: note: in expansion of macro 'compiletime_assert'
      39 | #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
         |                                     ^~~~~~~~~~~~~~~~~~
   include/linux/build_bug.h:50:9: note: in expansion of macro 'BUILD_BUG_ON_MSG'
      50 |         BUILD_BUG_ON_MSG(condition, "BUILD_BUG_ON failed: " #condition)
         |         ^~~~~~~~~~~~~~~~
   core.c:5317:9: note: in expansion of macro 'BUILD_BUG_ON'
    5317 |         BUILD_BUG_ON(sizeof(struct nvme_id_ctrl) != NVME_IDENTIFY_DATA_SIZE);
         |         ^~~~~~~~~~~~

vim +/__compiletime_assert_586 +568 include/linux/compiler_types.h

eb5c2d4b45e3d2 Will Deacon 2020-07-21  554  
eb5c2d4b45e3d2 Will Deacon 2020-07-21  555  #define _compiletime_assert(condition, msg, prefix, suffix) \
eb5c2d4b45e3d2 Will Deacon 2020-07-21  556  	__compiletime_assert(condition, msg, prefix, suffix)
eb5c2d4b45e3d2 Will Deacon 2020-07-21  557  
eb5c2d4b45e3d2 Will Deacon 2020-07-21  558  /**
eb5c2d4b45e3d2 Will Deacon 2020-07-21  559   * compiletime_assert - break build and emit msg if condition is false
eb5c2d4b45e3d2 Will Deacon 2020-07-21  560   * @condition: a compile-time constant condition to check
eb5c2d4b45e3d2 Will Deacon 2020-07-21  561   * @msg:       a message to emit if condition is false
eb5c2d4b45e3d2 Will Deacon 2020-07-21  562   *
eb5c2d4b45e3d2 Will Deacon 2020-07-21  563   * In tradition of POSIX assert, this macro will break the build if the
eb5c2d4b45e3d2 Will Deacon 2020-07-21  564   * supplied condition is *false*, emitting the supplied error message if the
eb5c2d4b45e3d2 Will Deacon 2020-07-21  565   * compiler has support to do so.
eb5c2d4b45e3d2 Will Deacon 2020-07-21  566   */
eb5c2d4b45e3d2 Will Deacon 2020-07-21  567  #define compiletime_assert(condition, msg) \

eb5c2d4b45e3d2 Will Deacon 2020-07-21 @568 _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
eb5c2d4b45e3d2 Will Deacon 2020-07-21 569

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#7 lore raw

RFC PATCH 3/4 nvme: export helpers to implement vfio-nvme lm

In reply to Chaitanya Kulkarni

Export the nvme_error_status() function to allow error status translation from external modules such as VFIO-based drivers.

Add helper functions in nvme-pci to support virtual function (VF) command submission and controller ID retrieval:

- nvme_submit_vf_cmd() submits a synchronous admin command from a VF context using the PF's admin queue.
- nvme_get_ctrl_id() returns the controller ID associated with a PCI device.

These changes support VFIO NVMe Live Migration workflows and infrastructure by enabling necessary low-level admin command access.

---
 drivers/nvme/host/core.c |  3 ++-
 drivers/nvme/host/nvme.h |  5 +++++
 drivers/nvme/host/pci.c  | 34 ++++++++++++++++++++++++++++++++++
 3 files changed, 41 insertions(+), 1 deletion(-)
drivers/nvme/host/core.c +2 -1
index 2445862ac7d4..3620e7cb21d1 100644
@@ -278,7 +278,7 @@ void nvme_delete_ctrl_sync(struct nvme_ctrl *ctrl)
 	nvme_put_ctrl(ctrl);
 }
 
-static blk_status_t nvme_error_status(u16 status)
+blk_status_t nvme_error_status(u16 status)
 {
 	switch (status & NVME_SCT_SC_MASK) {
 	case NVME_SC_SUCCESS:
@@ -318,6 +318,7 @@ static blk_status_t nvme_error_status(u16 status)
 		return BLK_STS_IOERR;
 	}
 }
+EXPORT_SYMBOL_GPL(nvme_error_status);
 
 static void nvme_retry_req(struct request *req)
 {
drivers/nvme/host/nvme.h +5 -0
index cfd2b5b90b91..5549c7e3bcd3 100644
@@ -1218,4 +1218,9 @@ static inline bool nvme_multi_css(struct nvme_ctrl *ctrl)
 	return (ctrl->ctrl_config & NVME_CC_CSS_MASK) == NVME_CC_CSS_CSI;
 }
 
+blk_status_t nvme_error_status(u16 status);
+int nvme_submit_vf_cmd(struct pci_dev *dev, struct nvme_command *cmd,
+		       size_t *result, void *buffer, unsigned int bufflen);
+u16 nvme_get_ctrl_id(struct pci_dev *dev);
+
 #endif /* _NVME_H */
drivers/nvme/host/pci.c +34 -0
index 4cf87fb5d857..b239d38485ee 100644
@@ -3936,6 +3936,40 @@ static struct pci_driver nvme_driver = {
 	.err_handler	= &nvme_err_handler,
 };
 
+u16 nvme_get_ctrl_id(struct pci_dev *dev)
+{
+	struct nvme_dev *ndev = pci_iov_get_pf_drvdata(dev, &nvme_driver);
+
+	return ndev->ctrl.cntlid;
+}
+EXPORT_SYMBOL_GPL(nvme_get_ctrl_id);
+
+int nvme_submit_vf_cmd(struct pci_dev *dev, struct nvme_command *cmd,
+		size_t *result, void *buffer, unsigned int bufflen)
+{
+	struct nvme_dev *ndev = NULL;
+	union nvme_result res = { };
+	int ret;
+
+	ndev = pci_iov_get_pf_drvdata(dev, &nvme_driver);
+	if (IS_ERR(ndev))
+		return PTR_ERR(ndev);
+	ret = __nvme_submit_sync_cmd(ndev->ctrl.admin_q, cmd, &res, buffer,
+					bufflen, NVME_QID_ANY, 0);
+	if (ret < 0)
+		return ret;
+
+	if (ret > 0) {
+		ret = blk_status_to_errno(nvme_error_status(ret));
+		return ret;
+	}
+
+	if (result)
+		*result = le32_to_cpu(res.u32);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(nvme_submit_vf_cmd);
+
 static int __init nvme_init(void)
 {
 	BUILD_BUG_ON(sizeof(struct nvme_create_cq) != 64);
Signature · 1 line
2.40.0
#8 lore raw

RFC PATCH 4/4 vfio-nvme: implement TP4159 live migration cmds

In reply to Chaitanya Kulkarni

Implements TP4159-based live migration support in vfio-nvme driver by integrating command execution, controller state handling, and vfio migration state transitions.

Key features:

- Use nvme_submit_vf_cmd() and nvme_get_ctrl_id() helpers in the NVMe core PCI driver for submitting admin commands on VFs.

- Implements Migration Send (opcode 0x43) and Receive (opcode 0x42) command handling for suspend, resume, get/set controller state.

_Remark_:-
We are currently in the process of defining the state in TP4193, so the current state management code will be replaced with TP4193. However, in this patch we include TP4159-compatible state management code for the sake of completeness.

- Adds parsing and serialization of controller state including:
- NVMeCS v0 controller state format (SCS-FIG6, FIG7, FIG8)
- Supported Controller State Formats (CNS=0x20 response)
- Migration file abstraction with read/write fileops

- Adds debug decoders to log IOSQ/IOCQ state during migration save

- Allocates anon inodes to handle save and resume file interfaces exposed via VFIO migration file descriptors

- Adds vfio migration state machine transitions:
- RUNNING → STOP: sends suspend command
- STOP → STOP_COPY: extracts controller state (save)
- STOP_COPY → STOP: disables file and frees buffer
- STOP → RESUMING: allocates resume file buffer
- RESUMING → STOP: loads controller state via set state
- STOP → RUNNING: resumes controller via resume command

- Hooks vfio_migration_ops into vfio_pci_ops using:
- migration_set_state() and migration_get_state()
- Uses state_mutex + reset_lock for proper concurrency

- Queries Identify Controller (CNS=01h) to check for HMLMS bit in OACS field, indicating controller migration capability

- Applies runtime checks for buffer alignment, format support, and state size bounds to ensure spec compliance

With this patch, vfio-nvme enables live migration of VF-based NVMe devices by implementing TP4159 migration command flows and vfio device state transitions required by QEMU/VMM.

---
 drivers/vfio/pci/nvme/Makefile |   3 +
 drivers/vfio/pci/nvme/nvme.c   | 840 +++++++++++++++++++++++++++++++++
 drivers/vfio/pci/nvme/nvme.h   |   3 +
 3 files changed, 846 insertions(+)
drivers/vfio/pci/nvme/Makefile +3 -0
index 2f4a0ad3d9cf..d434c943436b 100644
@@ -1,3 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0-only
+
+KBUILD_EXTRA_SYMBOLS := $(srctree)/drivers/nvme/Module.symvers
+
 obj-$(CONFIG_NVME_VFIO_PCI) += nvme-vfio-pci.o
 nvme-vfio-pci-y := nvme.o
drivers/vfio/pci/nvme/nvme.c +840 -0

862 lines not on this page · raw message

drivers/vfio/pci/nvme/nvme.h +3 -0
index ee602254679e..80dd75d33762 100644
@@ -33,4 +33,7 @@ struct nvmevf_pci_core_device {
 	struct nvmevf_migration_file *saving_migf;
 };
 
+extern int nvme_submit_vf_cmd(struct pci_dev *dev, struct nvme_command *cmd,
+			size_t *result, void *buffer, unsigned int bufflen);
+extern u16 nvme_get_ctrl_id(struct pci_dev *dev);
 #endif /* NVME_VFIO_PCI_H */
Signature · 1 line
2.40.0
#9 lore raw

Re: [RFC PATCH 4/4] vfio-nvme: implement TP4159 live migration cmds

kernel test robot noticed the following build warnings:

Hi Chaitanya,

[This is a private test report for your RFC patch.] kernel test robot noticed the following build warnings:

[auto build test WARNING on awilliam-vfio/next]
[also build test WARNING on axboe-block/for-next linus/master v6.16 next-20250801] [cannot apply to linux-nvme/for-next awilliam-vfio/for-linus]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Chaitanya-Kulkarni/vfio-nvme-add-vfio-nvme-lm-driver-infrastructure/20250803-105056
base:   https://github.com/awilliam/linux-vfio.git next
patch link:    https://lore.kernel.org/r/20250803024705.10256-5-kch%40nvidia.com
patch subject: [RFC PATCH 4/4] vfio-nvme: implement TP4159 live migration cmds
config: loongarch-allmodconfig (https://download.01.org/0day-ci/archive/20250804/[email protected]/config)
compiler: clang version 19.1.7 (https://github.com/llvm/llvm-project cd708029e0b2869e80abe31ddb175f7c35361f90)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250804/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All warnings (new ones prefixed by >>):

>> drivers/vfio/pci/nvme/nvme.c:332:18: warning: variable 'hdr' is uninitialized when used here [-Wuninitialized]
     332 |         if (le16_to_cpu(hdr->nvmecss) > NVME_LM_MAX_NVMECS ||
         |                         ^~~
   include/linux/byteorder/generic.h:91:21: note: expanded from macro 'le16_to_cpu'
      91 | #define le16_to_cpu __le16_to_cpu
         |                     ^
   include/uapi/linux/byteorder/little_endian.h:37:51: note: expanded from macro '__le16_to_cpu'
      37 | #define __le16_to_cpu(x) ((__force __u16)(__le16)(x))
         |                                                   ^
   drivers/vfio/pci/nvme/nvme.c:310:32: note: initialize the variable 'hdr' to silence this warning
     310 |         struct nvme_lm_ctrl_state *hdr;
         |                                       ^
         |                                        = NULL

1 warning generated.

vim +/hdr +332 drivers/vfio/pci/nvme/nvme.c

303
304 static int nvmevf_get_ctrl_state(struct pci_dev *dev,
305 __u8 csvi, __u8 csuuidi, __u8 csuidxp,
306 struct nvmevf_migration_file *migf,
307 struct nvme_lm_ctrl_state_info *state)
308 {
309 struct nvme_command c = { };
310 struct nvme_lm_ctrl_state *hdr;
311 /* Make sure hdr_len is a multiple of 4 */
312 size_t hdr_len = ALIGN(sizeof(*hdr), 4);
313 __u16 id = nvme_get_ctrl_id(dev);
314 void *local_buf;
315 size_t len;
316 int ret;
317
318 /* Step 1: Issue Migration Receive (Select = 0) to get header */
319 local_buf = kzalloc(hdr_len, GFP_KERNEL);
320 if (!local_buf)
321 return -ENOMEM;
322
323 nvmevf_init_get_ctrl_state_cmd(&c, id, csvi, csuuidi, csuidxp, hdr_len);
324 ret = nvme_submit_vf_cmd(dev, &c, NULL, local_buf, hdr_len);
325 if (ret) {
326 dev_warn(&dev->dev,
327 "nvme_admin_lm_recv failed (ret=0x%x)\n", ret);
328 kfree(local_buf);
329 return ret;
330 }
331

 > 332		if (le16_to_cpu(hdr->nvmecss) > NVME_LM_MAX_NVMECS ||

333 le16_to_cpu(hdr->vss) > NVME_LM_MAX_VSD) {
334 kfree(local_buf);
335 return -EINVAL;
336 }
337
338 hdr = local_buf;
339 len = hdr_len + 4 * (le16_to_cpu(hdr->nvmecss) + le16_to_cpu(hdr->vss));
340
341 kfree(local_buf);
342
343 if (len == hdr_len)
344 dev_warn(&dev->dev, "nvmecss == 0 or vss = 0\n");
345
346 /* Step 2: Allocate full buffer */
347 migf->total_length = len;
348 migf->vf_data = kvzalloc(migf->total_length, GFP_KERNEL);
349 if (!migf->vf_data)
350 return -ENOMEM;
351
352 memset(&c, 0, sizeof(c));
353 nvmevf_init_get_ctrl_state_cmd(&c, id, csvi, csuuidi, csuidxp, len);
354 ret = nvme_submit_vf_cmd(dev, &c, NULL, migf->vf_data, len);
355 if (ret)
356 goto free_big;
357
358 /* Populate state struct */
359 hdr = (struct nvme_lm_ctrl_state *)migf->vf_data;
360 state->raw = hdr;
361 state->total_len = len;
362 state->version = hdr->version;
363 state->csattr = hdr->csattr;
364 state->nvmecss = hdr->nvmecss;
365 state->vss = hdr->vss;
366 state->nvme_cs = hdr->data;
367 state->vsd = hdr->data + le16_to_cpu(hdr->nvmecss) * 4;
368
369 return ret;
370
371 free_big:
372 kvfree(migf->vf_data);
373 return ret;
374 }
375

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#10 lore raw

Re: [RFC PATCH 4/4] vfio-nvme: implement TP4159 live migration cmds

In reply to Chaitanya Kulkarni
Quoted 40 lines · Chaitanya Kulkarni

Implements TP4159-based live migration support in vfio-nvme driver by integrating command execution, controller state handling, and vfio migration state transitions.

Key features:

- Use nvme_submit_vf_cmd() and nvme_get_ctrl_id() helpers in the NVMe core PCI driver for submitting admin commands on VFs.

- Implements Migration Send (opcode 0x43) and Receive (opcode 0x42) command handling for suspend, resume, get/set controller state.

_Remark_:-
We are currently in the process of defining the state in TP4193, so the current state management code will be replaced with TP4193. However, in this patch we include TP4159-compatible state management code for the sake of completeness.

- Adds parsing and serialization of controller state including:
- NVMeCS v0 controller state format (SCS-FIG6, FIG7, FIG8)
- Supported Controller State Formats (CNS=0x20 response)
- Migration file abstraction with read/write fileops

- Adds debug decoders to log IOSQ/IOCQ state during migration save

- Allocates anon inodes to handle save and resume file interfaces exposed via VFIO migration file descriptors

- Adds vfio migration state machine transitions:
- RUNNING → STOP: sends suspend command
- STOP → STOP_COPY: extracts controller state (save)
- STOP_COPY → STOP: disables file and frees buffer
- STOP → RESUMING: allocates resume file buffer
- RESUMING → STOP: loads controller state via set state
- STOP → RUNNING: resumes controller via resume command

- Hooks vfio_migration_ops into vfio_pci_ops using:
- migration_set_state() and migration_get_state()
- Uses state_mutex + reset_lock for proper concurrency

Quoted 5 lines

- Queries Identify Controller (CNS=01h) to check for HMLMS bit in OACS field, indicating controller migration capability

- Applies runtime checks for buffer alignment, format support, and state size bounds to ensure spec compliance

Above mixes different verb forms: "Implements", "Adds", "Allocates", etc. vs "Use". I think there's some trend toward using the imperative, e.g., "Implement", "Add", "Allocate", etc.

Similar for "sends", "extracts", "disables", ...

Quoted 31 lines
drivers/vfio/pci/nvme/nvme.c +31 -0
+static int nvme_lm_get_ctrl_state_fmt(struct pci_dev *dev, bool debug,
+				      struct nvme_lm_ctrl_state_fmts_info *fmt)
+{
+	__u8 i;
+	int ret;
+
+	ret = nvme_lm_id_ctrl_state(dev, fmt);
+	if (ret) {
+		pr_err("Failed to get ctrl state formats (ret=%d)\n", ret);
+		return ret;
+	}
+
+	if (debug)
+		pr_info("NV = %u, NUUID = %u\n", fmt->nv, fmt->nuuid);
+
+	if (debug) {
+		for (i = 0; i < fmt->nv; i++) {
+			pr_info("  Format[%d] Version = 0x%04x\n",
+					i, le16_to_cpu(fmt->vers[i]));
+		}
+
+		for (i = 0; i < fmt->nuuid; i++) {
+			char uuid_str[37]; /* 36 chars + null */
+
+			snprintf(uuid_str, sizeof(uuid_str),
+					"%02x%02x%02x%02x-%02x%02x-%02x%02x-"
+					"%02x%02x-%02x%02x%02x%02x%02x%02x",
+					fmt->uuids[i][0], fmt->uuids[i][1],
+					fmt->uuids[i][2], fmt->uuids[i][3],
+					fmt->uuids[i][4], fmt->uuids[i][5],
+					fmt->uuids[i][6], fmt->uuids[i][7],

drivers/vfio/pci/nvme/nvme.c static struct nvmevf_pci_core_device *nvmevf_drvdata(struct pci_dev *pdev) · line 256

Quoted 10 lines
drivers/vfio/pci/nvme/nvme.c +10 -0
+					fmt->uuids[i][8], fmt->uuids[i][9],
+					fmt->uuids[i][10], fmt->uuids[i][11],
+					fmt->uuids[i][12], fmt->uuids[i][13],
+					fmt->uuids[i][14], fmt->uuids[i][15]);
+
+			pr_info("  UUID[%d] = %s\n", i, uuid_str);
+		}
+	}
+
+	return ret;

I think we know "ret == 0" here; if so, just use "return 0".

drivers/vfio/pci/nvme/nvme.c static struct nvmevf_pci_core_device *nvmevf_drvdata(struct pci_dev *pdev) · line 266

Quoted 10 lines
drivers/vfio/pci/nvme/nvme.c +10 -0
+}
+
+static void nvmevf_init_get_ctrl_state_cmd(struct nvme_command *c, __u16 cntlid,
+					   __u8 csvi, __u8 csuuidi,
+					   __u8 csuidxp, size_t buf_len)
+{
+	c->lm.recv.opcode = nvme_admin_lm_recv;
+	c->lm.recv.sel = NVME_LM_RECV_GET_CTRL_STATE;
+	/*
+	 * MOS fields treated as ctrl state version index, Use NVME V1 state.

s/, Use/; use/ ?

drivers/vfio/pci/nvme/nvme.c static struct nvmevf_pci_core_device *nvmevf_drvdata(struct pci_dev *pdev) · line 271

Quoted 5 lines
drivers/vfio/pci/nvme/nvme.c +5 -0
+	 */
+	/*
+	 * For upstream read the supported controller state formats using
+	 * identify command with cns value 0x20 and make sure NVME_LM_CSVI
+	 * matches the on of the reported formats for NVMe states.

Above you capitalized "CNS".

"matches one of the"? Not sure of your intent.

drivers/vfio/pci/nvme/nvme.c static struct nvmevf_pci_core_device *nvmevf_drvdata(struct pci_dev *pdev) · line 274

Quoted 3 lines
drivers/vfio/pci/nvme/nvme.c +3 -0
+	 */
+	c->lm.recv.mos = cpu_to_le16(csvi);
+	/* Target Controller is this a right way to get the controller ID */

Is this a TODO item, i.e., a question that needs to be resolved?

drivers/vfio/pci/nvme/nvme.c static struct nvmevf_pci_core_device *nvmevf_drvdata(struct pci_dev *pdev) · line 280

Quoted 6 lines
drivers/vfio/pci/nvme/nvme.c +6 -0
+	c->lm.recv.cntlid = cpu_to_le16(cntlid);
+
+	/*
+	 * For upstream read the supported controller state formats using
+	 * identify command with cns value 0x20 and make sure NVME_LM_CSVI
+	 * matches the on of the reported formats for Vender specific states.

Above you capitalized "CNS".

"matches one of the"? Not sure of your intent.

s/Vender/Vendor/

drivers/vfio/pci/nvme/nvme.c static struct nvmevf_pci_core_device *nvmevf_drvdata(struct pci_dev *pdev) · line 290

Quoted 10 lines
drivers/vfio/pci/nvme/nvme.c +10 -0
+	 */
+	/* adjust the state as per needed by setting the macro values */
+	c->lm.recv.csuuidi = cpu_to_le32(csuuidi);
+	c->lm.recv.csuidxp = cpu_to_le32(csuidxp);
+
+	/*
+	 * Associates the Migration Receive command with the correct migration
+	 * session UUID currently we set to 0. For now asssume that initiaor
+	 * and target has agreed on the UUIDX 0 for all the live migration
+	 * sessions.

s/Associates/Associate/
s/asssume/assume/
s/initiaor/initiator/
s/UUIDX/UUID/ ?

drivers/vfio/pci/nvme/nvme.c static struct nvmevf_pci_core_device *nvmevf_drvdata(struct pci_dev *pdev) · line 296

Quoted 6 lines
drivers/vfio/pci/nvme/nvme.c +6 -0
+	 */
+	c->lm.recv.uuid_index = cpu_to_le32(0);
+
+	/*
+	 * Assume that data buffer is big enoough to hold the state,
+	 * 0-based dword count.

s/enoough/enough/

Quoted 63 lines
drivers/vfio/pci/nvme/nvme.c +63 -0
+	 */
+	c->lm.recv.numd = cpu_to_le32(bytes_to_nvme_numd(buf_len));
+}
+
+#define NVME_LM_MAX_NVMECS	1024
+#define NVME_LM_MAX_VSD		1024
+
+static int nvmevf_get_ctrl_state(struct pci_dev *dev,
+				__u8 csvi, __u8 csuuidi, __u8 csuidxp,
+				struct nvmevf_migration_file *migf,
+				struct nvme_lm_ctrl_state_info *state)
+{
+	struct nvme_command c = { };
+	struct nvme_lm_ctrl_state *hdr;
+	/* Make sure hdr_len is a multiple of 4 */
+	size_t hdr_len = ALIGN(sizeof(*hdr), 4);
+	__u16 id = nvme_get_ctrl_id(dev);
+	void *local_buf;
+	size_t len;
+	int ret;
+
+	/* Step 1: Issue Migration Receive (Select = 0) to get header */
+	local_buf = kzalloc(hdr_len, GFP_KERNEL);
+	if (!local_buf)
+		return -ENOMEM;
+
+	nvmevf_init_get_ctrl_state_cmd(&c, id, csvi, csuuidi, csuidxp, hdr_len);
+	ret = nvme_submit_vf_cmd(dev, &c, NULL, local_buf, hdr_len);
+	if (ret) {
+		dev_warn(&dev->dev,
+			"nvme_admin_lm_recv failed (ret=0x%x)\n", ret);
+		kfree(local_buf);
+		return ret;
+	}
+
+	if (le16_to_cpu(hdr->nvmecss) > NVME_LM_MAX_NVMECS ||
+	    le16_to_cpu(hdr->vss) > NVME_LM_MAX_VSD) {
+		kfree(local_buf);
+		return -EINVAL;
+	}
+
+	hdr = local_buf;
+	len = hdr_len + 4 * (le16_to_cpu(hdr->nvmecss) + le16_to_cpu(hdr->vss));
+
+	kfree(local_buf);
+
+	if (len == hdr_len)
+		dev_warn(&dev->dev, "nvmecss == 0 or vss = 0\n");
+
+	/* Step 2: Allocate full buffer */
+	migf->total_length = len;
+	migf->vf_data = kvzalloc(migf->total_length, GFP_KERNEL);
+	if (!migf->vf_data)
+		return -ENOMEM;
+
+	memset(&c, 0, sizeof(c));
+	nvmevf_init_get_ctrl_state_cmd(&c, id, csvi, csuuidi, csuidxp, len);
+	ret = nvme_submit_vf_cmd(dev, &c, NULL, migf->vf_data, len);
+	if (ret)
+		goto free_big;
+
+	/* Populate state struct */
+	hdr = (struct nvme_lm_ctrl_state *)migf->vf_data;

drivers/vfio/pci/nvme/nvme.c static struct nvmevf_pci_core_device *nvmevf_drvdata(struct pci_dev *pdev) · line 369

Quoted 10 lines
drivers/vfio/pci/nvme/nvme.c +10 -0
+	state->raw = hdr;
+	state->total_len = len;
+	state->version = hdr->version;
+	state->csattr = hdr->csattr;
+	state->nvmecss = hdr->nvmecss;
+	state->vss = hdr->vss;
+	state->nvme_cs = hdr->data;
+	state->vsd = hdr->data + le16_to_cpu(hdr->nvmecss) * 4;
+
+	return ret;

I think we know "ret == 0" here; if so, just use "return 0".

Quoted 8 lines
drivers/vfio/pci/nvme/nvme.c +7 -0
+free_big:
+	kvfree(migf->vf_data);
+	return ret;
+}
 
+static void nvme_lm_debug_ctrl_state(struct nvme_lm_ctrl_state_info *state)
+{
+	const struct nvme_lm_nvme_cs_v0_state *cs;

drivers/vfio/pci/nvme/nvme.c static struct nvmevf_pci_core_device *nvmevf_drvdata(struct pci_dev *pdev) · line 418

Quoted 10 lines
drivers/vfio/pci/nvme/nvme.c +10 -0
+	const struct nvme_lm_iosq_state *iosq;
+	const struct nvme_lm_iocq_state *iocq;
+	u16 niosq, niocq;
+	int i;
+
+	pr_info("Controller State:\n");
+	pr_info("Version    : 0x%04x\n", le16_to_cpu(state->version));
+	pr_info("CSATTR     : 0x%02x\n", state->csattr);
+	pr_info("NVMECS Len : %u bytes\n", le16_to_cpu(state->nvmecss) * 4);
+	pr_info("VSD Len    : %u bytes\n", le16_to_cpu(state->vss) * 4);

I always wish messages like this were associated with a device, e.g., some flavor of dev_info().

Quoted 38 lines
drivers/vfio/pci/nvme/nvme.c +37 -0
+
+	cs = nvme_lm_parse_nvme_cs_v0_state(state->nvme_cs,
+					    le16_to_cpu(state->nvmecss) * 4,
+					    &niosq, &niocq);
+	if (!cs) {
+		pr_warn("Failed to parse NVMECS\n");
+		return;
+	}
+
+	iosq = cs->iosq;
+	iocq = (const void *)(iosq + niosq);
+
+	for (i = 0; i < niosq; i++) {
+		pr_info("IOSQ[%d]: SIZE=%u QID=%u CQID=%u ATTR=0x%x Head=%u "
+			"Tail=%u\n", i,
+			le16_to_cpu(iosq[i].qsize),
+			le16_to_cpu(iosq[i].qid),
+			le16_to_cpu(iosq[i].cqid),
+			le16_to_cpu(iosq[i].attr),
+			le16_to_cpu(iosq[i].head),
+			le16_to_cpu(iosq[i].tail));
+	}
+
+	for (i = 0; i < niocq; i++) {
+		pr_info("IOCQ[%d]: SIZE=%u QID=%u ATTR=%u Head=%u Tail=%u\n", i,
+			le16_to_cpu(iocq[i].qsize),
+			le16_to_cpu(iocq[i].qid),
+			le16_to_cpu(iocq[i].attr),
+			le16_to_cpu(iocq[i].head),
+			le16_to_cpu(iocq[i].tail));
+	}
+}
 
+static int nvmevf_cmd_get_ctrl_state(struct nvmevf_pci_core_device *nvmevf_dev,
+				     struct nvmevf_migration_file *migf)
+{
+	struct pci_dev *dev = nvmevf_dev->core_device.pdev;
+	struct nvme_lm_ctrl_state_fmts_info fmt = { };

drivers/vfio/pci/nvme/nvme.c static struct nvmevf_pci_core_device *nvmevf_drvdata(struct pci_dev *pdev) · line 469

Quoted 10 lines
drivers/vfio/pci/nvme/nvme.c +10 -0
+	struct nvme_lm_ctrl_state_info state = { };
+	__u8 csvi = NVME_LM_CSVI;
+	__u8 csuuidi = NVME_LM_CSUUIDI;
+	__u8 csuidxp = 0;
+	int ret;
+
+	/*
+	 * Read the supported controller state formats to make sure they match
+	 * csvi value specified in vfio-nvme without this check we'd not know
+	 * which controller state format we are working with.

Run-on sentence.

drivers/vfio/pci/nvme/nvme.c static struct nvmevf_pci_core_device *nvmevf_drvdata(struct pci_dev *pdev) · line 479

Quoted 10 lines
drivers/vfio/pci/nvme/nvme.c +10 -0
+	 */
+	ret = nvme_lm_get_ctrl_state_fmt(dev, true, &fmt);
+	if (ret)
+		return ret;
+	/*
+	 * Number of versions NV cannot be less than controller state version
+	 * index we are using, it's an error. Please note that CSVI is
+	 * a configurable value user can define this macro at the compile time
+	 * to select the required NVMe controller state version index from
+	 * Supported Controller State Formats Data Structure.

Capitalize "CSVI" (or "csvi") consistently. It's lower-case above and upper-case here. Also another run-on sentence.

Quoted 51 lines
drivers/vfio/pci/nvme/nvme.c +51 -0
+	 */
+	if (fmt.nv < csvi) {
+		dev_warn(&dev->dev,
+			 "required ctrl state format not found\n");
+		ret = -EINVAL;
+		goto out;
+	}
+
+	ret = nvmevf_get_ctrl_state(dev, csvi, csuuidi, csuidxp, migf, &state);
+	if (ret)
+		goto out;
+
+	if (le16_to_cpu(state.version) != csvi) {
+		dev_warn(&dev->dev,
+			 "Unexpected controller state version: 0x%04x\n",
+			 le16_to_cpu(state.version));
+		ret = -EINVAL;
+		goto out;
+	}
+
+	/*
+	 * Now that we have received the controller state decode the state
+	 * properly for debugging purpose
+	 */
+
+	nvme_lm_debug_ctrl_state(&state);
+
+	dev_info(&dev->dev, "Get controller state successful\n");
+
+out:
+	kfree(fmt.ctrl_state_raw_buf);
+	return ret;
+}
+
+static int nvmevf_cmd_set_ctrl_state(struct nvmevf_pci_core_device *nvmevf_dev,
+				     struct nvmevf_migration_file *migf)
+{
+	struct pci_dev *dev = nvmevf_dev->core_device.pdev;
+	struct nvme_command c = { };
+	u32 sel = NVME_LM_SEND_SEL_SET_CTRL_STATE;
+	/* assume that data buffer is big enough to hold state in one cmd */
+	u32 mos = NVME_LM_SEQIND_ONLY;
+	u32 cntlid = nvme_get_ctrl_id(dev);
+	u32 csvi = NVME_LM_CSVI;
+	u32 csuuidi = NVME_LM_CSUUIDI;
+	int ret;
+
+	c.lm.send.opcode = nvme_admin_lm_send;
+	/* mos = SEQIND = 0b11 (Only) in MOS bits [17:16] */
+	c.lm.send.cdw10 = cpu_to_le32((mos << 16) | sel);
+	/*

drivers/vfio/pci/nvme/nvme.c static struct nvmevf_pci_core_device *nvmevf_drvdata(struct pci_dev *pdev) · line 540

Quoted 10 lines
drivers/vfio/pci/nvme/nvme.c +10 -0
+	 * Assume that we are only working on NVMe state and not on vendor
+	 * specific state.
+	 */
+	c.lm.send.cdw11 = cpu_to_le32(csuuidi << 24 | csvi << 16 | cntlid);
+
+	/*
+	 * Associates the Migration Send command with the correct migration
+	 * session UUID currently we set to 0. For now asssume that initiaor
+	 * and target has agreed on the UUIDX 0 for all the live migration
+	 * sessions.

s/Associates/Associate/
s/asssume/assume/
s/initiaor/initiator/
s/UUIDX/UUID/ ?

drivers/vfio/pci/nvme/nvme.c static struct nvmevf_pci_core_device *nvmevf_drvdata(struct pci_dev *pdev) · line 545

Quoted 5 lines
drivers/vfio/pci/nvme/nvme.c +5 -0
+	 */
+	c.lm.send.cdw14 = cpu_to_le32(0);
+	/*
+	 * Assume that data buffer is big enoough to hold the state,
+	 * 0-based dword count.

s/enoough/enough/

Quoted 27 lines
drivers/vfio/pci/nvme/nvme.c +26 -0
+	 */
+	c.lm.send.cdw15 = cpu_to_le32(bytes_to_nvme_numd(migf->total_length));
+
+	ret = nvme_submit_vf_cmd(dev, &c, NULL, migf->vf_data,
+				 migf->total_length);
+	if (ret) {
+		dev_warn(&dev->dev,
+			 "Load the device states failed (ret=0x%x)\n", ret);
+		return ret;
+	}
+
+	dev_info(&dev->dev, "Set controller state successful\n");
+	return 0;
+}
 
+static struct nvmevf_migration_file *
+nvmevf_pci_resume_device_data(struct nvmevf_pci_core_device *nvmevf_dev)
+{
+	struct nvmevf_migration_file *migf;
+	int ret;
+
+	migf = kzalloc(sizeof(*migf), GFP_KERNEL);
+	if (!migf)
+		return ERR_PTR(-ENOMEM);
+
+	migf->filp = anon_inode_getfile("nvmevf_mig", &nvmevf_resume_fops, migf,
+					O_WRONLY);

drivers/vfio/pci/nvme/nvme.c static struct nvmevf_pci_core_device *nvmevf_drvdata(struct pci_dev *pdev) · line 637

Quoted 10 lines
drivers/vfio/pci/nvme/nvme.c +10 -0
+	if (IS_ERR(migf->filp)) {
+		int err = PTR_ERR(migf->filp);
+
+		kfree(migf);
+		return ERR_PTR(err);
+	}
+	stream_open(migf->filp->f_inode, migf->filp);
+	mutex_init(&migf->lock);
+
+	/* Allocate buffer to load the device states and max states is 256K */

Why repeat the "#define MAX_MIGRATION_SIZE (256 * 1024)" value in the comment?

Quoted 46 lines
drivers/vfio/pci/nvme/nvme.c +45 -0
+	migf->vf_data = kvzalloc(MAX_MIGRATION_SIZE, GFP_KERNEL);
+	if (!migf->vf_data) {
+		ret = -ENOMEM;
+		goto out_free;
+	}
+
+	return migf;
+
+out_free:
+	fput(migf->filp);
+	return ERR_PTR(ret);
+}
 
+static struct file *
+nvmevf_pci_set_device_state(struct vfio_device *vdev,
+			    enum vfio_device_mig_state new_state)
+{
+	struct nvmevf_pci_core_device *nvmevf_dev = container_of(vdev,
+			struct nvmevf_pci_core_device, core_device.vdev);
+	enum vfio_device_mig_state next_state;
+	struct file *res = NULL;
+	int ret;
+
+	mutex_lock(&nvmevf_dev->state_mutex);
+	while (new_state != nvmevf_dev->mig_state) {
+		ret = vfio_mig_get_next_state(vdev, nvmevf_dev->mig_state,
+					      new_state, &next_state);
+		if (ret) {
+			res = ERR_PTR(-EINVAL);
+			break;
+		}
+
+		res = nvmevf_pci_step_device_state_locked(nvmevf_dev,
+							  next_state);
+		if (IS_ERR(res))
+			break;
+		nvmevf_dev->mig_state = next_state;
+		if (WARN_ON(res && new_state != nvmevf_dev->mig_state)) {
+			fput(res);
+			res = ERR_PTR(-EINVAL);
+			break;
+		}
+	}
+	nvmevf_state_mutex_unlock(nvmevf_dev);
+	return res;
+}

drivers/vfio/pci/nvme/nvme.c static struct nvmevf_pci_core_device *nvmevf_drvdata(struct pci_dev *pdev) · line 836

Quoted 10 lines
drivers/vfio/pci/nvme/nvme.c +10 -0
+
+static int nvmevf_pci_get_device_state(struct vfio_device *vdev,
+				       enum vfio_device_mig_state *curr_state)
+{
+	struct nvmevf_pci_core_device *nvmevf_dev = container_of(
+			vdev, struct nvmevf_pci_core_device, core_device.vdev);
+
+	mutex_lock(&nvmevf_dev->state_mutex);
+	*curr_state = nvmevf_dev->mig_state;
+	nvmevf_state_mutex_unlock(nvmevf_dev);

I see that you added nvmevf_state_mutex_unlock() because you want to deal with deferred reset things at the same time as the mutex_unlock(), but it does make it harder to read when we have to know and verify that

mutex_lock(&nvmevf_dev->state_mutex)
nvmevf_state_mutex_unlock(nvmevf_dev)

is actually a lock/unlock of the same mutex. Maybe there's no better way and this is the best we can do.

Quoted 6 lines
drivers/vfio/pci/nvme/nvme.c +5 -0
+	return 0;
+}
 
+static int nvmevf_migration_init_dev(struct vfio_device *core_vdev)
+{
+	struct nvmevf_pci_core_device *nvmevf_dev;

drivers/vfio/pci/nvme/nvme.c static struct nvmevf_pci_core_device *nvmevf_drvdata(struct pci_dev *pdev) · line 888

Quoted 10 lines
drivers/vfio/pci/nvme/nvme.c +10 -0
+	struct pci_dev *pdev;
+	int vf_id;
+	int ret = -1;
+
+	nvmevf_dev = container_of(core_vdev, struct nvmevf_pci_core_device,
+				  core_device.vdev);
+	pdev = to_pci_dev(core_vdev->dev);
+
+	if (!pdev->is_virtfn)
+		return ret;

"ret" seems pointless here since it's always -1. You could just "return -1" directly, although it looks like this is supposed to be a negative errno, e.g., -EINVAL or whatever.

Quoted 25 lines
drivers/vfio/pci/nvme/nvme.c +23 -0
+
+	/*
+	 * Get the identify controller data structure to check the live
+	 * migration support.
+	 */
+	if (!nvmevf_migration_supp(pdev))
+		return ret;
+
+	nvmevf_dev->migrate_cap = 1;
+
+	vf_id = pci_iov_vf_id(pdev);
+	if (vf_id < 0)
+		return ret;
+	nvmevf_dev->vf_id = vf_id + 1;
+	core_vdev->migration_flags = VFIO_MIGRATION_STOP_COPY;
+
+	mutex_init(&nvmevf_dev->state_mutex);
+	spin_lock_init(&nvmevf_dev->reset_lock);
+	core_vdev->mig_ops = &nvmevf_pci_mig_ops;
+
+	return vfio_pci_core_init_dev(core_vdev);
+}
 
+++ b/drivers/vfio/pci/nvme/nvme.h

drivers/vfio/pci/nvme/nvme.c struct nvmevf_pci_core_device { · line 38

Quoted 6 lines
drivers/vfio/pci/nvme/nvme.c +3 -0
@@ -33,4 +33,7 @@ struct nvmevf_pci_core_device {
 	struct nvmevf_migration_file *saving_migf;
 };
 
+extern int nvme_submit_vf_cmd(struct pci_dev *dev, struct nvme_command *cmd,
+			size_t *result, void *buffer, unsigned int bufflen);
+extern u16 nvme_get_ctrl_id(struct pci_dev *dev);

Typical style in drivers/vfio/ omits the "extern" on function declarations.

Quoted 4 lines
drivers/vfio/pci/nvme/nvme.h +0 -0
 #endif /* NVME_VFIO_PCI_H */
Signature · 1 line
2.40.0