Skip to contentSkip to conversation

crypto: qat - enable QAT GEN4 SRIOV VF live migration for QAT GEN4

PATCH Superseded Xin Zeng 3 participants 17 messages Feb 28, 2024 – Mar 6, 2024

Subsystems (6)

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Xin Zeng

author
#0 lore raw

v4 cover crypto: qat - enable QAT GEN4 SRIOV VF live migration for QAT GEN4

This set enables live migration for Intel QAT GEN4 SRIOV Virtual Functions (VFs).
It is composed of 10 patches. Patch 1~6 refactor the original QAT PF driver implementation which will be reused by the following patches. Patch 7 introduces the logic to the QAT PF driver that allows to save and restore the state of a bank (a QAT VF is a wrapper around banks) and drain a ring pair. Patch 8 adds the QAT PF driver a set of interfaces to allow to save and restore the state of a VF that will be called by the module qat_vfio_pci which will be introduced in the last patch. Patch 9 implements the defined device interfaces. The last one adds a vfio pci extension specific for QAT which intercepts the vfio device operations for a QAT VF to allow live migration.

Here are the steps required to test the live migration of a QAT GEN4 VF:
1. Bind one or more QAT GEN4 VF devices to the module qat_vfio_pci.ko
2. Assign the VFs to the virtual machine and enable device live migration
3. Run a workload using a QAT VF inside the VM, for example using qatlib
(https://github.com/intel/qatlib)
4. Migrate the VM from the source node to a destination node

Changes in v4 since v3: https://lore.kernel.org/kvm/[email protected]
- Change the order of maintainer entry for QAT vfio pci driver in
MAINTAINERS to make it alphabetical (Alex)
- Put QAT VFIO PCI driver under vfio/pci directly instead of
vfio/pci/intel (Alex)
- Add id_table recheck during device probe (Alex)

Changes in v3 since v2: https://lore.kernel.org/kvm/[email protected]
- Use state_mutex directly instead of unnecessary deferred_reset mode
(Jason)

Changes in v2 since v1: https://lore.kernel.org/all/[email protected]

-  Add VFIO_MIGRATION_PRE_COPY support (Alex)
-  Remove unnecessary module dependancy in Kconfig (Alex)
-  Use direct function calls instead of function pointers in qat vfio

variant driver (Jason)
- Address the comments including uncessary pointer check and kfree, missing lock and direct use of pci_iov_vf_id (Shameer)
- Change CHECK_STAT macro to avoid repeat comparison (Kamlesh)

Changes in v1 since RFC: https://lore.kernel.org/all/[email protected]
- Address comments including the right module dependancy in Kconfig,
source file name and module description (Alex)

-  Added PCI error handler and P2P state handler (Suggested by Kevin)
-  Refactor the state check duing loading ring state (Kevin) 
-  Fix missed call to vfio_put_device in the error case (Breet)
-  Migrate the shadow states in PF driver
-  Rebase on top of 6.8-rc1

Giovanni Cabiddu (2):
crypto: qat - adf_get_etr_base() helper
crypto: qat - relocate CSR access code

Siming Wan (3):
crypto: qat - rename get_sla_arr_of_type()
crypto: qat - expand CSR operations for QAT GEN4 devices
crypto: qat - add bank save and restore flows

Xin Zeng (5):
crypto: qat - relocate and rename 4xxx PF2VM definitions
crypto: qat - move PFVF compat checker to a function
crypto: qat - add interface for live migration
crypto: qat - implement interface for live migration vfio/qat: Add vfio_pci driver for Intel QAT VF devices

 MAINTAINERS                                   |    8 +
 .../intel/qat/qat_420xx/adf_420xx_hw_data.c   |    3 +
 .../intel/qat/qat_4xxx/adf_4xxx_hw_data.c     |    5 +
 .../intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c   |    1 +
 .../qat/qat_c3xxxvf/adf_c3xxxvf_hw_data.c     |    1 +
 .../intel/qat/qat_c62x/adf_c62x_hw_data.c     |    1 +
 .../intel/qat/qat_c62xvf/adf_c62xvf_hw_data.c |    1 +
 drivers/crypto/intel/qat/qat_common/Makefile  |    6 +-
 .../intel/qat/qat_common/adf_accel_devices.h  |   88 ++
 .../intel/qat/qat_common/adf_common_drv.h     |   10 +
 .../qat/qat_common/adf_gen2_hw_csr_data.c     |  101 ++
 .../qat/qat_common/adf_gen2_hw_csr_data.h     |   86 ++
 .../intel/qat/qat_common/adf_gen2_hw_data.c   |   97 --
 .../intel/qat/qat_common/adf_gen2_hw_data.h   |   76 --
 .../qat/qat_common/adf_gen4_hw_csr_data.c     |  231 ++++
 .../qat/qat_common/adf_gen4_hw_csr_data.h     |  188 +++
 .../intel/qat/qat_common/adf_gen4_hw_data.c   |  380 +++++--
 .../intel/qat/qat_common/adf_gen4_hw_data.h   |  127 +--
 .../intel/qat/qat_common/adf_gen4_pfvf.c      |    8 +-
 .../intel/qat/qat_common/adf_gen4_vf_mig.c    | 1010 +++++++++++++++++
 .../intel/qat/qat_common/adf_gen4_vf_mig.h    |   10 +
 .../intel/qat/qat_common/adf_mstate_mgr.c     |  318 ++++++
 .../intel/qat/qat_common/adf_mstate_mgr.h     |   89 ++
 .../intel/qat/qat_common/adf_pfvf_pf_proto.c  |    8 +-
 .../intel/qat/qat_common/adf_pfvf_utils.h     |   11 +
 drivers/crypto/intel/qat/qat_common/adf_rl.c  |   10 +-
 drivers/crypto/intel/qat/qat_common/adf_rl.h  |    2 +
 .../crypto/intel/qat/qat_common/adf_sriov.c   |    7 +-
 .../intel/qat/qat_common/adf_transport.c      |    4 +-
 .../crypto/intel/qat/qat_common/qat_mig_dev.c |  130 +++
 .../qat/qat_dh895xcc/adf_dh895xcc_hw_data.c   |    1 +
 .../qat_dh895xccvf/adf_dh895xccvf_hw_data.c   |    1 +
 drivers/vfio/pci/Kconfig                      |    2 +
 drivers/vfio/pci/Makefile                     |    2 +
 drivers/vfio/pci/qat/Kconfig                  |   12 +
 drivers/vfio/pci/qat/Makefile                 |    3 +
 drivers/vfio/pci/qat/main.c                   |  666 +++++++++++
 include/linux/qat/qat_mig_dev.h               |   31 +
 38 files changed, 3348 insertions(+), 387 deletions(-)
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen2_hw_csr_data.c
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen2_hw_csr_data.h
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_hw_csr_data.c
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_hw_csr_data.h
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.c
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.h
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.c
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.h
 create mode 100644 drivers/crypto/intel/qat/qat_common/qat_mig_dev.c
 create mode 100644 drivers/vfio/pci/qat/Kconfig
 create mode 100644 drivers/vfio/pci/qat/Makefile
 create mode 100644 drivers/vfio/pci/qat/main.c
 create mode 100644 include/linux/qat/qat_mig_dev.h

base-commit: 318407ed77e4140d02e43a001b1f4753e3ce6b5f

Signature · 1 line
2.18.2
author
#1 lore raw

PATCH v4 1/10 crypto: qat - adf_get_etr_base() helper

From: Giovanni Cabiddu <[email protected]>

Add and use the new helper function adf_get_etr_base() which retrieves the virtual address of the ring bar.

This will be used extensively when adding support for Live Migration.

---
 drivers/crypto/intel/qat/qat_common/adf_common_drv.h   | 10 ++++++++++
 drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c |  4 +---
 drivers/crypto/intel/qat/qat_common/adf_transport.c    |  4 +---
 3 files changed, 12 insertions(+), 6 deletions(-)
drivers/crypto/intel/qat/qat_common/adf_common_drv.h +10 -0
index 57328249c89e..3bec9e20bad0 100644
@@ -248,6 +248,16 @@ static inline void __iomem *adf_get_pmisc_base(struct adf_accel_dev *accel_dev)
 	return pmisc->virt_addr;
 }
 
+static inline void __iomem *adf_get_etr_base(struct adf_accel_dev *accel_dev)
+{
+	struct adf_hw_device_data *hw_data = accel_dev->hw_device;
+	struct adf_bar *etr;
+
+	etr = &GET_BARS(accel_dev)[hw_data->get_etr_bar_id(hw_data)];
+
+	return etr->virt_addr;
+}
+
 static inline void __iomem *adf_get_aram_base(struct adf_accel_dev *accel_dev)
 {
 	struct adf_hw_device_data *hw_data = accel_dev->hw_device;
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +1 -3
index d28e1921940a..b8a6d24f791f 100644
@@ -321,8 +321,7 @@ static int reset_ring_pair(void __iomem *csr, u32 bank_number)
 int adf_gen4_ring_pair_reset(struct adf_accel_dev *accel_dev, u32 bank_number)
 {
 	struct adf_hw_device_data *hw_data = accel_dev->hw_device;
-	u32 etr_bar_id = hw_data->get_etr_bar_id(hw_data);
-	void __iomem *csr;
+	void __iomem *csr = adf_get_etr_base(accel_dev);
 	int ret;
 
 	if (bank_number >= hw_data->num_banks)
@@ -331,7 +330,6 @@ int adf_gen4_ring_pair_reset(struct adf_accel_dev *accel_dev, u32 bank_number)
 	dev_dbg(&GET_DEV(accel_dev),
 		"ring pair reset for bank:%d\n", bank_number);
 
-	csr = (&GET_BARS(accel_dev)[etr_bar_id])->virt_addr;
 	ret = reset_ring_pair(csr, bank_number);
 	if (ret)
 		dev_err(&GET_DEV(accel_dev),
drivers/crypto/intel/qat/qat_common/adf_transport.c +1 -3
index 630d0483c4e0..1efdf46490f1 100644
@@ -474,7 +474,6 @@ static int adf_init_bank(struct adf_accel_dev *accel_dev,
 int adf_init_etr_data(struct adf_accel_dev *accel_dev)
 {
 	struct adf_etr_data *etr_data;
-	struct adf_hw_device_data *hw_data = accel_dev->hw_device;
 	void __iomem *csr_addr;
 	u32 size;
 	u32 num_banks = 0;
@@ -495,8 +494,7 @@ int adf_init_etr_data(struct adf_accel_dev *accel_dev)
 	}
 
 	accel_dev->transport = etr_data;
-	i = hw_data->get_etr_bar_id(hw_data);
-	csr_addr = accel_dev->accel_pci_dev.pci_bars[i].virt_addr;
+	csr_addr = adf_get_etr_base(accel_dev);
 
 	/* accel_dev->debugfs_dir should always be non-NULL here */
 	etr_data->debug = debugfs_create_dir("transport",
Signature · 1 line
2.18.2
reviewer
#2 lore raw

RE: [PATCH v4 01/10] crypto: qat - adf_get_etr_base() helper

Quoted 5 lines

From: Zeng, Xin <[email protected]>
Sent: Wednesday, February 28, 2024 10:34 PM

From: Giovanni Cabiddu <[email protected]>

Quoted 8 lines

Add and use the new helper function adf_get_etr_base() which retrieves the virtual address of the ring bar.

This will be used extensively when adding support for Live Migration.

IMHO s-o-b already implies that the author has reviewed his own code...

author
#3 lore raw

PATCH v4 2/10 crypto: qat - relocate and rename 4xxx PF2VM definitions

In reply to Xin Zeng

Move and rename ADF_4XXX_PF2VM_OFFSET and ADF_4XXX_VM2PF_OFFSET to ADF_GEN4_PF2VM_OFFSET and ADF_GEN4_VM2PF_OFFSET respectively. These definitions are moved from adf_gen4_pfvf.c to adf_gen4_hw_data.h as they are specific to GEN4 and not just to qat_4xxx.

This change is made in anticipation of their use in live migration.

This does not introduce any functional change.

---
 drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.h | 4 ++++
 drivers/crypto/intel/qat/qat_common/adf_gen4_pfvf.c    | 8 +++-----
 2 files changed, 7 insertions(+), 5 deletions(-)
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.h +4 -0
index c6e80df5a85a..c153f41162ec 100644
@@ -197,6 +197,10 @@ do { \
 /* Arbiter threads mask with error value */
 #define ADF_GEN4_ENA_THD_MASK_ERROR	GENMASK(ADF_NUM_THREADS_PER_AE, 0)
 
+/* PF2VM communication channel */
+#define ADF_GEN4_PF2VM_OFFSET(i)	(0x40B010 + (i) * 0x20)
+#define ADF_GEN4_VM2PF_OFFSET(i)	(0x40B014 + (i) * 0x20)
+
 void adf_gen4_set_ssm_wdtimer(struct adf_accel_dev *accel_dev);
 
 enum icp_qat_gen4_slice_mask {
drivers/crypto/intel/qat/qat_common/adf_gen4_pfvf.c +3 -5
index 8e8efe93f3ee..21474d402d09 100644
@@ -6,12 +6,10 @@
 #include "adf_accel_devices.h"
 #include "adf_common_drv.h"
 #include "adf_gen4_pfvf.h"
+#include "adf_gen4_hw_data.h"
 #include "adf_pfvf_pf_proto.h"
 #include "adf_pfvf_utils.h"
 
-#define ADF_4XXX_PF2VM_OFFSET(i)	(0x40B010 + ((i) * 0x20))
-#define ADF_4XXX_VM2PF_OFFSET(i)	(0x40B014 + ((i) * 0x20))
-
 /* VF2PF interrupt source registers */
 #define ADF_4XXX_VM2PF_SOU		0x41A180
 #define ADF_4XXX_VM2PF_MSK		0x41A1C0
@@ -29,12 +27,12 @@ static const struct pfvf_csr_format csr_gen4_fmt = {
 
 static u32 adf_gen4_pf_get_pf2vf_offset(u32 i)
 {
-	return ADF_4XXX_PF2VM_OFFSET(i);
+	return ADF_GEN4_PF2VM_OFFSET(i);
 }
 
 static u32 adf_gen4_pf_get_vf2pf_offset(u32 i)
 {
-	return ADF_4XXX_VM2PF_OFFSET(i);
+	return ADF_GEN4_VM2PF_OFFSET(i);
 }
 
 static void adf_gen4_enable_vf2pf_interrupts(void __iomem *pmisc_addr, u32 vf_mask)
Signature · 1 line
2.18.2
author
#4 lore raw

PATCH v4 3/10 crypto: qat - move PFVF compat checker to a function

In reply to Xin Zeng

Move the code that implements VF version compatibility on the PF side to a separate function so that it can be reused when doing VM live migration.

This does not introduce any functional change.

---
 .../crypto/intel/qat/qat_common/adf_pfvf_pf_proto.c   |  8 +-------
 drivers/crypto/intel/qat/qat_common/adf_pfvf_utils.h  | 11 +++++++++++
 2 files changed, 12 insertions(+), 7 deletions(-)
drivers/crypto/intel/qat/qat_common/adf_pfvf_pf_proto.c +1 -7
index 9ab93fbfefde..b9b5e744a3f1 100644
@@ -242,13 +242,7 @@ static int adf_handle_vf2pf_msg(struct adf_accel_dev *accel_dev, u8 vf_nr,
 			"VersionRequest received from VF%d (vers %d) to PF (vers %d)\n",
 			vf_nr, vf_compat_ver, ADF_PFVF_COMPAT_THIS_VERSION);
 
-		if (vf_compat_ver == 0)
-			compat = ADF_PF2VF_VF_INCOMPATIBLE;
-		else if (vf_compat_ver <= ADF_PFVF_COMPAT_THIS_VERSION)
-			compat = ADF_PF2VF_VF_COMPATIBLE;
-		else
-			compat = ADF_PF2VF_VF_COMPAT_UNKNOWN;
-
+		compat = adf_vf_compat_checker(vf_compat_ver);
 		vf_info->vf_compat_ver = vf_compat_ver;
 
 		resp->type = ADF_PF2VF_MSGTYPE_VERSION_RESP;
drivers/crypto/intel/qat/qat_common/adf_pfvf_utils.h +11 -0
index 2be048e2287b..1a044297d873 100644
@@ -28,4 +28,15 @@ u32 adf_pfvf_csr_msg_of(struct adf_accel_dev *accel_dev, struct pfvf_message msg
 struct pfvf_message adf_pfvf_message_of(struct adf_accel_dev *accel_dev, u32 raw_msg,
 					const struct pfvf_csr_format *fmt);
 
+static inline u8 adf_vf_compat_checker(u8 vf_compat_ver)
+{
+	if (vf_compat_ver == 0)
+		return ADF_PF2VF_VF_INCOMPATIBLE;
+
+	if (vf_compat_ver <= ADF_PFVF_COMPAT_THIS_VERSION)
+		return ADF_PF2VF_VF_COMPATIBLE;
+
+	return ADF_PF2VF_VF_COMPAT_UNKNOWN;
+}
+
 #endif /* ADF_PFVF_UTILS_H */
Signature · 1 line
2.18.2
author
#5 lore raw

PATCH v4 4/10 crypto: qat - relocate CSR access code

In reply to Xin Zeng

From: Giovanni Cabiddu <[email protected]>

As the common hw_data files are growing and the adf_hw_csr_ops is going to be extended with new operations, move all logic related to ring CSRs to the newly created adf_gen[2|4]_hw_csr_data.[c|h] files.

This does not introduce any functional change.

---
 .../intel/qat/qat_420xx/adf_420xx_hw_data.c   |   1 +
 .../intel/qat/qat_4xxx/adf_4xxx_hw_data.c     |   1 +
 .../intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c   |   1 +
 .../qat/qat_c3xxxvf/adf_c3xxxvf_hw_data.c     |   1 +
 .../intel/qat/qat_c62x/adf_c62x_hw_data.c     |   1 +
 .../intel/qat/qat_c62xvf/adf_c62xvf_hw_data.c |   1 +
 drivers/crypto/intel/qat/qat_common/Makefile  |   2 +
 .../qat/qat_common/adf_gen2_hw_csr_data.c     | 101 ++++++++++++++++++
 .../qat/qat_common/adf_gen2_hw_csr_data.h     |  86 +++++++++++++++
 .../intel/qat/qat_common/adf_gen2_hw_data.c   |  97 -----------------
 .../intel/qat/qat_common/adf_gen2_hw_data.h   |  76 -------------
 .../qat/qat_common/adf_gen4_hw_csr_data.c     | 101 ++++++++++++++++++
 .../qat/qat_common/adf_gen4_hw_csr_data.h     |  97 +++++++++++++++++
 .../intel/qat/qat_common/adf_gen4_hw_data.c   |  97 -----------------
 .../intel/qat/qat_common/adf_gen4_hw_data.h   |  94 +---------------
 .../qat/qat_dh895xcc/adf_dh895xcc_hw_data.c   |   1 +
 .../qat_dh895xccvf/adf_dh895xccvf_hw_data.c   |   1 +
 17 files changed, 397 insertions(+), 362 deletions(-)
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen2_hw_csr_data.c
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen2_hw_csr_data.h
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_hw_csr_data.c
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_hw_csr_data.h
drivers/crypto/intel/qat/qat_420xx/adf_420xx_hw_data.c +1 -0
index 1102c47f8293..9ccbf5998d5c 100644
@@ -10,6 +10,7 @@
 #include <adf_fw_config.h>
 #include <adf_gen4_config.h>
 #include <adf_gen4_dc.h>
+#include <adf_gen4_hw_csr_data.h>
 #include <adf_gen4_hw_data.h>
 #include <adf_gen4_pfvf.h>
 #include <adf_gen4_pm.h>
drivers/crypto/intel/qat/qat_4xxx/adf_4xxx_hw_data.c +1 -0
index 927506cf271d..ef4b0aa36603 100644
@@ -10,6 +10,7 @@
 #include <adf_fw_config.h>
 #include <adf_gen4_config.h>
 #include <adf_gen4_dc.h>
+#include <adf_gen4_hw_csr_data.h>
 #include <adf_gen4_hw_data.h>
 #include <adf_gen4_pfvf.h>
 #include <adf_gen4_pm.h>
drivers/crypto/intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c +1 -0
index a882e0ea2279..201f9412c582 100644
@@ -6,6 +6,7 @@
 #include <adf_common_drv.h>
 #include <adf_gen2_config.h>
 #include <adf_gen2_dc.h>
+#include <adf_gen2_hw_csr_data.h>
 #include <adf_gen2_hw_data.h>
 #include <adf_gen2_pfvf.h>
 #include "adf_c3xxx_hw_data.h"
drivers/crypto/intel/qat/qat_c3xxxvf/adf_c3xxxvf_hw_data.c +1 -0
index 84d9486e04de..a512ca4efd3f 100644
@@ -4,6 +4,7 @@
 #include <adf_common_drv.h>
 #include <adf_gen2_config.h>
 #include <adf_gen2_dc.h>
+#include <adf_gen2_hw_csr_data.h>
 #include <adf_gen2_hw_data.h>
 #include <adf_gen2_pfvf.h>
 #include <adf_pfvf_vf_msg.h>
drivers/crypto/intel/qat/qat_c62x/adf_c62x_hw_data.c +1 -0
index 48cf3eb7c734..6b5b0cf9c7c7 100644
@@ -6,6 +6,7 @@
 #include <adf_common_drv.h>
 #include <adf_gen2_config.h>
 #include <adf_gen2_dc.h>
+#include <adf_gen2_hw_csr_data.h>
 #include <adf_gen2_hw_data.h>
 #include <adf_gen2_pfvf.h>
 #include "adf_c62x_hw_data.h"
drivers/crypto/intel/qat/qat_c62xvf/adf_c62xvf_hw_data.c +1 -0
index 751d7aa57fc7..4aaaaf921734 100644
@@ -4,6 +4,7 @@
 #include <adf_common_drv.h>
 #include <adf_gen2_config.h>
 #include <adf_gen2_dc.h>
+#include <adf_gen2_hw_csr_data.h>
 #include <adf_gen2_hw_data.h>
 #include <adf_gen2_pfvf.h>
 #include <adf_pfvf_vf_msg.h>
drivers/crypto/intel/qat/qat_common/Makefile +2 -0
index 5915cde8a7aa..ceaa685352ed 100644
@@ -14,9 +14,11 @@ intel_qat-objs := adf_cfg.o \
 	adf_hw_arbiter.o \
 	adf_sysfs.o \
 	adf_sysfs_ras_counters.o \
+	adf_gen2_hw_csr_data.o \
 	adf_gen2_hw_data.o \
 	adf_gen2_config.o \
 	adf_gen4_config.o \
+	adf_gen4_hw_csr_data.o \
 	adf_gen4_hw_data.o \
 	adf_gen4_pm.o \
 	adf_gen2_dc.o \
drivers/crypto/intel/qat/qat_common/adf_gen2_hw_csr_data.c new file +101 -0

101 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/adf_gen2_hw_csr_data.h new file +86 -0

86 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/adf_gen2_hw_data.c +0 -97

103 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/adf_gen2_hw_data.h +0 -76

97 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_csr_data.c new file +101 -0

101 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_csr_data.h new file +97 -0

97 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +0 -97

103 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.h +2 -92

111 lines not on this page · raw message

drivers/crypto/intel/qat/qat_dh895xcc/adf_dh895xcc_hw_data.c +1 -0
index af14090cc4be..6e24d57e6b98 100644
@@ -5,6 +5,7 @@
 #include <adf_common_drv.h>
 #include <adf_gen2_config.h>
 #include <adf_gen2_dc.h>
+#include <adf_gen2_hw_csr_data.h>
 #include <adf_gen2_hw_data.h>
 #include <adf_gen2_pfvf.h>
 #include "adf_dh895xcc_hw_data.h"
drivers/crypto/intel/qat/qat_dh895xccvf/adf_dh895xccvf_hw_data.c +1 -0
index 70e56cc16ece..f4ee4c2e00da 100644
@@ -4,6 +4,7 @@
 #include <adf_common_drv.h>
 #include <adf_gen2_config.h>
 #include <adf_gen2_dc.h>
+#include <adf_gen2_hw_csr_data.h>
 #include <adf_gen2_hw_data.h>
 #include <adf_gen2_pfvf.h>
 #include <adf_pfvf_vf_msg.h>
Signature · 1 line
2.18.2
author
#6 lore raw

PATCH v4 5/10 crypto: qat - rename get_sla_arr_of_type()

In reply to Xin Zeng

From: Siming Wan <[email protected]>

The function get_sla_arr_of_type() returns a pointer to an SLA type specific array.
Rename it and expose it as it will be used externally to this module.

This does not introduce any functional change.

---
 drivers/crypto/intel/qat/qat_common/adf_rl.c | 10 +++++-----
 drivers/crypto/intel/qat/qat_common/adf_rl.h |  2 ++
 2 files changed, 7 insertions(+), 5 deletions(-)
drivers/crypto/intel/qat/qat_common/adf_rl.c +5 -5
index d4f2db3c53d8..65f752f4792a 100644
@@ -183,14 +183,14 @@ static enum adf_cfg_service_type srv_to_cfg_svc_type(enum adf_base_services rl_s
 }
 
 /**
- * get_sla_arr_of_type() - Returns a pointer to SLA type specific array
+ * adf_rl_get_sla_arr_of_type() - Returns a pointer to SLA type specific array
  * @rl_data: pointer to ratelimiting data
  * @type: SLA type
  * @sla_arr: pointer to variable where requested pointer will be stored
  *
  * Return: Max number of elements allowed for the returned array
  */
-static u32 get_sla_arr_of_type(struct adf_rl *rl_data, enum rl_node_type type,
+u32 adf_rl_get_sla_arr_of_type(struct adf_rl *rl_data, enum rl_node_type type,
 			       struct rl_sla ***sla_arr)
 {
 	switch (type) {
@@ -778,7 +778,7 @@ static void clear_sla(struct adf_rl *rl_data, struct rl_sla *sla)
 		rp_in_use[sla->ring_pairs_ids[i]] = false;
 
 	update_budget(sla, old_cir, true);
-	get_sla_arr_of_type(rl_data, sla->type, &sla_type_arr);
+	adf_rl_get_sla_arr_of_type(rl_data, sla->type, &sla_type_arr);
 	assign_node_to_parent(rl_data->accel_dev, sla, true);
 	adf_rl_send_admin_delete_msg(rl_data->accel_dev, node_id, sla->type);
 	mark_rps_usage(sla, rl_data->rp_in_use, false);
@@ -875,7 +875,7 @@ static int add_update_sla(struct adf_accel_dev *accel_dev,
 
 	if (!is_update) {
 		mark_rps_usage(sla, rl_data->rp_in_use, true);
-		get_sla_arr_of_type(rl_data, sla->type, &sla_type_arr);
+		adf_rl_get_sla_arr_of_type(rl_data, sla->type, &sla_type_arr);
 		sla_type_arr[sla->node_id] = sla;
 		rl_data->sla[sla->sla_id] = sla;
 	}
@@ -1065,7 +1065,7 @@ void adf_rl_remove_sla_all(struct adf_accel_dev *accel_dev, bool incl_default)
 
 	/* Unregister and remove all SLAs */
 	for (j = RL_LEAF; j >= end_type; j--) {
-		max_id = get_sla_arr_of_type(rl_data, j, &sla_type_arr);
+		max_id = adf_rl_get_sla_arr_of_type(rl_data, j, &sla_type_arr);
 
 		for (i = 0; i < max_id; i++) {
 			if (!sla_type_arr[i])
drivers/crypto/intel/qat/qat_common/adf_rl.h +2 -0
index 269c6656fb90..bfe750ea0e83 100644
@@ -151,6 +151,8 @@ struct rl_sla {
 	u16 ring_pairs_cnt;
 };
 
+u32 adf_rl_get_sla_arr_of_type(struct adf_rl *rl_data, enum rl_node_type type,
+			       struct rl_sla ***sla_arr);
 int adf_rl_add_sla(struct adf_accel_dev *accel_dev,
 		   struct adf_rl_sla_input_data *sla_in);
 int adf_rl_update_sla(struct adf_accel_dev *accel_dev,
Signature · 1 line
2.18.2
author
#7 lore raw

PATCH v4 6/10 crypto: qat - expand CSR operations for QAT GEN4 devices

In reply to Xin Zeng

From: Siming Wan <[email protected]>

Extend the CSR operations for QAT GEN4 devices to allow saving and restoring the rings state.

The new operations will be used as a building block for implementing the state save and restore of Virtual Functions necessary for VM live migration.

This adds the following operations:
- read ring status register
- read ring underflow/overflow status register
- read ring nearly empty status register
- read ring nearly full status register
- read ring full status register
- read ring complete status register
- read ring exception status register
- read/write ring exception interrupt mask register
- read ring configuration register
- read ring base register
- read/write ring interrupt enable register
- read ring interrupt flag register
- read/write ring interrupt source select register
- read ring coalesced interrupt enable register
- read ring coalesced interrupt control register
- read ring flag and coalesced interrupt enable register
- read ring service arbiter enable register
- get ring coalesced interrupt control enable mask

---
 .../intel/qat/qat_common/adf_accel_devices.h  |  27 ++++
 .../qat/qat_common/adf_gen4_hw_csr_data.c     | 130 ++++++++++++++++++
 .../qat/qat_common/adf_gen4_hw_csr_data.h     |  93 ++++++++++++-
 3 files changed, 249 insertions(+), 1 deletion(-)
drivers/crypto/intel/qat/qat_common/adf_accel_devices.h +27 -0
index 08658c3a01e9..d1f3f5a822ff 100644
@@ -150,22 +150,49 @@ struct adf_hw_csr_ops {
 				  u32 ring);
 	void (*write_csr_ring_tail)(void __iomem *csr_base_addr, u32 bank,
 				    u32 ring, u32 value);
+	u32 (*read_csr_stat)(void __iomem *csr_base_addr, u32 bank);
+	u32 (*read_csr_uo_stat)(void __iomem *csr_base_addr, u32 bank);
 	u32 (*read_csr_e_stat)(void __iomem *csr_base_addr, u32 bank);
+	u32 (*read_csr_ne_stat)(void __iomem *csr_base_addr, u32 bank);
+	u32 (*read_csr_nf_stat)(void __iomem *csr_base_addr, u32 bank);
+	u32 (*read_csr_f_stat)(void __iomem *csr_base_addr, u32 bank);
+	u32 (*read_csr_c_stat)(void __iomem *csr_base_addr, u32 bank);
+	u32 (*read_csr_exp_stat)(void __iomem *csr_base_addr, u32 bank);
+	u32 (*read_csr_exp_int_en)(void __iomem *csr_base_addr, u32 bank);
+	void (*write_csr_exp_int_en)(void __iomem *csr_base_addr, u32 bank,
+				     u32 value);
+	u32 (*read_csr_ring_config)(void __iomem *csr_base_addr, u32 bank,
+				    u32 ring);
 	void (*write_csr_ring_config)(void __iomem *csr_base_addr, u32 bank,
 				      u32 ring, u32 value);
+	dma_addr_t (*read_csr_ring_base)(void __iomem *csr_base_addr, u32 bank,
+					 u32 ring);
 	void (*write_csr_ring_base)(void __iomem *csr_base_addr, u32 bank,
 				    u32 ring, dma_addr_t addr);
+	u32 (*read_csr_int_en)(void __iomem *csr_base_addr, u32 bank);
+	void (*write_csr_int_en)(void __iomem *csr_base_addr, u32 bank,
+				 u32 value);
+	u32 (*read_csr_int_flag)(void __iomem *csr_base_addr, u32 bank);
 	void (*write_csr_int_flag)(void __iomem *csr_base_addr, u32 bank,
 				   u32 value);
+	u32 (*read_csr_int_srcsel)(void __iomem *csr_base_addr, u32 bank);
 	void (*write_csr_int_srcsel)(void __iomem *csr_base_addr, u32 bank);
+	void (*write_csr_int_srcsel_w_val)(void __iomem *csr_base_addr,
+					   u32 bank, u32 value);
+	u32 (*read_csr_int_col_en)(void __iomem *csr_base_addr, u32 bank);
 	void (*write_csr_int_col_en)(void __iomem *csr_base_addr, u32 bank,
 				     u32 value);
+	u32 (*read_csr_int_col_ctl)(void __iomem *csr_base_addr, u32 bank);
 	void (*write_csr_int_col_ctl)(void __iomem *csr_base_addr, u32 bank,
 				      u32 value);
+	u32 (*read_csr_int_flag_and_col)(void __iomem *csr_base_addr,
+					 u32 bank);
 	void (*write_csr_int_flag_and_col)(void __iomem *csr_base_addr,
 					   u32 bank, u32 value);
+	u32 (*read_csr_ring_srv_arb_en)(void __iomem *csr_base_addr, u32 bank);
 	void (*write_csr_ring_srv_arb_en)(void __iomem *csr_base_addr, u32 bank,
 					  u32 value);
+	u32 (*get_int_col_ctl_enable_mask)(void);
 };
 
 struct adf_cfg_device_data;
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_csr_data.c +130 -0

201 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_csr_data.h +92 -1

148 lines not on this page · raw message

Signature · 1 line
2.18.2
author
#8 lore raw

PATCH v4 7/10 crypto: qat - add bank save and restore flows

In reply to Xin Zeng

From: Siming Wan <[email protected]>

Add logic to save, restore, quiesce and drain a ring bank for QAT GEN4 devices.
This allows to save and restore the state of a Virtual Function (VF) and will be used to implement VM live migration.

---
 .../intel/qat/qat_4xxx/adf_4xxx_hw_data.c     |   2 +
 .../intel/qat/qat_common/adf_accel_devices.h  |  38 +++
 .../intel/qat/qat_common/adf_gen4_hw_data.c   | 279 ++++++++++++++++++
 .../intel/qat/qat_common/adf_gen4_hw_data.h   |  19 ++
 4 files changed, 338 insertions(+)
drivers/crypto/intel/qat/qat_4xxx/adf_4xxx_hw_data.c +2 -0
index ef4b0aa36603..eaf055e6f938 100644
@@ -455,6 +455,8 @@ void adf_init_hw_data_4xxx(struct adf_hw_device_data *hw_data, u32 dev_id)
 	hw_data->get_ring_to_svc_map = adf_gen4_get_ring_to_svc_map;
 	hw_data->disable_iov = adf_disable_sriov;
 	hw_data->ring_pair_reset = adf_gen4_ring_pair_reset;
+	hw_data->bank_state_save = adf_gen4_bank_state_save;
+	hw_data->bank_state_restore = adf_gen4_bank_state_restore;
 	hw_data->enable_pm = adf_gen4_enable_pm;
 	hw_data->handle_pm_interrupt = adf_gen4_handle_pm_interrupt;
 	hw_data->dev_config = adf_gen4_dev_config;
drivers/crypto/intel/qat/qat_common/adf_accel_devices.h +38 -0
index d1f3f5a822ff..986e63ec702d 100644
@@ -140,6 +140,40 @@ struct admin_info {
 	u32 mailbox_offset;
 };
 
+struct ring_config {
+	u64 base;
+	u32 config;
+	u32 head;
+	u32 tail;
+	u32 reserved0;
+};
+
+struct bank_state {
+	u32 ringstat0;
+	u32 ringstat1;
+	u32 ringuostat;
+	u32 ringestat;
+	u32 ringnestat;
+	u32 ringnfstat;
+	u32 ringfstat;
+	u32 ringcstat0;
+	u32 ringcstat1;
+	u32 ringcstat2;
+	u32 ringcstat3;
+	u32 iaintflagen;
+	u32 iaintflagreg;
+	u32 iaintflagsrcsel0;
+	u32 iaintflagsrcsel1;
+	u32 iaintcolen;
+	u32 iaintcolctl;
+	u32 iaintflagandcolen;
+	u32 ringexpstat;
+	u32 ringexpintenable;
+	u32 ringsrvarben;
+	u32 reserved0;
+	struct ring_config rings[ADF_ETR_MAX_RINGS_PER_BANK];
+};
+
 struct adf_hw_csr_ops {
 	u64 (*build_csr_ring_base_addr)(dma_addr_t addr, u32 size);
 	u32 (*read_csr_ring_head)(void __iomem *csr_base_addr, u32 bank,
@@ -271,6 +305,10 @@ struct adf_hw_device_data {
 	void (*enable_ints)(struct adf_accel_dev *accel_dev);
 	void (*set_ssm_wdtimer)(struct adf_accel_dev *accel_dev);
 	int (*ring_pair_reset)(struct adf_accel_dev *accel_dev, u32 bank_nr);
+	int (*bank_state_save)(struct adf_accel_dev *accel_dev, u32 bank_number,
+			       struct bank_state *state);
+	int (*bank_state_restore)(struct adf_accel_dev *accel_dev,
+				  u32 bank_number, struct bank_state *state);
 	void (*reset_device)(struct adf_accel_dev *accel_dev);
 	void (*set_msix_rttable)(struct adf_accel_dev *accel_dev);
 	const char *(*uof_get_name)(struct adf_accel_dev *accel_dev, u32 obj_num);
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +279 -0

288 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.h +19 -0
index 719f7757e587..e8cb930e80c9 100644
@@ -77,10 +77,19 @@
 #define ADF_RPRESET_POLL_TIMEOUT_US	(5 * USEC_PER_SEC)
 #define ADF_RPRESET_POLL_DELAY_US	20
 #define ADF_WQM_CSR_RPRESETCTL_RESET	BIT(0)
+#define ADF_WQM_CSR_RPRESETCTL_DRAIN	BIT(2)
 #define ADF_WQM_CSR_RPRESETCTL(bank)	(0x6000 + ((bank) << 3))
 #define ADF_WQM_CSR_RPRESETSTS_STATUS	BIT(0)
 #define ADF_WQM_CSR_RPRESETSTS(bank)	(ADF_WQM_CSR_RPRESETCTL(bank) + 4)
 
+/* Ring interrupt */
+#define ADF_RP_INT_SRC_SEL_F_RISE_MASK	BIT(2)
+#define ADF_RP_INT_SRC_SEL_F_FALL_MASK	GENMASK(2, 0)
+#define ADF_RP_INT_SRC_SEL_RANGE_WIDTH	4
+#define ADF_COALESCED_POLL_DELAY_US	1000
+#define ADF_WQM_CSR_RPINTSOU(bank)	(0x200000 + ((bank) << 12))
+#define ADF_WQM_CSR_RP_IDX_RX		1
+
 /* Error source registers */
 #define ADF_GEN4_ERRSOU0	(0x41A200)
 #define ADF_GEN4_ERRSOU1	(0x41A204)
@@ -150,5 +159,15 @@ void adf_gen4_set_msix_default_rttable(struct adf_accel_dev *accel_dev);
 void adf_gen4_set_ssm_wdtimer(struct adf_accel_dev *accel_dev);
 int adf_gen4_init_thd2arb_map(struct adf_accel_dev *accel_dev);
 u16 adf_gen4_get_ring_to_svc_map(struct adf_accel_dev *accel_dev);
+int adf_gen4_bank_quiesce_coal_timer(struct adf_accel_dev *accel_dev,
+				     u32 bank_idx, int timeout_ms);
+int adf_gen4_bank_drain_start(struct adf_accel_dev *accel_dev,
+			      u32 bank_number, int timeout_us);
+void adf_gen4_bank_drain_finish(struct adf_accel_dev *accel_dev,
+				u32 bank_number);
+int adf_gen4_bank_state_save(struct adf_accel_dev *accel_dev, u32 bank_number,
+			     struct bank_state *state);
+int adf_gen4_bank_state_restore(struct adf_accel_dev *accel_dev,
+				u32 bank_number, struct bank_state *state);
 
 #endif
Signature · 1 line
2.18.2
author
#9 lore raw

PATCH v4 8/10 crypto: qat - add interface for live migration

In reply to Xin Zeng

Extend the driver with a new interface to be used for VF live migration. This allows to create and destroy a qat_mig_dev object that contains a set of methods to allow to save and restore the state of QAT VF. This interface will be used by the qat-vfio-pci module.

---
 drivers/crypto/intel/qat/qat_common/Makefile  |   2 +-
 .../intel/qat/qat_common/adf_accel_devices.h  |  17 +++
 .../intel/qat/qat_common/adf_gen4_vf_mig.h    |  10 ++
 .../crypto/intel/qat/qat_common/qat_mig_dev.c | 130 ++++++++++++++++++
 include/linux/qat/qat_mig_dev.h               |  31 +++++
 5 files changed, 189 insertions(+), 1 deletion(-)
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.h
 create mode 100644 drivers/crypto/intel/qat/qat_common/qat_mig_dev.c
 create mode 100644 include/linux/qat/qat_mig_dev.h
drivers/crypto/intel/qat/qat_common/Makefile +1 -1
index ceaa685352ed..9fba31d4ac7f 100644
@@ -54,6 +54,6 @@ intel_qat-$(CONFIG_DEBUG_FS) += adf_transport_debug.o \
 intel_qat-$(CONFIG_PCI_IOV) += adf_sriov.o adf_vf_isr.o adf_pfvf_utils.o \
 			       adf_pfvf_pf_msg.o adf_pfvf_pf_proto.o \
 			       adf_pfvf_vf_msg.o adf_pfvf_vf_proto.o \
-			       adf_gen2_pfvf.o adf_gen4_pfvf.o
+			       adf_gen2_pfvf.o adf_gen4_pfvf.o qat_mig_dev.o
 
 intel_qat-$(CONFIG_CRYPTO_DEV_QAT_ERROR_INJECTION) += adf_heartbeat_inject.o
drivers/crypto/intel/qat/qat_common/adf_accel_devices.h +17 -0
index 986e63ec702d..b08fea10121e 100644
@@ -9,6 +9,7 @@
 #include <linux/pci.h>
 #include <linux/ratelimit.h>
 #include <linux/types.h>
+#include <linux/qat/qat_mig_dev.h>
 #include "adf_cfg_common.h"
 #include "adf_rl.h"
 #include "adf_telemetry.h"
@@ -258,6 +259,20 @@ struct adf_dc_ops {
 	void (*build_deflate_ctx)(void *ctx);
 };
 
+struct qat_migdev_ops {
+	int (*init)(struct qat_mig_dev *mdev);
+	void (*cleanup)(struct qat_mig_dev *mdev);
+	void (*reset)(struct qat_mig_dev *mdev);
+	int (*open)(struct qat_mig_dev *mdev);
+	void (*close)(struct qat_mig_dev *mdev);
+	int (*suspend)(struct qat_mig_dev *mdev);
+	int (*resume)(struct qat_mig_dev *mdev);
+	int (*save_state)(struct qat_mig_dev *mdev);
+	int (*save_setup)(struct qat_mig_dev *mdev);
+	int (*load_state)(struct qat_mig_dev *mdev);
+	int (*load_setup)(struct qat_mig_dev *mdev, int size);
+};
+
 struct adf_dev_err_mask {
 	u32 cppagentcmdpar_mask;
 	u32 parerr_ath_cph_mask;
@@ -325,6 +340,7 @@ struct adf_hw_device_data {
 	struct adf_dev_err_mask dev_err_mask;
 	struct adf_rl_hw_data rl_data;
 	struct adf_tl_hw_data tl_data;
+	struct qat_migdev_ops vfmig_ops;
 	const char *fw_name;
 	const char *fw_mmp_name;
 	u32 fuses;
@@ -381,6 +397,7 @@ struct adf_hw_device_data {
 #define GET_CSR_OPS(accel_dev) (&(accel_dev)->hw_device->csr_ops)
 #define GET_PFVF_OPS(accel_dev) (&(accel_dev)->hw_device->pfvf_ops)
 #define GET_DC_OPS(accel_dev) (&(accel_dev)->hw_device->dc_ops)
+#define GET_VFMIG_OPS(accel_dev) (&(accel_dev)->hw_device->vfmig_ops)
 #define GET_TL_DATA(accel_dev) GET_HW_DATA(accel_dev)->tl_data
 #define accel_to_pci_dev(accel_ptr) accel_ptr->accel_pci_dev.pci_dev
 
drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.h new file +10 -0
new file mode 100644
index 000000000000..72216d078ee1
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright(c) 2024 Intel Corporation */
+#ifndef ADF_GEN4_VF_MIG_H_
+#define ADF_GEN4_VF_MIG_H_
+
+#include "adf_accel_devices.h"
+
+void adf_gen4_init_vf_mig_ops(struct qat_migdev_ops *vfmig_ops);
+
+#endif
drivers/crypto/intel/qat/qat_common/qat_mig_dev.c new file +130 -0

130 lines not on this page · raw message

include/linux/qat/qat_mig_dev.h new file +31 -0
new file mode 100644
index 000000000000..dbbb6a063dd2
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright(c) 2024 Intel Corporation */
+#ifndef QAT_MIG_DEV_H_
+#define QAT_MIG_DEV_H_
+
+struct pci_dev;
+
+struct qat_mig_dev {
+	void *parent_accel_dev;
+	u8 *state;
+	u32 setup_size;
+	u32 remote_setup_size;
+	u32 state_size;
+	s32 vf_id;
+};
+
+struct qat_mig_dev *qat_vfmig_create(struct pci_dev *pdev, int vf_id);
+int qat_vfmig_init(struct qat_mig_dev *mdev);
+void qat_vfmig_cleanup(struct qat_mig_dev *mdev);
+void qat_vfmig_reset(struct qat_mig_dev *mdev);
+int qat_vfmig_open(struct qat_mig_dev *mdev);
+void qat_vfmig_close(struct qat_mig_dev *mdev);
+int qat_vfmig_suspend(struct qat_mig_dev *mdev);
+int qat_vfmig_resume(struct qat_mig_dev *mdev);
+int qat_vfmig_save_state(struct qat_mig_dev *mdev);
+int qat_vfmig_save_setup(struct qat_mig_dev *mdev);
+int qat_vfmig_load_state(struct qat_mig_dev *mdev);
+int qat_vfmig_load_setup(struct qat_mig_dev *mdev, int size);
+void qat_vfmig_destroy(struct qat_mig_dev *mdev);
+
+#endif /*QAT_MIG_DEV_H_*/
Signature · 1 line
2.18.2
author
#10 lore raw

PATCH v4 9/10 crypto: qat - implement interface for live migration

In reply to Xin Zeng

Add logic to implement the interface for live migration defined in qat/qat_mig_dev.h. This is specific for QAT GEN4 Virtual Functions (VFs).

This introduces a migration data manager which is used to handle the device state during migration. The manager ensures that the device state is stored in a format that can be restored in the destination node.

The VF state is organized into a hierarchical structure that includes a preamble, a general state section, a MISC bar section and an ETR bar section. The latter contains the state of the 4 ring pairs contained on a VF. Here is a graphical representation of the state:

    preamble | general state section | leaf state
             | MISC bar state section| leaf state
             | ETR bar state section | bank0 state section | leaf state
                                     | bank1 state section | leaf state
                                     | bank2 state section | leaf state
                                     | bank3 state section | leaf state

In addition to the implementation of the qat_migdev_ops interface and the state manager framework, add a mutex in pfvf to avoid pf2vf messages during migration.

---
 .../intel/qat/qat_420xx/adf_420xx_hw_data.c   |    2 +
 .../intel/qat/qat_4xxx/adf_4xxx_hw_data.c     |    2 +
 drivers/crypto/intel/qat/qat_common/Makefile  |    2 +
 .../intel/qat/qat_common/adf_accel_devices.h  |    6 +
 .../intel/qat/qat_common/adf_gen4_hw_data.h   |   10 +
 .../intel/qat/qat_common/adf_gen4_vf_mig.c    | 1010 +++++++++++++++++
 .../intel/qat/qat_common/adf_mstate_mgr.c     |  318 ++++++
 .../intel/qat/qat_common/adf_mstate_mgr.h     |   89 ++
 .../crypto/intel/qat/qat_common/adf_sriov.c   |    7 +-
 9 files changed, 1445 insertions(+), 1 deletion(-)
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.c
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.c
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.h
drivers/crypto/intel/qat/qat_420xx/adf_420xx_hw_data.c +2 -0
index 9ccbf5998d5c..d255cb3ebd9c 100644
@@ -17,6 +17,7 @@
 #include <adf_gen4_ras.h>
 #include <adf_gen4_timer.h>
 #include <adf_gen4_tl.h>
+#include <adf_gen4_vf_mig.h>
 #include "adf_420xx_hw_data.h"
 #include "icp_qat_hw.h"
 
@@ -488,6 +489,7 @@ void adf_init_hw_data_420xx(struct adf_hw_device_data *hw_data, u32 dev_id)
 	adf_gen4_init_dc_ops(&hw_data->dc_ops);
 	adf_gen4_init_ras_ops(&hw_data->ras_ops);
 	adf_gen4_init_tl_data(&hw_data->tl_data);
+	adf_gen4_init_vf_mig_ops(&hw_data->vfmig_ops);
 	adf_init_rl_data(&hw_data->rl_data);
 }
 
drivers/crypto/intel/qat/qat_4xxx/adf_4xxx_hw_data.c +2 -0
index eaf055e6f938..1e77e189a938 100644
@@ -17,6 +17,7 @@
 #include "adf_gen4_ras.h"
 #include <adf_gen4_timer.h>
 #include <adf_gen4_tl.h>
+#include <adf_gen4_vf_mig.h>
 #include "adf_4xxx_hw_data.h"
 #include "icp_qat_hw.h"
 
@@ -472,6 +473,7 @@ void adf_init_hw_data_4xxx(struct adf_hw_device_data *hw_data, u32 dev_id)
 	adf_gen4_init_dc_ops(&hw_data->dc_ops);
 	adf_gen4_init_ras_ops(&hw_data->ras_ops);
 	adf_gen4_init_tl_data(&hw_data->tl_data);
+	adf_gen4_init_vf_mig_ops(&hw_data->vfmig_ops);
 	adf_init_rl_data(&hw_data->rl_data);
 }
 
drivers/crypto/intel/qat/qat_common/Makefile +2 -0
index 9fba31d4ac7f..6f9266edc9f1 100644
@@ -20,12 +20,14 @@ intel_qat-objs := adf_cfg.o \
 	adf_gen4_config.o \
 	adf_gen4_hw_csr_data.o \
 	adf_gen4_hw_data.o \
+	adf_gen4_vf_mig.o \
 	adf_gen4_pm.o \
 	adf_gen2_dc.o \
 	adf_gen4_dc.o \
 	adf_gen4_ras.o \
 	adf_gen4_timer.o \
 	adf_clock.o \
+	adf_mstate_mgr.o \
 	qat_crypto.o \
 	qat_compression.o \
 	qat_comp_algs.o \
drivers/crypto/intel/qat/qat_common/adf_accel_devices.h +6 -0
index b08fea10121e..7830ecb1a1f1 100644
@@ -412,11 +412,17 @@ struct adf_fw_loader_data {
 struct adf_accel_vf_info {
 	struct adf_accel_dev *accel_dev;
 	struct mutex pf2vf_lock; /* protect CSR access for PF2VF messages */
+	struct mutex pfvf_mig_lock; /* protects PFVF state for migration */
 	struct ratelimit_state vf2pf_ratelimit;
 	u32 vf_nr;
 	bool init;
 	bool restarting;
 	u8 vf_compat_ver;
+	/*
+	 * Private area used for device migration.
+	 * Memory allocation and free is managed by migration driver.
+	 */
+	void *mig_priv;
 };
 
 struct adf_dc_data {
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.h +10 -0
index e8cb930e80c9..8b10926cedba 100644
@@ -86,6 +86,7 @@
 #define ADF_RP_INT_SRC_SEL_F_RISE_MASK	BIT(2)
 #define ADF_RP_INT_SRC_SEL_F_FALL_MASK	GENMASK(2, 0)
 #define ADF_RP_INT_SRC_SEL_RANGE_WIDTH	4
+#define ADF_COALESCED_POLL_TIMEOUT_US	(1 * USEC_PER_SEC)
 #define ADF_COALESCED_POLL_DELAY_US	1000
 #define ADF_WQM_CSR_RPINTSOU(bank)	(0x200000 + ((bank) << 12))
 #define ADF_WQM_CSR_RP_IDX_RX		1
@@ -120,6 +121,15 @@
 /* PF2VM communication channel */
 #define ADF_GEN4_PF2VM_OFFSET(i)	(0x40B010 + (i) * 0x20)
 #define ADF_GEN4_VM2PF_OFFSET(i)	(0x40B014 + (i) * 0x20)
+#define ADF_GEN4_VINTMSKPF2VM_OFFSET(i)	(0x40B00C + (i) * 0x20)
+#define ADF_GEN4_VINTSOUPF2VM_OFFSET(i)	(0x40B008 + (i) * 0x20)
+#define ADF_GEN4_VINTMSK_OFFSET(i)	(0x40B004 + (i) * 0x20)
+#define ADF_GEN4_VINTSOU_OFFSET(i)	(0x40B000 + (i) * 0x20)
+
+struct adf_gen4_vfmig {
+	struct adf_mstate_mgr *mstate_mgr;
+	bool bank_stopped[ADF_GEN4_NUM_BANKS_PER_VF];
+};
 
 void adf_gen4_set_ssm_wdtimer(struct adf_accel_dev *accel_dev);
 
drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.c new file +1010 -0

1,010 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.c new file +318 -0

318 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.h new file +89 -0

89 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/adf_sriov.c +6 -1
index 87a70c00c41e..8d645e7e04aa 100644
@@ -26,10 +26,12 @@ static void adf_iov_send_resp(struct work_struct *work)
 	u32 vf_nr = vf_info->vf_nr;
 	bool ret;
 
+	mutex_lock(&vf_info->pfvf_mig_lock);
 	ret = adf_recv_and_handle_vf2pf_msg(accel_dev, vf_nr);
 	if (ret)
 		/* re-enable interrupt on PF from this VF */
 		adf_enable_vf2pf_interrupts(accel_dev, 1 << vf_nr);
+	mutex_unlock(&vf_info->pfvf_mig_lock);
 
 	kfree(pf2vf_resp);
 }
@@ -62,6 +64,7 @@ static int adf_enable_sriov(struct adf_accel_dev *accel_dev)
 		vf_info->vf_nr = i;
 
 		mutex_init(&vf_info->pf2vf_lock);
+		mutex_init(&vf_info->pfvf_mig_lock);
 		ratelimit_state_init(&vf_info->vf2pf_ratelimit,
 				     ADF_VF2PF_RATELIMIT_INTERVAL,
 				     ADF_VF2PF_RATELIMIT_BURST);
@@ -138,8 +141,10 @@ void adf_disable_sriov(struct adf_accel_dev *accel_dev)
 	if (hw_data->configure_iov_threads)
 		hw_data->configure_iov_threads(accel_dev, false);
 
-	for (i = 0, vf = accel_dev->pf.vf_info; i < totalvfs; i++, vf++)
+	for (i = 0, vf = accel_dev->pf.vf_info; i < totalvfs; i++, vf++) {
 		mutex_destroy(&vf->pf2vf_lock);
+		mutex_destroy(&vf->pfvf_mig_lock);
+	}
 
 	if (!test_bit(ADF_STATUS_RESTARTING, &accel_dev->status)) {
 		kfree(accel_dev->pf.vf_info);
Signature · 1 line
2.18.2
author
#11 lore raw

PATCH v4 10/10 vfio/qat: Add vfio_pci driver for Intel QAT VF devices

In reply to Xin Zeng

Add vfio pci driver for Intel QAT VF devices.

This driver uses vfio_pci_core to register to the VFIO subsystem. It acts as a vfio agent and interacts with the QAT PF driver to implement VF live migration.

---
 MAINTAINERS                   |   8 +
 drivers/vfio/pci/Kconfig      |   2 +
 drivers/vfio/pci/Makefile     |   2 +
 drivers/vfio/pci/qat/Kconfig  |  12 +
 drivers/vfio/pci/qat/Makefile |   3 +
 drivers/vfio/pci/qat/main.c   | 666 ++++++++++++++++++++++++++++++++++
 6 files changed, 693 insertions(+)
 create mode 100644 drivers/vfio/pci/qat/Kconfig
 create mode 100644 drivers/vfio/pci/qat/Makefile
 create mode 100644 drivers/vfio/pci/qat/main.c
MAINTAINERS +8 -0
index 5a4051996f1e..ee7a9dde4b19 100644
@@ -23105,6 +23105,14 @@ L:	[email protected]
 S:	Maintained
 F:	drivers/vfio/platform/
 
+VFIO QAT PCI DRIVER
+M:	Xin Zeng <[email protected]>
+M:	Giovanni Cabiddu <[email protected]>
+L:	[email protected]
+L:	[email protected]
+S:	Supported
+F:	drivers/vfio/pci/qat/
+
 VGA_SWITCHEROO
 R:	Lukas Wunner <[email protected]>
 S:	Maintained
drivers/vfio/pci/Kconfig +2 -0
index 18c397df566d..6fb5fb1dfc16 100644
@@ -67,4 +67,6 @@ source "drivers/vfio/pci/pds/Kconfig"
 
 source "drivers/vfio/pci/virtio/Kconfig"
 
+source "drivers/vfio/pci/qat/Kconfig"
+
 endmenu
drivers/vfio/pci/Makefile +2 -0
index 046139a4eca5..2e637ff66900 100644
@@ -15,3 +15,5 @@ obj-$(CONFIG_HISI_ACC_VFIO_PCI) += hisilicon/
 obj-$(CONFIG_PDS_VFIO_PCI) += pds/
 
 obj-$(CONFIG_VIRTIO_VFIO_PCI) += virtio/
+
+obj-$(CONFIG_QAT_VFIO_PCI) += qat/
drivers/vfio/pci/qat/Kconfig new file +12 -0
new file mode 100644
index 000000000000..bf52cfa4b595
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: GPL-2.0-only
+config QAT_VFIO_PCI
+	tristate "VFIO support for QAT VF PCI devices"
+	select VFIO_PCI_CORE
+	depends on CRYPTO_DEV_QAT_4XXX
+	help
+	  This provides migration support for Intel(R) QAT Virtual Function
+	  using the VFIO framework.
+
+	  To compile this as a module, choose M here: the module
+	  will be called qat_vfio_pci. If you don't know what to do here,
+	  say N.
drivers/vfio/pci/qat/Makefile new file +3 -0
new file mode 100644
index 000000000000..5fe5c4ec19d3
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-$(CONFIG_QAT_VFIO_PCI) += qat_vfio_pci.o
+qat_vfio_pci-y := main.o
drivers/vfio/pci/qat/main.c new file +666 -0

666 lines not on this page · raw message

Signature · 1 line
2.18.2
#12 lore raw

Re: [PATCH v4 10/10] vfio/qat: Add vfio_pci driver for Intel QAT VF devices

drivers/vfio/pci/qat/main.c line 613

Quoted 11 lines · On Wed, 28 Feb 2024 22:34:02 +0800 Xin Zeng <[email protected]> wrote:
drivers/vfio/pci/qat/main.c new file +11 -0
+static int
+qat_vf_vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
+{
+	struct device *dev = &pdev->dev;
+	struct qat_vf_core_device *qat_vdev;
+	int ret;
+
+	if (!pci_match_id(id, pdev)) {
+		pci_err(pdev, "Incompatible device, disallowing driver_override\n");
+		return -ENODEV;
+	}

I think the question of whether this is the right thing to do is still up for debate, but as noted in the thread where I raised the question, this mechanism doesn't actually work.

The probe callback is passed the matching ID from the set of dynamic IDs added via new_id, the driver id_table, or pci_device_id_any for a strictly driver_override match. Any of those would satisfy pci_match_id().

If we wanted the probe function to exclusively match devices in the id_table, we should call this as:

	if (!pci_match_id(qat_vf_vfio_pci_table, pdev))...

If we wanted to still allow dynamic IDs, the test might be more like:

	if (id == &pci_device_id_any)...

Allowing dynamic IDs but failing driver_override requires a slightly more sophisticated user, but is inconsistent. Do we have any consensus on this? Thanks,

Alex

Quoted 16 lines
drivers/vfio/pci/qat/main.c new file +16 -0
+
+	qat_vdev = vfio_alloc_device(qat_vf_core_device, core_device.vdev, dev, &qat_vf_pci_ops);
+	if (IS_ERR(qat_vdev))
+		return PTR_ERR(qat_vdev);
+
+	pci_set_drvdata(pdev, &qat_vdev->core_device);
+	ret = vfio_pci_core_register_device(&qat_vdev->core_device);
+	if (ret)
+		goto out_put_device;
+
+	return 0;
+
+out_put_device:
+	vfio_put_device(&qat_vdev->core_device.vdev);
+	return ret;
+}
author
#13 lore raw

RE: [PATCH v4 10/10] vfio/qat: Add vfio_pci driver for Intel QAT VF devices

Quoted 30 lines · Alex Williamson

Review

Quoted 2 lines · On Wed, 28 Feb 2024 22:34:02 +0800 Xin Zeng <[email protected]> wrote:
(quoted diff) +2 -0
+static int
+qat_vf_vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id

*id)

drivers/vfio/pci/qat/main.c

Quoted 7 lines
drivers/vfio/pci/qat/main.c new file +7 -0
+{
+	struct device *dev = &pdev->dev;
+	struct qat_vf_core_device *qat_vdev;
+	int ret;
+
+	if (!pci_match_id(id, pdev)) {
+		pci_err(pdev, "Incompatible device, disallowing

driver_override\n");

drivers/vfio/pci/qat/main.c line 613

Quoted 2 lines
drivers/vfio/pci/qat/main.c new file +2 -0
+		return -ENODEV;
+	}

I think the question of whether this is the right thing to do is still up for debate, but as noted in the thread where I raised the question, this mechanism doesn't actually work.

The probe callback is passed the matching ID from the set of dynamic IDs added via new_id, the driver id_table, or pci_device_id_any for a strictly driver_override match. Any of those would satisfy pci_match_id().

If we wanted the probe function to exclusively match devices in the id_table, we should call this as:

	if (!pci_match_id(qat_vf_vfio_pci_table, pdev))...
Quoted 8 lines

If we wanted to still allow dynamic IDs, the test might be more like:

	if (id == &pci_device_id_any)...

Allowing dynamic IDs but failing driver_override requires a slightly more sophisticated user, but is inconsistent. Do we have any consensus on this? Thanks,

As migration support is required to be reported at probe time in the variant driver, and migration support is device specific, the variant driver should be considered as device specific driver, not a general purpose vfio-pci stand-in.
From this point of view, maybe it's not necessary to allow dynamic id arbitrarily, matching devices in the id_table provided by the variant driver exclusively would be the preference. If this sound good to you, I'll update the implementation (to use the correct id table) in next version.
Thanks,
Xin

reviewer
#14 lore raw

RE: [PATCH v4 10/10] vfio/qat: Add vfio_pci driver for Intel QAT VF devices

Quoted 44 lines

From: Zeng, Xin <[email protected]>
Sent: Monday, March 4, 2024 7:11 PM

Quoted 30 lines · Alex Williamson

Review

Quoted 2 lines · On Wed, 28 Feb 2024 22:34:02 +0800 Xin Zeng <[email protected]> wrote:
(quoted diff) +2 -0
+static int
+qat_vf_vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id

*id)

drivers/vfio/pci/qat/main.c

Quoted 7 lines
drivers/vfio/pci/qat/main.c new file +7 -0
+{
+	struct device *dev = &pdev->dev;
+	struct qat_vf_core_device *qat_vdev;
+	int ret;
+
+	if (!pci_match_id(id, pdev)) {
+		pci_err(pdev, "Incompatible device, disallowing

driver_override\n");

drivers/vfio/pci/qat/main.c line 613

Quoted 2 lines
drivers/vfio/pci/qat/main.c new file +2 -0
+		return -ENODEV;
+	}

I think the question of whether this is the right thing to do is still up for debate, but as noted in the thread where I raised the question, this mechanism doesn't actually work.

The probe callback is passed the matching ID from the set of dynamic IDs added via new_id, the driver id_table, or pci_device_id_any for a strictly driver_override match. Any of those would satisfy pci_match_id().

If we wanted the probe function to exclusively match devices in the id_table, we should call this as:

	if (!pci_match_id(qat_vf_vfio_pci_table, pdev))...
Quoted 8 lines

If we wanted to still allow dynamic IDs, the test might be more like:

	if (id == &pci_device_id_any)...

Allowing dynamic IDs but failing driver_override requires a slightly more sophisticated user, but is inconsistent. Do we have any consensus on this? Thanks,

Quoted 9 lines

As migration support is required to be reported at probe time in the variant driver, and migration support is device specific, the variant driver should be considered as device specific driver, not a general purpose vfio-pci stand-in.
From this point of view, maybe it's not necessary to allow dynamic id arbitrarily, matching devices in the id_table provided by the variant driver exclusively would be the preference. If this sound good to you, I'll update the implementation (to use the correct id table) in next version.

I'm leaning toward this view too. Somehow driver_override is more restrictive than dynamic IDs based on its description then it sounds a bit weird that on one hand we are doing strict match upon static table but on the other hand lifting the bar for dynamic IDs.

If consensus cannot be reached quickly I prefer to not adding the match and wait to change all drivers together when it is made later.

reviewer
#15 lore raw

RE: [PATCH v4 10/10] vfio/qat: Add vfio_pci driver for Intel QAT VF devices

In reply to Xin Zeng
Quoted 19 lines

From: Zeng, Xin <[email protected]>
Sent: Wednesday, February 28, 2024 10:34 PM

drivers/vfio/pci/qat/main.c new file +17 -0
+
+static long qat_vf_precopy_ioctl(struct file *filp, unsigned int cmd,
+				 unsigned long arg)
+{
+	struct qat_vf_migration_file *migf = filp->private_data;
+	struct qat_vf_core_device *qat_vdev = migf->qat_vdev;
+	struct qat_mig_dev *mig_dev = qat_vdev->mdev;
+	struct vfio_precopy_info info;
+	loff_t *pos = &filp->f_pos;
+	unsigned long minsz;
+	int ret = 0;
+
+	if (cmd != VFIO_MIG_GET_PRECOPY_INFO)
+		return -ENOTTY;
+
+	minsz = offsetofend(struct vfio_precopy_info, dirty_bytes);
+

drivers/vfio/pci/qat/main.c line 121

Quoted 10 lines
drivers/vfio/pci/qat/main.c new file +10 -0
+	if (copy_from_user(&info, (void __user *)arg, minsz))
+		return -EFAULT;
+	if (info.argsz < minsz)
+		return -EINVAL;
+
+	mutex_lock(&qat_vdev->state_mutex);
+	if (qat_vdev->mig_state != VFIO_DEVICE_STATE_PRE_COPY) {
+		mutex_unlock(&qat_vdev->state_mutex);
+		return -EINVAL;
+	}

what about PRE_COPY_P2P?

Quoted 20 lines
(quoted diff) +2 -0
+static struct qat_vf_migration_file *
+qat_vf_save_device_data(struct qat_vf_core_device *qat_vdev, bool

pre_copy)

drivers/vfio/pci/qat/main.c new file +17 -0
+{
+	struct qat_vf_migration_file *migf;
+	int ret;
+
+	migf = kzalloc(sizeof(*migf), GFP_KERNEL);
+	if (!migf)
+		return ERR_PTR(-ENOMEM);
+
+	migf->filp = anon_inode_getfile("qat_vf_mig", &qat_vf_save_fops,
+					migf, O_RDONLY);
+	ret = PTR_ERR_OR_ZERO(migf->filp);
+	if (ret) {
+		kfree(migf);
+		return ERR_PTR(ret);
+	}
+
+	stream_open(migf->filp->f_inode, migf->filp);

drivers/vfio/pci/qat/main.c line 259

Quoted 10 lines
drivers/vfio/pci/qat/main.c new file +10 -0
+	mutex_init(&migf->lock);
+
+	if (pre_copy)
+		ret = qat_vf_save_setup(qat_vdev, migf);
+	else
+		ret = qat_vf_save_state(qat_vdev, migf);
+	if (ret) {
+		fput(migf->filp);
+		return ERR_PTR(ret);
+	}

lack of kfree(migf). Using goto can avoid such typo in error unwind.

Quoted 11 lines
(quoted diff) +2 -0
+
+static struct file *qat_vf_pci_step_device_state(struct qat_vf_core_device

*qat_vdev, u32 new)

drivers/vfio/pci/qat/main.c new file +5 -0
+{
+	u32 cur = qat_vdev->mig_state;
+	int ret;
+
+	if ((cur == VFIO_DEVICE_STATE_RUNNING && new ==

VFIO_DEVICE_STATE_RUNNING_P2P) ||

(quoted diff) +1 -0
+	    (cur == VFIO_DEVICE_STATE_PRE_COPY && new ==

VFIO_DEVICE_STATE_PRE_COPY_P2P)) {

drivers/vfio/pci/qat/main.c line 360

Quoted 5 lines
drivers/vfio/pci/qat/main.c new file +5 -0
+		ret = qat_vfmig_suspend(qat_vdev->mdev);
+		if (ret)
+			return ERR_PTR(ret);
+		return NULL;
+	}

could you arrange the transitions in pair as mlx does e.g. following above is the reverse one calling qat_vfmig_resume()?

drivers/vfio/pci/qat/main.c line 664

Quoted 3 lines
drivers/vfio/pci/qat/main.c new file +3 -0
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Intel Corporation");

please use your name as the author.

author
#16 lore raw

RE: [PATCH v4 10/10] vfio/qat: Add vfio_pci driver for Intel QAT VF devices

Thanks for the comments, Kevin.

Quoted 23 lines · Kevin
Quoted 23 lines

From: Zeng, Xin <[email protected]>
Sent: Wednesday, February 28, 2024 10:34 PM

drivers/vfio/pci/qat/main.c new file +21 -0
+
+static long qat_vf_precopy_ioctl(struct file *filp, unsigned int cmd,
+				 unsigned long arg)
+{
+	struct qat_vf_migration_file *migf = filp->private_data;
+	struct qat_vf_core_device *qat_vdev = migf->qat_vdev;
+	struct qat_mig_dev *mig_dev = qat_vdev->mdev;
+	struct vfio_precopy_info info;
+	loff_t *pos = &filp->f_pos;
+	unsigned long minsz;
+	int ret = 0;
+
+	if (cmd != VFIO_MIG_GET_PRECOPY_INFO)
+		return -ENOTTY;
+
+	minsz = offsetofend(struct vfio_precopy_info, dirty_bytes);
+
+	if (copy_from_user(&info, (void __user *)arg, minsz))
+		return -EFAULT;
+	if (info.argsz < minsz)
+		return -EINVAL;
Quoted 8 lines

drivers/vfio/pci/qat/main.c line 121

Quoted 6 lines
drivers/vfio/pci/qat/main.c new file +6 -0
+
+	mutex_lock(&qat_vdev->state_mutex);
+	if (qat_vdev->mig_state != VFIO_DEVICE_STATE_PRE_COPY) {
+		mutex_unlock(&qat_vdev->state_mutex);
+		return -EINVAL;
+	}

what about PRE_COPY_P2P?

Good catch, will add it in next version.

Quoted 25 lines
Quoted 25 lines
(quoted diff) +2 -0
+static struct qat_vf_migration_file *
+qat_vf_save_device_data(struct qat_vf_core_device *qat_vdev, bool

pre_copy)

drivers/vfio/pci/qat/main.c new file +21 -0
+{
+	struct qat_vf_migration_file *migf;
+	int ret;
+
+	migf = kzalloc(sizeof(*migf), GFP_KERNEL);
+	if (!migf)
+		return ERR_PTR(-ENOMEM);
+
+	migf->filp = anon_inode_getfile("qat_vf_mig", &qat_vf_save_fops,
+					migf, O_RDONLY);
+	ret = PTR_ERR_OR_ZERO(migf->filp);
+	if (ret) {
+		kfree(migf);
+		return ERR_PTR(ret);
+	}
+
+	stream_open(migf->filp->f_inode, migf->filp);
+	mutex_init(&migf->lock);
+
+	if (pre_copy)
+		ret = qat_vf_save_setup(qat_vdev, migf);
Quoted 8 lines

drivers/vfio/pci/qat/main.c line 259

Quoted 6 lines
drivers/vfio/pci/qat/main.c new file +6 -0
+	else
+		ret = qat_vf_save_state(qat_vdev, migf);
+	if (ret) {
+		fput(migf->filp);
+		return ERR_PTR(ret);
+	}

lack of kfree(migf). Using goto can avoid such typo in error unwind.

kfree(migf) will be invoked in fput. This was pointed out by Shameer and I updated it in v2. :-)

Quoted 13 lines

Review

Quoted 3 lines
(quoted diff) +2 -0
+
+static struct file *qat_vf_pci_step_device_state(struct

qat_vf_core_device

Quoted 9 lines

*qat_vdev, u32 new)

drivers/vfio/pci/qat/main.c new file +5 -0
+{
+	u32 cur = qat_vdev->mig_state;
+	int ret;
+
+	if ((cur == VFIO_DEVICE_STATE_RUNNING && new ==

VFIO_DEVICE_STATE_RUNNING_P2P) ||

(quoted diff) +1 -0
+	    (cur == VFIO_DEVICE_STATE_PRE_COPY && new ==

VFIO_DEVICE_STATE_PRE_COPY_P2P)) {

Quoted 8 lines

drivers/vfio/pci/qat/main.c line 360

Quoted 5 lines
drivers/vfio/pci/qat/main.c new file +5 -0
+		ret = qat_vfmig_suspend(qat_vdev->mdev);
+		if (ret)
+			return ERR_PTR(ret);
+		return NULL;
+	}

could you arrange the transitions in pair as mlx does e.g. following above is the reverse one calling qat_vfmig_resume()?

Sure, will update it in next version.

Quoted 6 lines

drivers/vfio/pci/qat/main.c line 664

Quoted 4 lines
drivers/vfio/pci/qat/main.c new file +3 -0
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Intel Corporation");

please use your name as the author.

Sure, will do.