Skip to contentSkip to conversation

crypto: qat - enable SRIOV VF live migration

RFC In review Xin Zeng 5 participants 19 messages Jun 30, 2023 – Aug 24, 2023

Subsystems (6)
lore
Share & tools
lore

Xin Zeng

author
#0 lore raw

cover crypto: qat - enable SRIOV VF live migration

This set enables live migration for Intel QAT GEN4 SRIOV Virtual Functions (VFs). It is composed of 5 patches.
The first is a pre-requisite. It adds logic to the QAT PF driver that allows to save and restore the state of a bank (a virtual function is a wrapper around a bank) and drain a ring pair. The second patch adds to the QAT PF driver a set of interfaces to allow to save and restore the state of a VF that will be called by the modules qat_vfio_pci which will be introduced in the last patch. The third adds HZ_PER_GHZ which will be required by the fourth one. The fourth one implements the defined interfaces. The last one adds a vfio pci extension specific for QAT which intercepts the vfio device operations for a QAT VF to allow live migration.

Here are the steps required to test the live migration of a QAT GEN4 VF:
1. Bind one or more QAT GEN4 VF devices to the module qat_vfio_pci.ko
2. Assign the VFs to the virtual machine and enable device live migration
3. Run a workload using a QAT VF inside the VM, for example using qatlib
(https://github.com/intel/qatlib)
4. Migrate the VM from the source node to a destination node

For P2P states support and AER support, we are going to implement these in the final version. Any feedback is appreciated!

Andy Shevchenko (1):
units: Add HZ_PER_GHZ

Siming Wan (1):
crypto: qat - add bank save/restore and RP drain

Xin Zeng (3):
crypto: qat - add interface for live migration
crypto: qat - implement interface for live migration vfio/qat: Add vfio_pci driver for Intel QAT VF devices

 .../intel/qat/qat_4xxx/adf_4xxx_hw_data.c     |   9 +-
 .../intel/qat/qat_4xxx/adf_4xxx_hw_data.h     |   3 +-
 .../intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c   |   2 +-
 .../qat/qat_c3xxxvf/adf_c3xxxvf_hw_data.c     |   2 +-
 .../intel/qat/qat_c62x/adf_c62x_hw_data.c     |   2 +-
 .../intel/qat/qat_c62xvf/adf_c62xvf_hw_data.c |   2 +-
 drivers/crypto/intel/qat/qat_common/Makefile  |   4 +-
 .../intel/qat/qat_common/adf_accel_devices.h  |  78 ++-
 .../intel/qat/qat_common/adf_gen2_hw_data.c   |  17 +-
 .../intel/qat/qat_common/adf_gen2_hw_data.h   |  10 +-
 .../intel/qat/qat_common/adf_gen4_hw_data.c   | 415 +++++++++++-
 .../intel/qat/qat_common/adf_gen4_hw_data.h   | 152 ++++-
 .../intel/qat/qat_common/adf_gen4_pfvf.c      |   7 +-
 .../intel/qat/qat_common/adf_gen4_pfvf.h      |   7 +
 .../intel/qat/qat_common/adf_gen4_vf_mig.c    | 609 ++++++++++++++++++
 .../intel/qat/qat_common/adf_mstate_mgr.c     | 267 ++++++++
 .../intel/qat/qat_common/adf_mstate_mgr.h     |  99 +++
 .../intel/qat/qat_common/adf_transport.c      |  11 +-
 .../crypto/intel/qat/qat_common/adf_vf_isr.c  |   2 +-
 .../crypto/intel/qat/qat_common/qat_vf_mig.c  | 106 +++
 .../qat/qat_dh895xcc/adf_dh895xcc_hw_data.c   |   2 +-
 .../qat_dh895xccvf/adf_dh895xccvf_hw_data.c   |   2 +-
 drivers/vfio/pci/Kconfig                      |   2 +
 drivers/vfio/pci/Makefile                     |   1 +
 drivers/vfio/pci/qat/Kconfig                  |  13 +
 drivers/vfio/pci/qat/Makefile                 |   4 +
 drivers/vfio/pci/qat/qat_vfio_pci_main.c      | 518 +++++++++++++++
 include/linux/qat/qat_vf_mig.h                |  15 +
 include/linux/units.h                         |   6 +-
 29 files changed, 2332 insertions(+), 35 deletions(-)
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.c
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.c
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.h
 create mode 100644 drivers/crypto/intel/qat/qat_common/qat_vf_mig.c
 create mode 100644 drivers/vfio/pci/qat/Kconfig
 create mode 100644 drivers/vfio/pci/qat/Makefile
 create mode 100644 drivers/vfio/pci/qat/qat_vfio_pci_main.c
 create mode 100644 include/linux/qat/qat_vf_mig.h
Signature · 1 line
2.18.2
author
#1 lore raw

RFC 1/5 crypto: qat - add bank save/restore and RP drain

From: Siming Wan <[email protected]>

Extend CSR ops and add logic to save and restore banks and drain the ring pairs. This will be used to implement live migration.

This is implemented only for QAT GEN4 devices.

---
 .../intel/qat/qat_4xxx/adf_4xxx_hw_data.c     |   5 +-
 .../intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c   |   2 +-
 .../qat/qat_c3xxxvf/adf_c3xxxvf_hw_data.c     |   2 +-
 .../intel/qat/qat_c62x/adf_c62x_hw_data.c     |   2 +-
 .../intel/qat/qat_c62xvf/adf_c62xvf_hw_data.c |   2 +-
 .../intel/qat/qat_common/adf_accel_devices.h  |  60 ++-
 .../intel/qat/qat_common/adf_gen2_hw_data.c   |  17 +-
 .../intel/qat/qat_common/adf_gen2_hw_data.h   |  10 +-
 .../intel/qat/qat_common/adf_gen4_hw_data.c   | 362 +++++++++++++++++-
 .../intel/qat/qat_common/adf_gen4_hw_data.h   | 131 ++++++-
 .../intel/qat/qat_common/adf_transport.c      |  11 +-
 .../crypto/intel/qat/qat_common/adf_vf_isr.c  |   2 +-
 .../qat/qat_dh895xcc/adf_dh895xcc_hw_data.c   |   2 +-
 .../qat_dh895xccvf/adf_dh895xccvf_hw_data.c   |   2 +-
 14 files changed, 584 insertions(+), 26 deletions(-)
drivers/crypto/intel/qat/qat_4xxx/adf_4xxx_hw_data.c +4 -1
index e543a9e24a06..22fe4e6834c1 100644
@@ -505,11 +505,14 @@ void adf_init_hw_data_4xxx(struct adf_hw_device_data *hw_data, u32 dev_id)
 	hw_data->set_ssm_wdtimer = adf_gen4_set_ssm_wdtimer;
 	hw_data->disable_iov = adf_disable_sriov;
 	hw_data->ring_pair_reset = adf_gen4_ring_pair_reset;
+	hw_data->ring_pair_drain = adf_gen4_ring_pair_drain;
+	hw_data->bank_state_save = adf_gen4_bank_state_save;
+	hw_data->bank_state_restore = adf_gen4_bank_state_restore;
 	hw_data->enable_pm = adf_gen4_enable_pm;
 	hw_data->handle_pm_interrupt = adf_gen4_handle_pm_interrupt;
 	hw_data->dev_config = adf_gen4_dev_config;
 
-	adf_gen4_init_hw_csr_ops(&hw_data->csr_ops);
+	adf_gen4_init_hw_csr_ops(&hw_data->csr_info);
 	adf_gen4_init_pf_pfvf_ops(&hw_data->pfvf_ops);
 	adf_gen4_init_dc_ops(&hw_data->dc_ops);
 }
drivers/crypto/intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c +1 -1
index 475643654e64..04a833affb29 100644
@@ -129,7 +129,7 @@ void adf_init_hw_data_c3xxx(struct adf_hw_device_data *hw_data)
 	hw_data->dev_config = adf_gen2_dev_config;
 
 	adf_gen2_init_pf_pfvf_ops(&hw_data->pfvf_ops);
-	adf_gen2_init_hw_csr_ops(&hw_data->csr_ops);
+	adf_gen2_init_hw_csr_ops(&hw_data->csr_info);
 	adf_gen2_init_dc_ops(&hw_data->dc_ops);
 }
 
drivers/crypto/intel/qat/qat_c3xxxvf/adf_c3xxxvf_hw_data.c +1 -1
index 84d9486e04de..5cb46b2cd278 100644
@@ -91,7 +91,7 @@ void adf_init_hw_data_c3xxxiov(struct adf_hw_device_data *hw_data)
 	hw_data->dev_config = adf_gen2_dev_config;
 	adf_devmgr_update_class_index(hw_data);
 	adf_gen2_init_vf_pfvf_ops(&hw_data->pfvf_ops);
-	adf_gen2_init_hw_csr_ops(&hw_data->csr_ops);
+	adf_gen2_init_hw_csr_ops(&hw_data->csr_info);
 	adf_gen2_init_dc_ops(&hw_data->dc_ops);
 }
 
drivers/crypto/intel/qat/qat_c62x/adf_c62x_hw_data.c +1 -1
index e14270703670..fd6f18c5f8e2 100644
@@ -131,7 +131,7 @@ void adf_init_hw_data_c62x(struct adf_hw_device_data *hw_data)
 	hw_data->dev_config = adf_gen2_dev_config;
 
 	adf_gen2_init_pf_pfvf_ops(&hw_data->pfvf_ops);
-	adf_gen2_init_hw_csr_ops(&hw_data->csr_ops);
+	adf_gen2_init_hw_csr_ops(&hw_data->csr_info);
 	adf_gen2_init_dc_ops(&hw_data->dc_ops);
 }
 
drivers/crypto/intel/qat/qat_c62xvf/adf_c62xvf_hw_data.c +1 -1
index 751d7aa57fc7..ca3e0cea056f 100644
@@ -91,7 +91,7 @@ void adf_init_hw_data_c62xiov(struct adf_hw_device_data *hw_data)
 	hw_data->dev_config = adf_gen2_dev_config;
 	adf_devmgr_update_class_index(hw_data);
 	adf_gen2_init_vf_pfvf_ops(&hw_data->pfvf_ops);
-	adf_gen2_init_hw_csr_ops(&hw_data->csr_ops);
+	adf_gen2_init_hw_csr_ops(&hw_data->csr_info);
 	adf_gen2_init_dc_ops(&hw_data->dc_ops);
 }
 
drivers/crypto/intel/qat/qat_common/adf_accel_devices.h +57 -3

99 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/adf_gen2_hw_data.c +14 -3
index d1884547b5a1..d956910f3228 100644
@@ -160,9 +160,10 @@ static void write_csr_int_flag(void __iomem *csr_base_addr, u32 bank, u32 value)
 	WRITE_CSR_INT_FLAG(csr_base_addr, bank, value);
 }
 
-static void write_csr_int_srcsel(void __iomem *csr_base_addr, u32 bank)
+static void write_csr_int_srcsel(void __iomem *csr_base_addr, u32 bank,
+				 u32 idx, u32 value)
 {
-	WRITE_CSR_INT_SRCSEL(csr_base_addr, bank);
+	WRITE_CSR_INT_SRCSEL(csr_base_addr, bank, idx, value);
 }
 
 static void write_csr_int_col_en(void __iomem *csr_base_addr, u32 bank,
@@ -189,8 +190,17 @@ static void write_csr_ring_srv_arb_en(void __iomem *csr_base_addr, u32 bank,
 	WRITE_CSR_RING_SRV_ARB_EN(csr_base_addr, bank, value);
 }
 
-void adf_gen2_init_hw_csr_ops(struct adf_hw_csr_ops *csr_ops)
+static u32 get_src_sel_mask(void)
 {
+	return ADF_BANK_INT_SRC_SEL_MASK_X;
+}
+
+void adf_gen2_init_hw_csr_ops(struct adf_hw_csr_info *csr_info)
+{
+	struct adf_hw_csr_ops *csr_ops = &csr_info->csr_ops;
+
+	csr_info->num_rings_per_int_srcsel = ADF_RINGS_PER_INT_SRCSEL;
+
 	csr_ops->build_csr_ring_base_addr = build_csr_ring_base_addr;
 	csr_ops->read_csr_ring_head = read_csr_ring_head;
 	csr_ops->write_csr_ring_head = write_csr_ring_head;
@@ -205,6 +215,7 @@ void adf_gen2_init_hw_csr_ops(struct adf_hw_csr_ops *csr_ops)
 	csr_ops->write_csr_int_col_ctl = write_csr_int_col_ctl;
 	csr_ops->write_csr_int_flag_and_col = write_csr_int_flag_and_col;
 	csr_ops->write_csr_ring_srv_arb_en = write_csr_ring_srv_arb_en;
+	csr_ops->get_src_sel_mask = get_src_sel_mask;
 }
 EXPORT_SYMBOL_GPL(adf_gen2_init_hw_csr_ops);
 
drivers/crypto/intel/qat/qat_common/adf_gen2_hw_data.h +8 -2
index e4bc07529be4..631eb2e2f334 100644
@@ -7,8 +7,10 @@
 #include "adf_cfg_common.h"
 
 /* Transport access */
+#define ADF_RINGS_PER_INT_SRCSEL	BIT(3)
 #define ADF_BANK_INT_SRC_SEL_MASK_0	0x4444444CUL
 #define ADF_BANK_INT_SRC_SEL_MASK_X	0x44444444UL
+#define ADF_RING_SRV_ARB_EN_MASK	GENMASK(7, 0)
 #define ADF_RING_CSR_RING_CONFIG	0x000
 #define ADF_RING_CSR_RING_LBASE		0x040
 #define ADF_RING_CSR_RING_UBASE		0x080
@@ -25,6 +27,7 @@
 #define ADF_RING_BUNDLE_SIZE		0x1000
 #define ADF_GEN2_RX_RINGS_OFFSET	8
 #define ADF_GEN2_TX_RINGS_MASK		0xFF
+#define ADF_RING_CSR_NEXT_INT_SRCSEL	BIT(2)
 
 #define BUILD_RING_BASE_ADDR(addr, size) \
 	(((addr) >> 6) & (GENMASK_ULL(63, 0) << (size)))
@@ -60,7 +63,10 @@ do { \
 #define WRITE_CSR_INT_FLAG(csr_base_addr, bank, value) \
 	ADF_CSR_WR(csr_base_addr, (ADF_RING_BUNDLE_SIZE * (bank)) + \
 		   ADF_RING_CSR_INT_FLAG, value)
-#define WRITE_CSR_INT_SRCSEL(csr_base_addr, bank) \
+#define READ_CSR_INT_SRCSEL(csr_base_addr, bank, idx) \
+	ADF_CSR_RD(csr_base_addr, ADF_RING_BUNDLE_SIZE * (bank) + \
+	ADF_RING_CSR_INT_SRCSEL + (idx) * ADF_RING_CSR_NEXT_INT_SRCSEL)
+#define WRITE_CSR_INT_SRCSEL(csr_base_addr, bank, idx, value) \
 do { \
 	ADF_CSR_WR(csr_base_addr, (ADF_RING_BUNDLE_SIZE * (bank)) + \
 	ADF_RING_CSR_INT_SRCSEL, ADF_BANK_INT_SRC_SEL_MASK_0); \
@@ -155,7 +161,7 @@ u32 adf_gen2_get_num_aes(struct adf_hw_device_data *self);
 void adf_gen2_enable_error_correction(struct adf_accel_dev *accel_dev);
 void adf_gen2_cfg_iov_thds(struct adf_accel_dev *accel_dev, bool enable,
 			   int num_a_regs, int num_b_regs);
-void adf_gen2_init_hw_csr_ops(struct adf_hw_csr_ops *csr_ops);
+void adf_gen2_init_hw_csr_ops(struct adf_hw_csr_info *csr_info);
 void adf_gen2_get_admin_info(struct admin_info *admin_csrs_info);
 void adf_gen2_get_arb_info(struct arb_info *arb_info);
 void adf_gen2_enable_ints(struct adf_accel_dev *accel_dev);
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +359 -3

431 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.h +126 -5

205 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/adf_transport.c +9 -2
index 630d0483c4e0..ce71ac17e617 100644
@@ -387,10 +387,12 @@ static int adf_init_bank(struct adf_accel_dev *accel_dev,
 {
 	struct adf_hw_device_data *hw_data = accel_dev->hw_device;
 	u8 num_rings_per_bank = hw_data->num_rings_per_bank;
-	struct adf_hw_csr_ops *csr_ops = &hw_data->csr_ops;
+	struct adf_hw_csr_info *csr_info = &hw_data->csr_info;
+	struct adf_hw_csr_ops *csr_ops = &csr_info->csr_ops;
 	u32 irq_mask = BIT(num_rings_per_bank) - 1;
 	struct adf_etr_ring_data *ring;
 	struct adf_etr_ring_data *tx_ring;
+	u32 num_rings_per_int_srcsel;
 	u32 i, coalesc_enabled = 0;
 	unsigned long ring_mask;
 	int size;
@@ -447,7 +449,12 @@ static int adf_init_bank(struct adf_accel_dev *accel_dev,
 	}
 
 	csr_ops->write_csr_int_flag(csr_addr, bank_num, irq_mask);
-	csr_ops->write_csr_int_srcsel(csr_addr, bank_num);
+
+	num_rings_per_int_srcsel = csr_info->num_rings_per_int_srcsel;
+
+	for (i = 0; i < num_rings_per_bank / num_rings_per_int_srcsel; i++)
+		csr_ops->write_csr_int_srcsel(csr_addr, bank_num, i,
+					      csr_ops->get_src_sel_mask());
 
 	return 0;
 err:
drivers/crypto/intel/qat/qat_common/adf_vf_isr.c +1 -1
index b05c3957a160..eab42026df8e 100644
@@ -131,7 +131,7 @@ static irqreturn_t adf_isr(int irq, void *privdata)
 {
 	struct adf_accel_dev *accel_dev = privdata;
 	struct adf_hw_device_data *hw_data = accel_dev->hw_device;
-	struct adf_hw_csr_ops *csr_ops = &hw_data->csr_ops;
+	struct adf_hw_csr_ops *csr_ops = GET_CSR_OPS(accel_dev);
 	struct adf_bar *pmisc =
 			&GET_BARS(accel_dev)[hw_data->get_misc_bar_id(hw_data)];
 	void __iomem *pmisc_bar_addr = pmisc->virt_addr;
drivers/crypto/intel/qat/qat_dh895xcc/adf_dh895xcc_hw_data.c +1 -1
index 1ebe0b351fae..2a01985ca7f5 100644
@@ -242,7 +242,7 @@ void adf_init_hw_data_dh895xcc(struct adf_hw_device_data *hw_data)
 	hw_data->pfvf_ops.enable_vf2pf_interrupts = enable_vf2pf_interrupts;
 	hw_data->pfvf_ops.disable_all_vf2pf_interrupts = disable_all_vf2pf_interrupts;
 	hw_data->pfvf_ops.disable_pending_vf2pf_interrupts = disable_pending_vf2pf_interrupts;
-	adf_gen2_init_hw_csr_ops(&hw_data->csr_ops);
+	adf_gen2_init_hw_csr_ops(&hw_data->csr_info);
 	adf_gen2_init_dc_ops(&hw_data->dc_ops);
 }
 
drivers/crypto/intel/qat/qat_dh895xccvf/adf_dh895xccvf_hw_data.c +1 -1
index 70e56cc16ece..ac668c038fc6 100644
@@ -91,7 +91,7 @@ void adf_init_hw_data_dh895xcciov(struct adf_hw_device_data *hw_data)
 	hw_data->dev_config = adf_gen2_dev_config;
 	adf_devmgr_update_class_index(hw_data);
 	adf_gen2_init_vf_pfvf_ops(&hw_data->pfvf_ops);
-	adf_gen2_init_hw_csr_ops(&hw_data->csr_ops);
+	adf_gen2_init_hw_csr_ops(&hw_data->csr_info);
 	adf_gen2_init_dc_ops(&hw_data->dc_ops);
 }
 
Signature · 1 line
2.18.2
reviewer
#2 lore raw

RE: [RFC 1/5] crypto: qat - add bank save/restore and RP drain

Quoted 18 lines

From: Xin Zeng <[email protected]>
Sent: Friday, June 30, 2023 9:13 PM

---
 .../intel/qat/qat_4xxx/adf_4xxx_hw_data.c     |   5 +-
 .../intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c   |   2 +-
 .../qat/qat_c3xxxvf/adf_c3xxxvf_hw_data.c     |   2 +-
 .../intel/qat/qat_c62x/adf_c62x_hw_data.c     |   2 +-
 .../intel/qat/qat_c62xvf/adf_c62xvf_hw_data.c |   2 +-
 .../intel/qat/qat_common/adf_accel_devices.h  |  60 ++-
 .../intel/qat/qat_common/adf_gen2_hw_data.c   |  17 +-
 .../intel/qat/qat_common/adf_gen2_hw_data.h   |  10 +-
 .../intel/qat/qat_common/adf_gen4_hw_data.c   | 362 +++++++++++++++++-
 .../intel/qat/qat_common/adf_gen4_hw_data.h   | 131 ++++++-
 .../intel/qat/qat_common/adf_transport.c      |  11 +-
 .../crypto/intel/qat/qat_common/adf_vf_isr.c  |   2 +-
 .../qat/qat_dh895xcc/adf_dh895xcc_hw_data.c   |   2 +-
 .../qat_dh895xccvf/adf_dh895xccvf_hw_data.c   |   2 +-
 14 files changed, 584 insertions(+), 26 deletions(-)

this could be split into 3 patches.

one is moving from hw_data->csr_ops to hw_data->csr_info. apply to all qat drivers.

the 2nd is adding new csr_ops.

the last one then covers bank save/restore.

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c int adf_gen4_ring_pair_reset(struct adf_accel_dev *accel_dev, u32 bank_number) · line 396

Quoted 7 lines
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +7 -0
+
+#define ADF_RP_INT_SRC_SEL_F_RISE_MASK BIT(2)
+#define ADF_RP_INT_SRC_SEL_F_FALL_MASK GENMASK(2, 0)
+static int gen4_bank_state_restore(void __iomem *csr, u32 bank_number,
+				   struct bank_state *state, u32 num_rings,
+				   int tx_rx_gap)
+{

restore is the most tricky part. it's worth of some comments to help understand the flow, e.g. what is rx/tx layout, why there are multiple ring tails writes, etc.

Quoted 110 lines
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +6 -0
+	u32 val, tmp_val, i;
+
+	write_csr_ring_srv_arb_en(csr, bank_number, 0);
+
+	for (i = 0; i < num_rings; i++)
+		write_csr_ring_base(csr, bank_number, i, state-
Quoted 1 line

rings[i].base);

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +3 -0
+
+	for (i = 0; i < num_rings; i++)
+		write_csr_ring_config(csr, bank_number, i, state-
Quoted 1 line

rings[i].config);

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +8 -0
+
+	for (i = 0; i < num_rings / 2; i++) {
+		int tx = i * (tx_rx_gap + 1);
+		int rx = tx + tx_rx_gap;
+		u32 tx_idx = tx / ADF_RINGS_PER_INT_SRCSEL;
+		u32 rx_idx = rx / ADF_RINGS_PER_INT_SRCSEL;
+
+		write_csr_ring_head(csr, bank_number, tx, state-
Quoted 1 line

rings[tx].head);

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +6 -0
+
+		write_csr_ring_tail(csr, bank_number, tx, state->rings[tx].tail);
+
+		if (state->ringestat & (BIT(tx))) {
+			val = read_csr_int_srcsel(csr, bank_number, tx_idx);
+			val |= (ADF_RP_INT_SRC_SEL_F_RISE_MASK << (8 *

tx));

(quoted diff) +2 -0
+			write_csr_int_srcsel(csr, bank_number, tx_idx, val);
+			write_csr_ring_head(csr, bank_number, tx, state-
Quoted 1 line

rings[tx].head);

drivers/crypto/intel/qat/qat_common/adf_gen2_hw_data.c +3 -0
+		}
+
+		write_csr_ring_tail(csr, bank_number, rx, state-
Quoted 1 line

rings[rx].tail);

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +6 -0
+
+		val = read_csr_int_srcsel(csr, bank_number, rx_idx);
+		val |= (ADF_RP_INT_SRC_SEL_F_RISE_MASK << (8 * rx));
+		write_csr_int_srcsel(csr, bank_number, rx_idx, val);
+
+		write_csr_ring_head(csr, bank_number, rx, state-
Quoted 1 line

rings[rx].head);

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +7 -0
+
+		val = read_csr_int_srcsel(csr, bank_number, rx_idx);
+		val |= (ADF_RP_INT_SRC_SEL_F_FALL_MASK << (8 * rx));
+		write_csr_int_srcsel(csr, bank_number, rx_idx, val);
+
+		if (state->ringfstat & BIT(rx))
+			write_csr_ring_tail(csr, bank_number, rx, state-
Quoted 1 line

rings[rx].tail);

drivers/crypto/intel/qat/qat_common/adf_gen2_hw_data.c +3 -0
+	}
+
+	write_csr_int_flag_and_col(csr, bank_number, state-
Quoted 1 line

iaintflagandcolen);

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +11 -0
+	write_csr_int_en(csr, bank_number, state->iaintflagen);
+	write_csr_int_col_en(csr, bank_number, state->iaintcolen);
+	write_csr_int_srcsel(csr, bank_number, 0, state->iaintflagsrcsel0);
+	write_csr_exp_int_en(csr, bank_number, state->ringexpintenable);
+	write_csr_int_col_ctl(csr, bank_number, state->iaintcolctl);
+
+	/* Check that all ring statuses are restored into a saved state. */
+	tmp_val = read_csr_stat(csr, bank_number);
+	val = state->ringstat0;
+	if (tmp_val != val) {
+		pr_err("Fail to restore ringstat register. Expected 0x%x, but

actual is 0x%x\n",

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +8 -0
+		       tmp_val, val);
+		return -EINVAL;
+	}
+
+	tmp_val = read_csr_e_stat(csr, bank_number);
+	val = state->ringestat;
+	if (tmp_val != val) {
+		pr_err("Fail to restore ringestat register. Expected 0x%x, but

actual is 0x%x\n",

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +8 -0
+		       tmp_val, val);
+		return -EINVAL;
+	}
+
+	tmp_val = read_csr_ne_stat(csr, bank_number);
+	val = state->ringnestat;
+	if (tmp_val != val) {
+		pr_err("Fail to restore ringnestat register. Expected 0x%x, but

actual is 0x%x\n",

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +8 -0
+		       tmp_val, val);
+		return -EINVAL;
+	}
+
+	tmp_val = read_csr_nf_stat(csr, bank_number);
+	val = state->ringnfstat;
+	if (tmp_val != val) {
+		pr_err("Fail to restore ringnfstat register. Expected 0x%x, but

actual is 0x%x\n",

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +8 -0
+		       tmp_val, val);
+		return -EINVAL;
+	}
+
+	tmp_val = read_csr_f_stat(csr, bank_number);
+	val = state->ringfstat;
+	if (tmp_val != val) {
+		pr_err("Fail to restore ringfstat register. Expected 0x%x, but

actual is 0x%x\n",

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +8 -0
+		       tmp_val, val);
+		return -EINVAL;
+	}
+
+	tmp_val = read_csr_c_stat(csr, bank_number);
+	val = state->ringcstat0;
+	if (tmp_val != val) {
+		pr_err("Fail to restore ringcstat register. Expected 0x%x, but

actual is 0x%x\n",

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c int adf_gen4_ring_pair_reset(struct adf_accel_dev *accel_dev, u32 bank_number) · line 501

Quoted 10 lines
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +10 -0
+		       tmp_val, val);
+		return -EINVAL;
+	}
+
+	tmp_val = read_csr_exp_stat(csr, bank_number);
+	val = state->ringexpstat;
+	if (tmp_val && !val) {
+		pr_err("Bank was restored with exception: 0x%x\n", val);
+		return -EINVAL;
+	}

above checks could be wrapped in macros.

#3 lore raw

RE: [RFC 1/5] crypto: qat - add bank save/restore and RP drain

-----Original Message-----
From: Tian, Kevin <[email protected]>
Sent: Friday, August 4, 2023 3:51 PM
To: Zeng, Xin <[email protected]>; [email protected]; [email protected]
Cc: Cabiddu, Giovanni <[email protected]>; [email protected]; Wan, Siming <[email protected]>; Pankratov, Svyatoslav <[email protected]>; Zeng, Xin <[email protected]>
Subject: RE: [RFC 1/5] crypto: qat - add bank save/restore and RP drain

Quoted 18 lines

From: Xin Zeng <[email protected]>
Sent: Friday, June 30, 2023 9:13 PM

---
 .../intel/qat/qat_4xxx/adf_4xxx_hw_data.c     |   5 +-
 .../intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c   |   2 +-
 .../qat/qat_c3xxxvf/adf_c3xxxvf_hw_data.c     |   2 +-
 .../intel/qat/qat_c62x/adf_c62x_hw_data.c     |   2 +-
 .../intel/qat/qat_c62xvf/adf_c62xvf_hw_data.c |   2 +-
 .../intel/qat/qat_common/adf_accel_devices.h  |  60 ++-
 .../intel/qat/qat_common/adf_gen2_hw_data.c   |  17 +-
 .../intel/qat/qat_common/adf_gen2_hw_data.h   |  10 +-
 .../intel/qat/qat_common/adf_gen4_hw_data.c   | 362 +++++++++++++++++-
 .../intel/qat/qat_common/adf_gen4_hw_data.h   | 131 ++++++-
 .../intel/qat/qat_common/adf_transport.c      |  11 +-
 .../crypto/intel/qat/qat_common/adf_vf_isr.c  |   2 +-
 .../qat/qat_dh895xcc/adf_dh895xcc_hw_data.c   |   2 +-
 .../qat_dh895xccvf/adf_dh895xccvf_hw_data.c   |   2 +-
 14 files changed, 584 insertions(+), 26 deletions(-)

this could be split into 3 patches.

one is moving from hw_data->csr_ops to hw_data->csr_info. apply to all qat drivers.

the 2nd is adding new csr_ops.

the last one then covers bank save/restore.
->Will split it, thanks.

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c int adf_gen4_ring_pair_reset(struct adf_accel_dev *accel_dev, u32 bank_number) · line 396

Quoted 7 lines
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +7 -0
+
+#define ADF_RP_INT_SRC_SEL_F_RISE_MASK BIT(2) #define 
+ADF_RP_INT_SRC_SEL_F_FALL_MASK GENMASK(2, 0) static int 
+gen4_bank_state_restore(void __iomem *csr, u32 bank_number,
+				   struct bank_state *state, u32 num_rings,
+				   int tx_rx_gap)
+{

restore is the most tricky part. it's worth of some comments to help understand the flow, e.g. what is rx/tx layout, why there are multiple ring tails writes, etc. ->Will add some comments to explain.

Quoted 110 lines
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +6 -0
+	u32 val, tmp_val, i;
+
+	write_csr_ring_srv_arb_en(csr, bank_number, 0);
+
+	for (i = 0; i < num_rings; i++)
+		write_csr_ring_base(csr, bank_number, i, state-
Quoted 1 line

rings[i].base);

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +3 -0
+
+	for (i = 0; i < num_rings; i++)
+		write_csr_ring_config(csr, bank_number, i, state-
Quoted 1 line

rings[i].config);

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +8 -0
+
+	for (i = 0; i < num_rings / 2; i++) {
+		int tx = i * (tx_rx_gap + 1);
+		int rx = tx + tx_rx_gap;
+		u32 tx_idx = tx / ADF_RINGS_PER_INT_SRCSEL;
+		u32 rx_idx = rx / ADF_RINGS_PER_INT_SRCSEL;
+
+		write_csr_ring_head(csr, bank_number, tx, state-
Quoted 1 line

rings[tx].head);

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +6 -0
+
+		write_csr_ring_tail(csr, bank_number, tx, state->rings[tx].tail);
+
+		if (state->ringestat & (BIT(tx))) {
+			val = read_csr_int_srcsel(csr, bank_number, tx_idx);
+			val |= (ADF_RP_INT_SRC_SEL_F_RISE_MASK << (8 *

tx));

(quoted diff) +2 -0
+			write_csr_int_srcsel(csr, bank_number, tx_idx, val);
+			write_csr_ring_head(csr, bank_number, tx, state-
Quoted 1 line

rings[tx].head);

drivers/crypto/intel/qat/qat_common/adf_gen2_hw_data.c +3 -0
+		}
+
+		write_csr_ring_tail(csr, bank_number, rx, state-
Quoted 1 line

rings[rx].tail);

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +6 -0
+
+		val = read_csr_int_srcsel(csr, bank_number, rx_idx);
+		val |= (ADF_RP_INT_SRC_SEL_F_RISE_MASK << (8 * rx));
+		write_csr_int_srcsel(csr, bank_number, rx_idx, val);
+
+		write_csr_ring_head(csr, bank_number, rx, state-
Quoted 1 line

rings[rx].head);

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +7 -0
+
+		val = read_csr_int_srcsel(csr, bank_number, rx_idx);
+		val |= (ADF_RP_INT_SRC_SEL_F_FALL_MASK << (8 * rx));
+		write_csr_int_srcsel(csr, bank_number, rx_idx, val);
+
+		if (state->ringfstat & BIT(rx))
+			write_csr_ring_tail(csr, bank_number, rx, state-
Quoted 1 line

rings[rx].tail);

drivers/crypto/intel/qat/qat_common/adf_gen2_hw_data.c +3 -0
+	}
+
+	write_csr_int_flag_and_col(csr, bank_number, state-
Quoted 1 line

iaintflagandcolen);

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +11 -0
+	write_csr_int_en(csr, bank_number, state->iaintflagen);
+	write_csr_int_col_en(csr, bank_number, state->iaintcolen);
+	write_csr_int_srcsel(csr, bank_number, 0, state->iaintflagsrcsel0);
+	write_csr_exp_int_en(csr, bank_number, state->ringexpintenable);
+	write_csr_int_col_ctl(csr, bank_number, state->iaintcolctl);
+
+	/* Check that all ring statuses are restored into a saved state. */
+	tmp_val = read_csr_stat(csr, bank_number);
+	val = state->ringstat0;
+	if (tmp_val != val) {
+		pr_err("Fail to restore ringstat register. Expected 0x%x, but

actual is 0x%x\n",

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +8 -0
+		       tmp_val, val);
+		return -EINVAL;
+	}
+
+	tmp_val = read_csr_e_stat(csr, bank_number);
+	val = state->ringestat;
+	if (tmp_val != val) {
+		pr_err("Fail to restore ringestat register. Expected 0x%x, but

actual is 0x%x\n",

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +8 -0
+		       tmp_val, val);
+		return -EINVAL;
+	}
+
+	tmp_val = read_csr_ne_stat(csr, bank_number);
+	val = state->ringnestat;
+	if (tmp_val != val) {
+		pr_err("Fail to restore ringnestat register. Expected 0x%x, but

actual is 0x%x\n",

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +8 -0
+		       tmp_val, val);
+		return -EINVAL;
+	}
+
+	tmp_val = read_csr_nf_stat(csr, bank_number);
+	val = state->ringnfstat;
+	if (tmp_val != val) {
+		pr_err("Fail to restore ringnfstat register. Expected 0x%x, but

actual is 0x%x\n",

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +8 -0
+		       tmp_val, val);
+		return -EINVAL;
+	}
+
+	tmp_val = read_csr_f_stat(csr, bank_number);
+	val = state->ringfstat;
+	if (tmp_val != val) {
+		pr_err("Fail to restore ringfstat register. Expected 0x%x, but

actual is 0x%x\n",

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +8 -0
+		       tmp_val, val);
+		return -EINVAL;
+	}
+
+	tmp_val = read_csr_c_stat(csr, bank_number);
+	val = state->ringcstat0;
+	if (tmp_val != val) {
+		pr_err("Fail to restore ringcstat register. Expected 0x%x, but

actual is 0x%x\n",

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c int adf_gen4_ring_pair_reset(struct adf_accel_dev *accel_dev, u32 bank_number) · line 501

Quoted 10 lines
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +10 -0
+		       tmp_val, val);
+		return -EINVAL;
+	}
+
+	tmp_val = read_csr_exp_stat(csr, bank_number);
+	val = state->ringexpstat;
+	if (tmp_val && !val) {
+		pr_err("Bank was restored with exception: 0x%x\n", val);
+		return -EINVAL;
+	}

above checks could be wrapped in macros.
->If define macro as below, checkpatch.pl will appear warning-> WARNING: Macros with flow control statements should be avoided. So do we still need to change?
#define CHECK_STAT(expect, actual, csr_name) \

do { \
    u32 _expect = expect;                                \
    u32 _actual = actual;                                \
    char *_csr_name = csr_name;                            \
    if (_expect != _actual) {                            \
        pr_err("Fail to restore %s register. Expected 0x%x, but actual is 0x%x\n", \
            _csr_name, _expect, _actual);                    \
        return -EINVAL;                                \
    }                                        \
} while (0)
author
#4 lore raw

RFC 2/5 crypto: qat - add interface for live migration

In reply to Xin Zeng

Extend the driver with a new interface to be used for VF live migration. This will be called by the QAT extension in vfio-pci.

---
 drivers/crypto/intel/qat/qat_common/Makefile  |   2 +-
 .../intel/qat/qat_common/adf_accel_devices.h  |  13 +++
 .../crypto/intel/qat/qat_common/qat_vf_mig.c  | 106 ++++++++++++++++++
 include/linux/qat/qat_vf_mig.h                |  15 +++
 4 files changed, 135 insertions(+), 1 deletion(-)
 create mode 100644 drivers/crypto/intel/qat/qat_common/qat_vf_mig.c
 create mode 100644 include/linux/qat/qat_vf_mig.h
drivers/crypto/intel/qat/qat_common/Makefile +1 -1
index 38de3aba6e8c..3855f2fa5733 100644
@@ -33,4 +33,4 @@ intel_qat-$(CONFIG_DEBUG_FS) += adf_transport_debug.o \
 intel_qat-$(CONFIG_PCI_IOV) += adf_sriov.o adf_vf_isr.o adf_pfvf_utils.o \
 			       adf_pfvf_pf_msg.o adf_pfvf_pf_proto.o \
 			       adf_pfvf_vf_msg.o adf_pfvf_vf_proto.o \
-			       adf_gen2_pfvf.o adf_gen4_pfvf.o
+			       adf_gen2_pfvf.o adf_gen4_pfvf.o qat_vf_mig.o
drivers/crypto/intel/qat/qat_common/adf_accel_devices.h +13 -0
index 7fc2fd042916..adda2cac6af1 100644
@@ -217,6 +217,17 @@ struct adf_dc_ops {
 	void (*build_deflate_ctx)(void *ctx);
 };
 
+struct adf_vfmig_ops {
+	int (*init_device)(struct adf_accel_dev *accel_dev, u32 vf_nr);
+	void (*shutdown_device)(struct adf_accel_dev *accel_dev, u32 vf_nr);
+	int (*save_state)(struct adf_accel_dev *accel_dev, u32 vf_nr,
+			  u8 *buf, u64 buf_sz);
+	int (*load_state)(struct adf_accel_dev *accel_dev, u32 vf_nr,
+			  u8 *buf, u64 buf_sz);
+	int (*suspend_device)(struct adf_accel_dev *accel_dev, u32 vf_nr);
+	int (*resume_device)(struct adf_accel_dev *accel_dev, u32 vf_nr);
+};
+
 struct adf_hw_device_data {
 	struct adf_hw_device_class *dev_class;
 	u32 (*get_accel_mask)(struct adf_hw_device_data *self);
@@ -263,6 +274,7 @@ struct adf_hw_device_data {
 	struct adf_hw_csr_info csr_info;
 	struct adf_pfvf_ops pfvf_ops;
 	struct adf_dc_ops dc_ops;
+	struct adf_vfmig_ops vfmig_ops;
 	const char *fw_name;
 	const char *fw_mmp_name;
 	u32 fuses;
@@ -309,6 +321,7 @@ struct adf_hw_device_data {
 #define GET_CSR_OPS(accel_dev) (&(accel_dev)->hw_device->csr_info.csr_ops)
 #define GET_PFVF_OPS(accel_dev) (&(accel_dev)->hw_device->pfvf_ops)
 #define GET_DC_OPS(accel_dev) (&(accel_dev)->hw_device->dc_ops)
+#define GET_VFMIG_OPS(accel_dev) (&(accel_dev)->hw_device->vfmig_ops)
 #define accel_to_pci_dev(accel_ptr) accel_ptr->accel_pci_dev.pci_dev
 
 struct adf_admin_comms;
drivers/crypto/intel/qat/qat_common/qat_vf_mig.c new file +106 -0

106 lines not on this page · raw message

include/linux/qat/qat_vf_mig.h new file +15 -0
new file mode 100644
index 000000000000..09101be800ce
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright(c) 2023 Intel Corporation */
+
+#ifndef QAT_VF_MIG_H_
+#define QAT_VF_MIG_H_
+
+struct pci_dev;
+
+int qat_vfmig_init_device(struct pci_dev *pdev, u32 vf_nr);
+void qat_vfmig_shutdown_device(struct pci_dev *pdev, u32 vf_nr);
+int qat_vfmig_save_state(struct pci_dev *pdev, u32 vf_nr, u8 *buf, u64 buf_sz);
+int qat_vfmig_load_state(struct pci_dev *pdev, u32 vf_nr, u8 *buf, u64 buf_sz);
+int qat_vfmig_suspend_device(struct pci_dev *pdev, u32 vf_nr);
+int qat_vfmig_resume_device(struct pci_dev *pdev, u32 vf_nr);
+#endif /*QAT_VF_MIG_H_*/
Signature · 1 line
2.18.2
reviewer
#5 lore raw

RE: [RFC 2/5] crypto: qat - add interface for live migration

Quoted 7 lines

From: Xin Zeng <[email protected]>
Sent: Friday, June 30, 2023 9:13 PM

drivers/crypto/intel/qat/qat_common/qat_vf_mig.c new file +4 -0
+
+int qat_vfmig_suspend_device(struct pci_dev *pdev, u32 vf_nr)
+{
+	struct adf_accel_dev *accel_dev =

adf_devmgr_pci_to_accel_dev(pdev);

drivers/crypto/intel/qat/qat_common/qat_vf_mig.c line 54

Quoted 8 lines
drivers/crypto/intel/qat/qat_common/qat_vf_mig.c new file +8 -0
+
+	if (!accel_dev) {
+		dev_err(&pdev->dev, "Failed to find accel_dev\n");
+		return -ENODEV;
+	}
+
+	if (WARN_ON(!GET_VFMIG_OPS(accel_dev)->suspend_device))
+		return -EINVAL;

this and other warns should be done one-off at device registration point instead of letting it triggerable by every user ioctl.

author
#6 lore raw

RE: [RFC 2/5] crypto: qat - add interface for live migration

Thanks for the comments, Kevin.

Quoted 20 lines

-----Original Message-----
From: Tian, Kevin <[email protected]>
Sent: Friday, August 4, 2023 3:53 PM
To: Zeng, Xin <[email protected]>; [email protected]; [email protected]
Cc: Cabiddu, Giovanni <[email protected]>; [email protected]; Zeng, Xin <[email protected]>; Cao, Yahui <[email protected]>
Subject: RE: [RFC 2/5] crypto: qat - add interface for live migration

Quoted 10 lines

From: Xin Zeng <[email protected]>
Sent: Friday, June 30, 2023 9:13 PM

drivers/crypto/intel/qat/qat_common/qat_vf_mig.c new file +4 -0
+
+int qat_vfmig_suspend_device(struct pci_dev *pdev, u32 vf_nr)
+{
+	struct adf_accel_dev *accel_dev =

adf_devmgr_pci_to_accel_dev(pdev);

drivers/crypto/intel/qat/qat_common/qat_vf_mig.c new file +3 -0
+
+	if (!accel_dev) {
+		dev_err(&pdev->dev, "Failed to find accel_dev\n");
Quoted 8 lines

drivers/crypto/intel/qat/qat_common/qat_vf_mig.c line 54

Quoted 5 lines
drivers/crypto/intel/qat/qat_common/qat_vf_mig.c new file +5 -0
+		return -ENODEV;
+	}
+
+	if (WARN_ON(!GET_VFMIG_OPS(accel_dev)->suspend_device))
+		return -EINVAL;

this and other warns should be done one-off at device registration point instead of letting it triggerable by every user ioctl.

Ok, will address this.

author
#7 lore raw

RFC 3/5 units: Add HZ_PER_GHZ

In reply to Xin Zeng

From: Andy Shevchenko <[email protected]>

There is going to be a new user of the HZ_PER_GHZ definition besides possibly existing ones. Add that one to the header.

While at it, split Hz and kHz groups of the multipliers for better maintenance and readability.

---
 include/linux/units.h | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
include/linux/units.h +5 -1
index 2793a41e73a2..8d1dd5609906 100644
@@ -20,12 +20,16 @@
 #define PICO	1000000000000ULL
 #define FEMTO	1000000000000000ULL
 
+/* Hz based multipliers */
 #define NANOHZ_PER_HZ		1000000000UL
 #define MICROHZ_PER_HZ		1000000UL
 #define MILLIHZ_PER_HZ		1000UL
 #define HZ_PER_KHZ		1000UL
-#define KHZ_PER_MHZ		1000UL
 #define HZ_PER_MHZ		1000000UL
+#define HZ_PER_GHZ		1000000000UL
+
+/* kHz based multipliers */
+#define KHZ_PER_MHZ		1000UL
 
 #define MILLIWATT_PER_WATT	1000UL
 #define MICROWATT_PER_MILLIWATT	1000UL
Signature · 1 line
2.18.2
author
#8 lore raw

RFC 4/5 crypto: qat - implement interface for live migration

In reply to Xin Zeng

Add logic to implement interface for live migration for QAT GEN4 Virtual Functions (VFs).
This introduces a migration data manager which is used to hold the device state during migration.

The VF state is organized in a section hierarchy, as reported below:

    preamble | general state section | leaf state
             | MISC bar state section| leaf state
             | ETR bar state section | bank0 state section | leaf state
                                     | bank1 state section | leaf state
                                     | bank2 state section | leaf state
                                     | bank3 state section | leaf state
---
 .../intel/qat/qat_4xxx/adf_4xxx_hw_data.c     |   4 +-
 .../intel/qat/qat_4xxx/adf_4xxx_hw_data.h     |   3 +-
 drivers/crypto/intel/qat/qat_common/Makefile  |   2 +
 .../intel/qat/qat_common/adf_accel_devices.h  |   5 +
 .../intel/qat/qat_common/adf_gen4_hw_data.c   |  53 ++
 .../intel/qat/qat_common/adf_gen4_hw_data.h   |  21 +
 .../intel/qat/qat_common/adf_gen4_pfvf.c      |   7 +-
 .../intel/qat/qat_common/adf_gen4_pfvf.h      |   7 +
 .../intel/qat/qat_common/adf_gen4_vf_mig.c    | 609 ++++++++++++++++++
 .../intel/qat/qat_common/adf_mstate_mgr.c     | 267 ++++++++
 .../intel/qat/qat_common/adf_mstate_mgr.h     |  99 +++
 11 files changed, 1070 insertions(+), 7 deletions(-)
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.c
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.c
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.h
drivers/crypto/intel/qat/qat_4xxx/adf_4xxx_hw_data.c +3 -1
index 22fe4e6834c1..e859350bdfb0 100644
@@ -455,7 +455,7 @@ void adf_init_hw_data_4xxx(struct adf_hw_device_data *hw_data, u32 dev_id)
 	hw_data->dev_class = &adf_4xxx_class;
 	hw_data->instance_id = adf_4xxx_class.instances++;
 	hw_data->num_banks = ADF_4XXX_ETR_MAX_BANKS;
-	hw_data->num_banks_per_vf = ADF_4XXX_NUM_BANKS_PER_VF;
+	hw_data->num_banks_per_vf = ADF_GEN4_NUM_BANKS_PER_VF;
 	hw_data->num_rings_per_bank = ADF_4XXX_NUM_RINGS_PER_BANK;
 	hw_data->num_accel = ADF_4XXX_MAX_ACCELERATORS;
 	hw_data->num_engines = ADF_4XXX_MAX_ACCELENGINES;
@@ -487,6 +487,7 @@ void adf_init_hw_data_4xxx(struct adf_hw_device_data *hw_data, u32 dev_id)
 	hw_data->init_device = adf_init_device;
 	hw_data->reset_device = adf_reset_flr;
 	hw_data->admin_ae_mask = ADF_4XXX_ADMIN_AE_MASK;
+	hw_data->clock_frequency = ADF_4XXX_AE_FREQ;
 	switch (dev_id) {
 	case ADF_402XX_PCI_DEVICE_ID:
 		hw_data->fw_name = ADF_402XX_FW;
@@ -515,6 +516,7 @@ void adf_init_hw_data_4xxx(struct adf_hw_device_data *hw_data, u32 dev_id)
 	adf_gen4_init_hw_csr_ops(&hw_data->csr_info);
 	adf_gen4_init_pf_pfvf_ops(&hw_data->pfvf_ops);
 	adf_gen4_init_dc_ops(&hw_data->dc_ops);
+	adf_gen4_init_vf_mig_ops(&hw_data->vfmig_ops);
 }
 
 void adf_clean_hw_data_4xxx(struct adf_hw_device_data *hw_data)
drivers/crypto/intel/qat/qat_4xxx/adf_4xxx_hw_data.h +2 -1
index e5b314d2b60e..1f96c7f8ca6f 100644
@@ -3,6 +3,7 @@
 #ifndef ADF_4XXX_HW_DATA_H_
 #define ADF_4XXX_HW_DATA_H_
 
+#include <linux/units.h>
 #include <adf_accel_devices.h>
 
 /* PCIe configuration space */
@@ -26,6 +27,7 @@
 #define ADF_4XXX_ACCELERATORS_MASK	(0x1)
 #define ADF_4XXX_ACCELENGINES_MASK	(0x1FF)
 #define ADF_4XXX_ADMIN_AE_MASK		(0x100)
+#define ADF_4XXX_AE_FREQ		(1 * HZ_PER_GHZ)
 
 #define ADF_4XXX_ETR_MAX_BANKS		64
 
@@ -37,7 +39,6 @@
 
 /* Bank and ring configuration */
 #define ADF_4XXX_NUM_RINGS_PER_BANK	2
-#define ADF_4XXX_NUM_BANKS_PER_VF	4
 
 /* Arbiter configuration */
 #define ADF_4XXX_ARB_CONFIG			(BIT(31) | BIT(6) | BIT(0))
drivers/crypto/intel/qat/qat_common/Makefile +2 -0
index 3855f2fa5733..e0de2d0901f9 100644
@@ -14,9 +14,11 @@ intel_qat-objs := adf_cfg.o \
 	adf_gen2_hw_data.o \
 	adf_gen2_config.o \
 	adf_gen4_hw_data.o \
+	adf_gen4_vf_mig.o \
 	adf_gen4_pm.o \
 	adf_gen2_dc.o \
 	adf_gen4_dc.o \
+	adf_mstate_mgr.o \
 	qat_crypto.o \
 	qat_compression.o \
 	qat_comp_algs.o \
drivers/crypto/intel/qat/qat_common/adf_accel_devices.h +5 -0
index adda2cac6af1..b21a38e776a7 100644
@@ -339,6 +339,11 @@ struct adf_accel_vf_info {
 	u32 vf_nr;
 	bool init;
 	u8 vf_compat_ver;
+	/*
+	 * Private area used for device migration.
+	 * Memory allocation and free is managed by migration driver.
+	 */
+	void *mig_priv;
 };
 
 struct adf_dc_data {
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +53 -0

65 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.h +21 -0
index d2a4192aaa6d..29774841af39 100644
@@ -7,6 +7,9 @@
 #include "adf_accel_devices.h"
 #include "adf_cfg_common.h"
 
+/* Bank configuration */
+#define ADF_GEN4_NUM_BANKS_PER_VF	4
+
 /* Transport access */
 #define ADF_RINGS_PER_INT_SRCSEL	BIT(1)
 #define ADF_BANK_INT_SRC_SEL_MASK	0x44UL
@@ -237,6 +240,13 @@ static inline u64 read_base(void __iomem *csr_base_addr,
 /* Ring drain */
 #define ADF_WQM_CSR_RPRESETCTL_DRAIN	BIT(2)
 
+/* Ring interrupt*/
+#define ADF_COALESCED_POLL_TIMEOUT_US	(1 * USEC_PER_SEC)
+#define ADF_COALESCED_POLL_DELAY_US	1000
+#define ADF_WQM_CSR_RPINTSTS(bank)	(0x200000 + ((bank) << 12))
+
+#define ADF_WQM_CSR_RP_IDX_RX		1
+
 /* Error source registers */
 #define ADF_GEN4_ERRSOU0	(0x41A200)
 #define ADF_GEN4_ERRSOU1	(0x41A204)
@@ -251,13 +261,24 @@ static inline u64 read_base(void __iomem *csr_base_addr,
 
 #define ADF_GEN4_VFLNOTIFY	BIT(7)
 
+/* Number of heartbeat counter pairs */
+#define ADF_NUM_HB_CNT_PER_AE ADF_NUM_THREADS_PER_AE
+
+struct adf_gen4_vfmig {
+	u32 ringsrvarben[ADF_GEN4_NUM_BANKS_PER_VF];
+	void *mstate_mgr;
+};
+
 void adf_gen4_set_ssm_wdtimer(struct adf_accel_dev *accel_dev);
 void adf_gen4_init_hw_csr_ops(struct adf_hw_csr_info *csr_info);
 int adf_gen4_ring_pair_reset(struct adf_accel_dev *accel_dev, u32 bank_number);
 int adf_gen4_ring_pair_drain(struct adf_accel_dev *accel_dev, u32 bank_number,
 			     int timeout_us);
+int adf_gen4_rp_quiesce_coal_timer(struct adf_accel_dev *accel_dev,
+				   u32 bank_idx, int timeout_ms);
 int adf_gen4_bank_state_save(struct adf_accel_dev *accel_dev, u32 bank_number,
 			     struct bank_state *state);
 int adf_gen4_bank_state_restore(struct adf_accel_dev *accel_dev,
 				u32 bank_number, struct bank_state *state);
+void adf_gen4_init_vf_mig_ops(struct adf_vfmig_ops *vfmig_ops);
 #endif
drivers/crypto/intel/qat/qat_common/adf_gen4_pfvf.c +2 -5
index 8e8efe93f3ee..fe202ab3bc9d 100644
@@ -9,9 +9,6 @@
 #include "adf_pfvf_pf_proto.h"
 #include "adf_pfvf_utils.h"
 
-#define ADF_4XXX_PF2VM_OFFSET(i)	(0x40B010 + ((i) * 0x20))
-#define ADF_4XXX_VM2PF_OFFSET(i)	(0x40B014 + ((i) * 0x20))
-
 /* VF2PF interrupt source registers */
 #define ADF_4XXX_VM2PF_SOU		0x41A180
 #define ADF_4XXX_VM2PF_MSK		0x41A1C0
@@ -29,12 +26,12 @@ static const struct pfvf_csr_format csr_gen4_fmt = {
 
 static u32 adf_gen4_pf_get_pf2vf_offset(u32 i)
 {
-	return ADF_4XXX_PF2VM_OFFSET(i);
+	return ADF_GEN4_PF2VM_OFFSET(i);
 }
 
 static u32 adf_gen4_pf_get_vf2pf_offset(u32 i)
 {
-	return ADF_4XXX_VM2PF_OFFSET(i);
+	return ADF_GEN4_VM2PF_OFFSET(i);
 }
 
 static void adf_gen4_enable_vf2pf_interrupts(void __iomem *pmisc_addr, u32 vf_mask)
drivers/crypto/intel/qat/qat_common/adf_gen4_pfvf.h +7 -0
index 17d1b774d4a8..38edf02dbf8d 100644
@@ -5,6 +5,13 @@
 
 #include "adf_accel_devices.h"
 
+#define ADF_GEN4_PF2VM_OFFSET(i)	(0x40B010 + (i) * 0x20)
+#define ADF_GEN4_VM2PF_OFFSET(i)	(0x40B014 + (i) * 0x20)
+#define ADF_GEN4_VINTMSKPF2VM_OFFSET(i)	(0x40B00C + (i) * 0x20)
+#define ADF_GEN4_VINTSOUPF2VM_OFFSET(i)	(0x40B008 + (i) * 0x20)
+#define ADF_GEN4_VINTMSK_OFFSET(i)	(0x40B004 + (i) * 0x20)
+#define ADF_GEN4_VINTSOU_OFFSET(i)	(0x40B000 + (i) * 0x20)
+
 #ifdef CONFIG_PCI_IOV
 void adf_gen4_init_pf_pfvf_ops(struct adf_pfvf_ops *pfvf_ops);
 #else
drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.c new file +609 -0

609 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.c new file +267 -0

267 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.h new file +99 -0

99 lines not on this page · raw message

Signature · 1 line
2.18.2
reviewer
#9 lore raw

RE: [RFC 4/5] crypto: qat - implement interface for live migration

Quoted 16 lines

From: Xin Zeng <[email protected]>
Sent: Friday, June 30, 2023 9:13 PM

Add logic to implement interface for live migration for QAT GEN4 Virtual Functions (VFs).
This introduces a migration data manager which is used to hold the device state during migration.

The VF state is organized in a section hierarchy, as reported below:

    preamble | general state section | leaf state
             | MISC bar state section| leaf state
             | ETR bar state section | bank0 state section | leaf state
                                     | bank1 state section | leaf state
                                     | bank2 state section | leaf state
                                     | bank3 state section | leaf state
Quoted 3 lines

this is a big patch. Need reviewed-by from qat/crypto maintainers.

author
#10 lore raw

RE: [RFC 4/5] crypto: qat - implement interface for live migration

On Friday, August 4, 2023 3:56 PM, Tian, Kevin Kevin <[email protected]> Wrote:

Quoted 23 lines

To: Zeng, Xin <[email protected]>; [email protected]; [email protected]
Cc: Cabiddu, Giovanni <[email protected]>; [email protected]; Zeng, Xin <[email protected]>; Wan, Siming <[email protected]>
Subject: RE: [RFC 4/5] crypto: qat - implement interface for live migration

Quoted 16 lines

From: Xin Zeng <[email protected]>
Sent: Friday, June 30, 2023 9:13 PM

Add logic to implement interface for live migration for QAT GEN4 Virtual Functions (VFs).
This introduces a migration data manager which is used to hold the device state during migration.

The VF state is organized in a section hierarchy, as reported below:

    preamble | general state section | leaf state
             | MISC bar state section| leaf state
             | ETR bar state section | bank0 state section | leaf state
                                     | bank1 state section | leaf state
                                     | bank2 state section | leaf state
                                     | bank3 state section | leaf state
Quoted 5 lines
Quoted 3 lines

this is a big patch. Need reviewed-by from qat/crypto maintainers.

Sure, would do that for next version.

author
#11 lore raw

RFC 5/5 vfio/qat: Add vfio_pci driver for Intel QAT VF devices

In reply to Xin Zeng

Add vfio pci driver for Intel QAT VF devices.

This driver uses vfio_pci_core to register to the VFIO subsystem. It acts as a vfio agent and interacts with the QAT PF driver to implement VF live migration.

---
 drivers/vfio/pci/Kconfig                 |   2 +
 drivers/vfio/pci/Makefile                |   1 +
 drivers/vfio/pci/qat/Kconfig             |  13 +
 drivers/vfio/pci/qat/Makefile            |   4 +
 drivers/vfio/pci/qat/qat_vfio_pci_main.c | 518 +++++++++++++++++++++++
 5 files changed, 538 insertions(+)
 create mode 100644 drivers/vfio/pci/qat/Kconfig
 create mode 100644 drivers/vfio/pci/qat/Makefile
 create mode 100644 drivers/vfio/pci/qat/qat_vfio_pci_main.c
drivers/vfio/pci/Kconfig +2 -0
index f9d0c908e738..47c9773cf0c7 100644
@@ -59,4 +59,6 @@ source "drivers/vfio/pci/mlx5/Kconfig"
 
 source "drivers/vfio/pci/hisilicon/Kconfig"
 
+source "drivers/vfio/pci/qat/Kconfig"
+
 endif
drivers/vfio/pci/Makefile +1 -0
index 24c524224da5..dcc6366df8fa 100644
@@ -11,3 +11,4 @@ obj-$(CONFIG_VFIO_PCI) += vfio-pci.o
 obj-$(CONFIG_MLX5_VFIO_PCI)           += mlx5/
 
 obj-$(CONFIG_HISI_ACC_VFIO_PCI) += hisilicon/
+obj-$(CONFIG_QAT_VFIO_PCI) += qat/
drivers/vfio/pci/qat/Kconfig new file +13 -0
new file mode 100644
index 000000000000..38e5b4a0ca9c
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: GPL-2.0-only
+config QAT_VFIO_PCI
+	tristate "VFIO support for QAT VF PCI devices"
+	depends on X86
+	depends on VFIO_PCI_CORE
+	depends on CRYPTO_DEV_QAT
+	help
+	  This provides migration support for Intel(R) QAT Virtual Function
+	  using the VFIO framework.
+
+	  To compile this as a module, choose M here: the module
+	  will be called qat_vfio_pci. If you don't know what to do here,
+	  say N.
drivers/vfio/pci/qat/Makefile new file +4 -0
new file mode 100644
index 000000000000..106791887b91
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-$(CONFIG_QAT_VFIO_PCI) += qat_vfio_pci.o
+qat_vfio_pci-y := qat_vfio_pci_main.o
+
drivers/vfio/pci/qat/qat_vfio_pci_main.c new file +518 -0

518 lines not on this page · raw message

Signature · 1 line
2.18.2
#12 lore raw

Re: [RFC 5/5] vfio/qat: Add vfio_pci driver for Intel QAT VF devices

Please Cc the vfio-pci variant driver reviewers listed in MAINTAINERS. Also useful to put the subsystem maintainer on the cc rather than just the list.

Quoted 9 lines · On Fri, 30 Jun 2023 21:13:04 +0800 Xin Zeng <[email protected]> wrote:

Add vfio pci driver for Intel QAT VF devices.

This driver uses vfio_pci_core to register to the VFIO subsystem. It acts as a vfio agent and interacts with the QAT PF driver to implement VF live migration.

Quoted 6 lines
---
 drivers/vfio/pci/Kconfig                 |   2 +
 drivers/vfio/pci/Makefile                |   1 +
 drivers/vfio/pci/qat/Kconfig             |  13 +
 drivers/vfio/pci/qat/Makefile            |   4 +
 drivers/vfio/pci/qat/qat_vfio_pci_main.c | 518 +++++++++++++++++++++++

Rename to main.c.

Quoted 31 lines
 5 files changed, 538 insertions(+)
 create mode 100644 drivers/vfio/pci/qat/Kconfig
 create mode 100644 drivers/vfio/pci/qat/Makefile
 create mode 100644 drivers/vfio/pci/qat/qat_vfio_pci_main.c
drivers/vfio/pci/Kconfig +2 -0
index f9d0c908e738..47c9773cf0c7 100644
@@ -59,4 +59,6 @@ source "drivers/vfio/pci/mlx5/Kconfig"
 
 source "drivers/vfio/pci/hisilicon/Kconfig"
 
+source "drivers/vfio/pci/qat/Kconfig"
+
 endif
drivers/vfio/pci/Makefile +1 -0
index 24c524224da5..dcc6366df8fa 100644
@@ -11,3 +11,4 @@ obj-$(CONFIG_VFIO_PCI) += vfio-pci.o
 obj-$(CONFIG_MLX5_VFIO_PCI)           += mlx5/
 
 obj-$(CONFIG_HISI_ACC_VFIO_PCI) += hisilicon/
+obj-$(CONFIG_QAT_VFIO_PCI) += qat/
drivers/vfio/pci/qat/Kconfig new file +0 -0
new file mode 100644
index 000000000000..38e5b4a0ca9c

drivers/vfio/pci/qat/Kconfig line 4

Quoted 4 lines
drivers/vfio/pci/qat/Kconfig new file +4 -0
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: GPL-2.0-only
+config QAT_VFIO_PCI
+	tristate "VFIO support for QAT VF PCI devices"
+	depends on X86

What specific X86 dependency exists here? CRYPTO_DEV_QAT and the various versions of the QAT driver don't seem to have an explicit arch dependency, therefore this shouldn't either.

drivers/vfio/pci/qat/Kconfig line 5

Quoted 1 line
drivers/vfio/pci/qat/Kconfig new file +1 -0
+	depends on VFIO_PCI_CORE

select VFIO_PCI_CORE, this was updated for all vfio-pci variant drivers for v6.5.

Quoted 516 lines
drivers/vfio/pci/qat/Kconfig new file +8 -0
+	depends on CRYPTO_DEV_QAT
+	help
+	  This provides migration support for Intel(R) QAT Virtual Function
+	  using the VFIO framework.
+
+	  To compile this as a module, choose M here: the module
+	  will be called qat_vfio_pci. If you don't know what to do here,
+	  say N.
drivers/vfio/pci/qat/Makefile new file +4 -0
new file mode 100644
index 000000000000..106791887b91
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-$(CONFIG_QAT_VFIO_PCI) += qat_vfio_pci.o
+qat_vfio_pci-y := qat_vfio_pci_main.o
+
drivers/vfio/pci/qat/qat_vfio_pci_main.c new file +492 -0
new file mode 100644
index 000000000000..af971fd05fd2
@@ -0,0 +1,518 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright(c) 2023 Intel Corporation */
+#include <linux/anon_inodes.h>
+#include <linux/container_of.h>
+#include <linux/device.h>
+#include <linux/file.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/pci.h>
+#include <linux/sizes.h>
+#include <linux/types.h>
+#include <linux/uaccess.h>
+#include <linux/vfio_pci_core.h>
+#include <linux/qat/qat_vf_mig.h>
+
+struct qat_vf_mig_data {
+	u8 state[SZ_4K];
+};
+
+struct qat_vf_migration_file {
+	struct file *filp;
+	struct mutex lock; /* protect migration region context */
+	bool disabled;
+
+	size_t total_length;
+	struct qat_vf_mig_data mig_data;
+};
+
+struct qat_vf_core_device {
+	struct vfio_pci_core_device core_device;
+	struct pci_dev *parent;
+	int vf_id;
+
+	struct mutex state_mutex; /* protect migration state */
+	enum vfio_device_mig_state mig_state;
+	struct qat_vf_migration_file *resuming_migf;
+	struct qat_vf_migration_file *saving_migf;
+};
+
+static int qat_vf_init(struct qat_vf_core_device *qat_vdev)
+{
+	return qat_vfmig_init_device(qat_vdev->parent, qat_vdev->vf_id);
+}
+
+static void qat_vf_cleanup(struct qat_vf_core_device *qat_vdev)
+{
+	qat_vfmig_shutdown_device(qat_vdev->parent, qat_vdev->vf_id);
+}
+
+static int qat_vf_pci_open_device(struct vfio_device *core_vdev)
+{
+	int ret;
+	struct qat_vf_core_device *qat_vdev =
+		container_of(core_vdev, struct qat_vf_core_device,
+			     core_device.vdev);
+	struct vfio_pci_core_device *vdev = &qat_vdev->core_device;
+
+	ret = vfio_pci_core_enable(vdev);
+	if (ret)
+		return ret;
+
+	ret = qat_vf_init(qat_vdev);
+	if (ret) {
+		vfio_pci_core_disable(vdev);
+		return ret;
+	}
+	qat_vdev->mig_state = VFIO_DEVICE_STATE_RUNNING;
+
+	vfio_pci_core_finish_enable(vdev);
+
+	return 0;
+}
+
+static void qat_vf_disable_fd(struct qat_vf_migration_file *migf)
+{
+	mutex_lock(&migf->lock);
+	migf->disabled = true;
+	migf->total_length = 0;
+	migf->filp->f_pos = 0;
+	mutex_unlock(&migf->lock);
+}
+
+static void qat_vf_disable_fds(struct qat_vf_core_device *qat_vdev)
+{
+	if (qat_vdev->resuming_migf) {
+		qat_vf_disable_fd(qat_vdev->resuming_migf);
+		fput(qat_vdev->resuming_migf->filp);
+		qat_vdev->resuming_migf = NULL;
+	}
+
+	if (qat_vdev->saving_migf) {
+		qat_vf_disable_fd(qat_vdev->saving_migf);
+		fput(qat_vdev->saving_migf->filp);
+		qat_vdev->saving_migf = NULL;
+	}
+}
+
+static void qat_vf_pci_close_device(struct vfio_device *core_vdev)
+{
+	struct qat_vf_core_device *qat_vdev = container_of(core_vdev,
+			struct qat_vf_core_device, core_device.vdev);
+
+	qat_vf_cleanup(qat_vdev);
+	qat_vf_disable_fds(qat_vdev);
+	vfio_pci_core_close_device(core_vdev);
+}
+
+static int qat_vf_stop_device(struct qat_vf_core_device *qat_vdev)
+{
+	return qat_vfmig_suspend_device(qat_vdev->parent, qat_vdev->vf_id);
+}
+
+static ssize_t qat_vf_save_read(struct file *filp, char __user *buf,
+				size_t len, loff_t *pos)
+{
+	struct qat_vf_migration_file *migf = filp->private_data;
+	ssize_t done = 0;
+	loff_t *offs;
+	int ret;
+
+	if (pos)
+		return -ESPIPE;
+	offs = &filp->f_pos;
+
+	mutex_lock(&migf->lock);
+	if (*offs > migf->total_length || *offs < 0) {
+		done = -EINVAL;
+		goto out_unlock;
+	}
+
+	if (migf->disabled) {
+		done = -ENODEV;
+		goto out_unlock;
+	}
+
+	len = min_t(size_t, migf->total_length - *offs, len);
+	if (len) {
+		ret = copy_to_user(buf, &migf->mig_data + *offs, len);
+		if (ret) {
+			done = -EFAULT;
+			goto out_unlock;
+		}
+		*offs += len;
+		done = len;
+	}
+
+out_unlock:
+	mutex_unlock(&migf->lock);
+	return done;
+}
+
+static int qat_vf_release_file(struct inode *inode, struct file *filp)
+{
+	struct qat_vf_migration_file *migf = filp->private_data;
+
+	qat_vf_disable_fd(migf);
+	mutex_destroy(&migf->lock);
+	kfree(migf);
+
+	return 0;
+}
+
+static const struct file_operations qat_vf_save_fops = {
+	.owner = THIS_MODULE,
+	.read = qat_vf_save_read,
+	.release = qat_vf_release_file,
+	.llseek = no_llseek,
+};
+
+static int qat_vf_save_state(struct qat_vf_core_device *qat_vdev,
+			     struct qat_vf_migration_file *migf)
+{
+	struct qat_vf_mig_data *mig_data = &migf->mig_data;
+	int ret;
+
+	ret = qat_vfmig_save_state(qat_vdev->parent, qat_vdev->vf_id,
+				   mig_data->state,
+				   sizeof(mig_data->state));
+	if (ret)
+		return ret;
+
+	migf->total_length = sizeof(struct qat_vf_mig_data);
+
+	return 0;
+}
+
+static struct qat_vf_migration_file *
+qat_vf_save_device_data(struct qat_vf_core_device *qat_vdev)
+{
+	struct qat_vf_migration_file *migf;
+	int ret;
+
+	migf = kzalloc(sizeof(*migf), GFP_KERNEL);
+	if (!migf)
+		return ERR_PTR(-ENOMEM);
+
+	migf->filp = anon_inode_getfile("qat_vf_mig", &qat_vf_save_fops,
+					migf, O_RDONLY);
+	ret = PTR_ERR_OR_ZERO(migf->filp);
+	if (ret) {
+		kfree(migf);
+		return ERR_PTR(ret);
+	}
+
+	stream_open(migf->filp->f_inode, migf->filp);
+	mutex_init(&migf->lock);
+
+	ret = qat_vf_save_state(qat_vdev, migf);
+	if (ret) {
+		fput(migf->filp);
+		kfree(migf);
+		return ERR_PTR(ret);
+	}
+
+	return migf;
+}
+
+static ssize_t qat_vf_resume_write(struct file *filp, const char __user *buf,
+				   size_t len, loff_t *pos)
+{
+	struct qat_vf_migration_file *migf = filp->private_data;
+	loff_t requested_length;
+	ssize_t done = 0;
+	loff_t *offs;
+	int ret;
+
+	if (pos)
+		return -ESPIPE;
+	offs = &filp->f_pos;
+
+	if (*offs < 0 ||
+	    check_add_overflow((loff_t)len, *offs, &requested_length))
+		return -EOVERFLOW;
+
+	if (requested_length > sizeof(struct qat_vf_mig_data))
+		return -ENOMEM;
+
+	mutex_lock(&migf->lock);
+	if (migf->disabled) {
+		done = -ENODEV;
+		goto out_unlock;
+	}
+
+	ret = copy_from_user(&migf->mig_data + *offs, buf, len);
+	if (ret) {
+		done = -EFAULT;
+		goto out_unlock;
+	}
+	*offs += len;
+	done = len;
+	migf->total_length += len;
+
+out_unlock:
+	mutex_unlock(&migf->lock);
+	return done;
+}
+
+static const struct file_operations qat_vf_resume_fops = {
+	.owner = THIS_MODULE,
+	.write = qat_vf_resume_write,
+	.release = qat_vf_release_file,
+	.llseek = no_llseek,
+};
+
+static struct qat_vf_migration_file *
+qat_vf_resume_device_data(struct qat_vf_core_device *qat_vdev)
+{
+	struct qat_vf_migration_file *migf;
+	int ret;
+
+	migf = kzalloc(sizeof(*migf), GFP_KERNEL);
+	if (!migf)
+		return ERR_PTR(-ENOMEM);
+
+	migf->filp = anon_inode_getfile("qat_vf_mig", &qat_vf_resume_fops,
+					migf, O_WRONLY);
+	ret = PTR_ERR_OR_ZERO(migf->filp);
+	if (ret) {
+		kfree(migf);
+		return ERR_PTR(ret);
+	}
+	stream_open(migf->filp->f_inode, migf->filp);
+	mutex_init(&migf->lock);
+
+	return migf;
+}
+
+static int qat_vf_load_state(struct qat_vf_core_device *qat_vdev,
+			     struct qat_vf_migration_file *migf)
+{
+	struct qat_vf_mig_data *mig_data = &migf->mig_data;
+
+	return qat_vfmig_load_state(qat_vdev->parent, qat_vdev->vf_id,
+				    mig_data->state,
+				    sizeof(mig_data->state));
+}
+
+static int qat_vf_load_device_data(struct qat_vf_core_device *qat_vdev)
+{
+	return qat_vf_load_state(qat_vdev, qat_vdev->resuming_migf);
+}
+
+static int qat_vf_start_device(struct qat_vf_core_device *qat_vdev)
+{
+	return qat_vfmig_resume_device(qat_vdev->parent, qat_vdev->vf_id);
+}
+
+static struct file *qat_vf_pci_step_device_state(struct qat_vf_core_device *qat_vdev, u32 new)
+{
+	u32 cur = qat_vdev->mig_state;
+	int ret;
+
+	if (cur == VFIO_DEVICE_STATE_RUNNING  && new == VFIO_DEVICE_STATE_STOP) {
+		ret = qat_vf_stop_device(qat_vdev);
+		if (ret)
+			return ERR_PTR(ret);
+		return NULL;
+	}
+
+	if (cur == VFIO_DEVICE_STATE_STOP && new == VFIO_DEVICE_STATE_STOP_COPY) {
+		struct qat_vf_migration_file *migf;
+
+		migf = qat_vf_save_device_data(qat_vdev);
+		if (IS_ERR(migf))
+			return ERR_CAST(migf);
+		get_file(migf->filp);
+		qat_vdev->saving_migf = migf;
+		return migf->filp;
+	}
+
+	if (cur == VFIO_DEVICE_STATE_STOP_COPY && new == VFIO_DEVICE_STATE_STOP) {
+		qat_vf_disable_fds(qat_vdev);
+		return NULL;
+	}
+
+	if (cur == VFIO_DEVICE_STATE_STOP && new == VFIO_DEVICE_STATE_RESUMING) {
+		struct qat_vf_migration_file *migf;
+
+		migf = qat_vf_resume_device_data(qat_vdev);
+		if (IS_ERR(migf))
+			return ERR_CAST(migf);
+		get_file(migf->filp);
+		qat_vdev->resuming_migf = migf;
+		return migf->filp;
+	}
+
+	if (cur == VFIO_DEVICE_STATE_RESUMING && new == VFIO_DEVICE_STATE_STOP) {
+		ret = qat_vf_load_device_data(qat_vdev);
+		if (ret)
+			return ERR_PTR(ret);
+
+		qat_vf_disable_fds(qat_vdev);
+		return NULL;
+	}
+
+	if (cur == VFIO_DEVICE_STATE_STOP && new == VFIO_DEVICE_STATE_RUNNING) {
+		qat_vf_start_device(qat_vdev);
+		return NULL;
+	}
+
+	/* vfio_mig_get_next_state() does not use arcs other than the above */
+	WARN_ON(true);
+	return ERR_PTR(-EINVAL);
+}
+
+static struct file *qat_vf_pci_set_device_state(struct vfio_device *vdev,
+						enum vfio_device_mig_state new_state)
+{
+	struct qat_vf_core_device *qat_vdev = container_of(vdev,
+			struct qat_vf_core_device, core_device.vdev);
+	enum vfio_device_mig_state next_state;
+	struct file *res = NULL;
+	int ret;
+
+	mutex_lock(&qat_vdev->state_mutex);
+	while (new_state != qat_vdev->mig_state) {
+		ret = vfio_mig_get_next_state(vdev, qat_vdev->mig_state,
+					      new_state, &next_state);
+		if (ret) {
+			res = ERR_PTR(-EINVAL);
+			break;
+		}
+
+		res = qat_vf_pci_step_device_state(qat_vdev, next_state);
+		if (IS_ERR(res))
+			break;
+		qat_vdev->mig_state = next_state;
+		if (WARN_ON(res && new_state != qat_vdev->mig_state)) {
+			fput(res);
+			res = ERR_PTR(-EINVAL);
+			break;
+		}
+	}
+	mutex_unlock(&qat_vdev->state_mutex);
+
+	return res;
+}
+
+static int qat_vf_pci_get_device_state(struct vfio_device *vdev,
+				       enum vfio_device_mig_state *curr_state)
+{
+	struct qat_vf_core_device *qat_vdev = container_of(vdev,
+			struct qat_vf_core_device, core_device.vdev);
+
+	mutex_lock(&qat_vdev->state_mutex);
+	*curr_state = qat_vdev->mig_state;
+	mutex_unlock(&qat_vdev->state_mutex);
+
+	return 0;
+}
+
+static int qat_vf_pci_get_data_size(struct vfio_device *vdev,
+				    unsigned long *stop_copy_length)
+{
+	*stop_copy_length = sizeof(struct qat_vf_mig_data);
+	return 0;
+}
+
+static const struct vfio_migration_ops qat_vf_pci_mig_ops = {
+	.migration_set_state = qat_vf_pci_set_device_state,
+	.migration_get_state = qat_vf_pci_get_device_state,
+	.migration_get_data_size = qat_vf_pci_get_data_size,
+};
+
+static int qat_vf_pci_init_dev(struct vfio_device *core_vdev)
+{
+	struct qat_vf_core_device *qat_vdev = container_of(core_vdev,
+			struct qat_vf_core_device, core_device.vdev);
+
+	mutex_init(&qat_vdev->state_mutex);
+
+	core_vdev->migration_flags = VFIO_MIGRATION_STOP_COPY;
+	core_vdev->mig_ops = &qat_vf_pci_mig_ops;
+
+	return vfio_pci_core_init_dev(core_vdev);
+}
+
+static const struct vfio_device_ops qat_vf_pci_ops = {
+	.name = "qat-vf-vfio-pci",
+	.init = qat_vf_pci_init_dev,
+	.release = vfio_pci_core_release_dev,
+	.open_device = qat_vf_pci_open_device,
+	.close_device = qat_vf_pci_close_device,
+	.ioctl = vfio_pci_core_ioctl,
+	.read = vfio_pci_core_read,
+	.write = vfio_pci_core_write,
+	.mmap = vfio_pci_core_mmap,
+	.request = vfio_pci_core_request,
+	.match = vfio_pci_core_match,
+	.bind_iommufd = vfio_iommufd_physical_bind,
+	.unbind_iommufd = vfio_iommufd_physical_unbind,
+	.attach_ioas = vfio_iommufd_physical_attach_ioas,
+};
+
+static int
+qat_vf_vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
+{
+	struct device *dev = &pdev->dev;
+	struct qat_vf_core_device *qat_vdev;
+	int ret;
+
+	qat_vdev = vfio_alloc_device(qat_vf_core_device, core_device.vdev, dev, &qat_vf_pci_ops);
+	if (IS_ERR(qat_vdev))
+		return PTR_ERR(qat_vdev);
+
+	qat_vdev->vf_id = pci_iov_vf_id(pdev);
+	qat_vdev->parent = pdev->physfn;
+	if (!qat_vdev->parent || qat_vdev->vf_id < 0)
+		return -EINVAL;
+
+	pci_set_drvdata(pdev, &qat_vdev->core_device);
+	ret = vfio_pci_core_register_device(&qat_vdev->core_device);
+	if (ret)
+		goto out_put_device;
+
+	return 0;
+
+out_put_device:
+	vfio_put_device(&qat_vdev->core_device.vdev);
+	return ret;
+}
+
+static struct qat_vf_core_device *qat_vf_drvdata(struct pci_dev *pdev)
+{
+	struct vfio_pci_core_device *core_device = pci_get_drvdata(pdev);
+
+	return container_of(core_device, struct qat_vf_core_device, core_device);
+}
+
+static void qat_vf_vfio_pci_remove(struct pci_dev *pdev)

drivers/vfio/pci/qat/qat_vfio_pci_main.c line 502

Quoted 10 lines
drivers/vfio/pci/qat/qat_vfio_pci_main.c new file +10 -0
+{
+	struct qat_vf_core_device *qat_vdev = qat_vf_drvdata(pdev);
+
+	vfio_pci_core_unregister_device(&qat_vdev->core_device);
+	vfio_put_device(&qat_vdev->core_device.vdev);
+}
+
+static const struct pci_device_id qat_vf_vfio_pci_table[] = {
+	/* Intel QAT GEN4 4xxx VF device */
+	{ PCI_DRIVER_OVERRIDE_DEVICE_VFIO(PCI_VENDOR_ID_INTEL, 0x4941) },

Should this driver depend on CRYPTO_DEV_QAT_4XXX if that's the only supported PF driver?

Quoted 6 lines
drivers/vfio/pci/qat/qat_vfio_pci_main.c new file +6 -0
+	{}
+};
+MODULE_DEVICE_TABLE(pci, qat_vf_vfio_pci_table);
+
+static struct pci_driver qat_vf_vfio_pci_driver = {
+	.name = "qat_vfio_pci",

drivers/vfio/pci/qat/qat_vfio_pci_main.c line 518

Quoted 10 lines
drivers/vfio/pci/qat/qat_vfio_pci_main.c new file +10 -0
+	.id_table = qat_vf_vfio_pci_table,
+	.probe = qat_vf_vfio_pci_probe,
+	.remove = qat_vf_vfio_pci_remove,
+	.driver_managed_dma = true,
+};
+module_pci_driver(qat_vf_vfio_pci_driver)
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Intel Corporation");
+MODULE_DESCRIPTION("QAT VFIO PCI - VFIO PCI driver with live migration support for Intel(R) QAT device family");

Or at least one version of the QAT device family ;) Thanks,

Alex

author
#13 lore raw

RE: [RFC 5/5] vfio/qat: Add vfio_pci driver for Intel QAT VF devices

Thanks for the comments, Alex.

Quoted 8 lines · Alex Williamson
Quoted 5 lines
 drivers/vfio/pci/Kconfig                 |   2 +
 drivers/vfio/pci/Makefile                |   1 +
 drivers/vfio/pci/qat/Kconfig             |  13 +
 drivers/vfio/pci/qat/Makefile            |   4 +
 drivers/vfio/pci/qat/qat_vfio_pci_main.c | 518

+++++++++++++++++++++++

Rename to main.c.

Will do in next version.

Quoted 24 lines
Quoted 24 lines
 5 files changed, 538 insertions(+)
 create mode 100644 drivers/vfio/pci/qat/Kconfig
 create mode 100644 drivers/vfio/pci/qat/Makefile
 create mode 100644 drivers/vfio/pci/qat/qat_vfio_pci_main.c
drivers/vfio/pci/Kconfig +2 -0
index f9d0c908e738..47c9773cf0c7 100644
@@ -59,4 +59,6 @@ source "drivers/vfio/pci/mlx5/Kconfig"
 
 source "drivers/vfio/pci/hisilicon/Kconfig"
 
+source "drivers/vfio/pci/qat/Kconfig"
+
 endif
drivers/vfio/pci/Makefile +0 -0
index 24c524224da5..dcc6366df8fa 100644
@@ -11,3 +11,4 @@ obj-$(CONFIG_VFIO_PCI) += vfio-pci.o
 obj-$(CONFIG_MLX5_VFIO_PCI)           += mlx5/
 
Quoted 16 lines

drivers/vfio/pci/qat/Kconfig line 4

Quoted 12 lines
drivers/vfio/pci/Makefile +1 -0
 obj-$(CONFIG_HISI_ACC_VFIO_PCI) += hisilicon/
+obj-$(CONFIG_QAT_VFIO_PCI) += qat/
drivers/vfio/pci/qat/Kconfig new file +4 -0
new file mode 100644
index 000000000000..38e5b4a0ca9c
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: GPL-2.0-only
+config QAT_VFIO_PCI
+	tristate "VFIO support for QAT VF PCI devices"
+	depends on X86

What specific X86 dependency exists here? CRYPTO_DEV_QAT and the various versions of the QAT driver don't seem to have an explicit arch dependency, therefore this shouldn't either.

You are right. Will remove it.

Quoted 5 lines

drivers/vfio/pci/qat/Kconfig line 5

Quoted 2 lines
drivers/vfio/pci/qat/Kconfig new file +1 -0
+	depends on VFIO_PCI_CORE

select VFIO_PCI_CORE, this was updated for all vfio-pci variant drivers for v6.5.

Will update it.

Quoted 52 lines

Review

Quoted 3 lines
drivers/vfio/pci/qat/Kconfig new file +1 -0
+
(quoted diff) +0 -0

b/drivers/vfio/pci/qat/qat_vfio_pci_main.c

Quoted 36 lines

new file mode 100644
index 000000000000..af971fd05fd2

drivers/vfio/pci/qat/qat_vfio_pci_main.c new file +31 -0
@@ -0,0 +1,518 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright(c) 2023 Intel Corporation */
+#include <linux/anon_inodes.h>
+#include <linux/container_of.h>
+#include <linux/device.h>
+#include <linux/file.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/pci.h>
+#include <linux/sizes.h>
+#include <linux/types.h>
+#include <linux/uaccess.h>
+#include <linux/vfio_pci_core.h>
+#include <linux/qat/qat_vf_mig.h>
+
+struct qat_vf_mig_data {
+	u8 state[SZ_4K];
+};
+
+struct qat_vf_migration_file {
+	struct file *filp;
+	struct mutex lock; /* protect migration region context */
+	bool disabled;
+
+	size_t total_length;
+	struct qat_vf_mig_data mig_data;
+};
+
+static void qat_vf_vfio_pci_remove(struct pci_dev *pdev)

drivers/vfio/pci/qat/qat_vfio_pci_main.c line 41

Quoted 10 lines
drivers/vfio/pci/qat/qat_vfio_pci_main.c new file +10 -0
+{
+	struct qat_vf_core_device *qat_vdev = qat_vf_drvdata(pdev);
+
+	vfio_pci_core_unregister_device(&qat_vdev->core_device);
+	vfio_put_device(&qat_vdev->core_device.vdev);
+}
+
+static const struct pci_device_id qat_vf_vfio_pci_table[] = {
+	/* Intel QAT GEN4 4xxx VF device */
+	{ PCI_DRIVER_OVERRIDE_DEVICE_VFIO(PCI_VENDOR_ID_INTEL,

0x4941) },

Quoted 2 lines

Should this driver depend on CRYPTO_DEV_QAT_4XXX if that's the only supported PF driver?

This module has not any dependency to QAT_4XXX module at build time, but it indeed has implicit dependency on QAT_4XXX runtime to enable SRIOV and complete the QAT 4xxx VF migration, do you think we still need to put this dependency explicitly in Kconfig?

Quoted 19 lines
Quoted 7 lines
drivers/vfio/pci/qat/qat_vfio_pci_main.c new file +6 -0
+	{}
+};
+MODULE_DEVICE_TABLE(pci, qat_vf_vfio_pci_table);
+
+static struct pci_driver qat_vf_vfio_pci_driver = {
+	.name = "qat_vfio_pci",

drivers/vfio/pci/qat/qat_vfio_pci_main.c

Quoted 10 lines
drivers/vfio/pci/qat/qat_vfio_pci_main.c new file +10 -0
+	.id_table = qat_vf_vfio_pci_table,
+	.probe = qat_vf_vfio_pci_probe,
+	.remove = qat_vf_vfio_pci_remove,
+	.driver_managed_dma = true,
+};
+module_pci_driver(qat_vf_vfio_pci_driver)
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Intel Corporation");
+MODULE_DESCRIPTION("QAT VFIO PCI - VFIO PCI driver with live

migration support for Intel(R) QAT device family");

Quoted 1 line

Or at least one version of the QAT device family ;) Thanks,

Will update it.

Quoted 2 lines

Alex

#14 lore raw

Re: [RFC 5/5] vfio/qat: Add vfio_pci driver for Intel QAT VF devices

Quoted 116 lines · Xin

Thanks for the comments, Alex.

Quoted 8 lines · Alex Williamson
Quoted 5 lines
 drivers/vfio/pci/Kconfig                 |   2 +
 drivers/vfio/pci/Makefile                |   1 +
 drivers/vfio/pci/qat/Kconfig             |  13 +
 drivers/vfio/pci/qat/Makefile            |   4 +
 drivers/vfio/pci/qat/qat_vfio_pci_main.c | 518  

+++++++++++++++++++++++

Rename to main.c.

Will do in next version.

Quoted 24 lines
Quoted 24 lines
  
 5 files changed, 538 insertions(+)
 create mode 100644 drivers/vfio/pci/qat/Kconfig
 create mode 100644 drivers/vfio/pci/qat/Makefile
 create mode 100644 drivers/vfio/pci/qat/qat_vfio_pci_main.c
drivers/vfio/pci/Kconfig +2 -0
index f9d0c908e738..47c9773cf0c7 100644
@@ -59,4 +59,6 @@ source "drivers/vfio/pci/mlx5/Kconfig"
 
 source "drivers/vfio/pci/hisilicon/Kconfig"
 
+source "drivers/vfio/pci/qat/Kconfig"
+
 endif
drivers/vfio/pci/Makefile +0 -0
index 24c524224da5..dcc6366df8fa 100644
@@ -11,3 +11,4 @@ obj-$(CONFIG_VFIO_PCI) += vfio-pci.o
 obj-$(CONFIG_MLX5_VFIO_PCI)           += mlx5/
 
Quoted 16 lines

drivers/vfio/pci/qat/Kconfig line 4

Quoted 12 lines
drivers/vfio/pci/Makefile +1 -0
 obj-$(CONFIG_HISI_ACC_VFIO_PCI) += hisilicon/
+obj-$(CONFIG_QAT_VFIO_PCI) += qat/
drivers/vfio/pci/qat/Kconfig new file +4 -0
new file mode 100644
index 000000000000..38e5b4a0ca9c
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: GPL-2.0-only
+config QAT_VFIO_PCI
+	tristate "VFIO support for QAT VF PCI devices"
+	depends on X86  

What specific X86 dependency exists here? CRYPTO_DEV_QAT and the various versions of the QAT driver don't seem to have an explicit arch dependency, therefore this shouldn't either.

You are right. Will remove it.

Quoted 5 lines

drivers/vfio/pci/qat/Kconfig line 5

Quoted 2 lines
drivers/vfio/pci/qat/Kconfig new file +1 -0
+	depends on VFIO_PCI_CORE  

select VFIO_PCI_CORE, this was updated for all vfio-pci variant drivers for v6.5.

Will update it.

Quoted 52 lines

Review

Quoted 3 lines
drivers/vfio/pci/qat/Kconfig new file +1 -0
+
(quoted diff) +0 -0

b/drivers/vfio/pci/qat/qat_vfio_pci_main.c

Quoted 36 lines

new file mode 100644
index 000000000000..af971fd05fd2

drivers/vfio/pci/qat/qat_vfio_pci_main.c new file +31 -0
@@ -0,0 +1,518 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright(c) 2023 Intel Corporation */
+#include <linux/anon_inodes.h>
+#include <linux/container_of.h>
+#include <linux/device.h>
+#include <linux/file.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/pci.h>
+#include <linux/sizes.h>
+#include <linux/types.h>
+#include <linux/uaccess.h>
+#include <linux/vfio_pci_core.h>
+#include <linux/qat/qat_vf_mig.h>
+
+struct qat_vf_mig_data {
+	u8 state[SZ_4K];
+};
+
+struct qat_vf_migration_file {
+	struct file *filp;
+	struct mutex lock; /* protect migration region context */
+	bool disabled;
+
+	size_t total_length;
+	struct qat_vf_mig_data mig_data;
+};
+
+static void qat_vf_vfio_pci_remove(struct pci_dev *pdev)

drivers/vfio/pci/qat/qat_vfio_pci_main.c line 41

Quoted 10 lines
drivers/vfio/pci/qat/qat_vfio_pci_main.c new file +10 -0
+{
+	struct qat_vf_core_device *qat_vdev = qat_vf_drvdata(pdev);
+
+	vfio_pci_core_unregister_device(&qat_vdev->core_device);
+	vfio_put_device(&qat_vdev->core_device.vdev);
+}
+
+static const struct pci_device_id qat_vf_vfio_pci_table[] = {
+	/* Intel QAT GEN4 4xxx VF device */
+	{ PCI_DRIVER_OVERRIDE_DEVICE_VFIO(PCI_VENDOR_ID_INTEL,  

0x4941) },

Quoted 6 lines
Quoted 2 lines

Should this driver depend on CRYPTO_DEV_QAT_4XXX if that's the only supported PF driver?

This module has not any dependency to QAT_4XXX module at build time, but it indeed has implicit dependency on QAT_4XXX runtime to enable SRIOV and complete the QAT 4xxx VF migration, do you think we still need to put this dependency explicitly in Kconfig?

What benefit does it serve a user to be able to build this module if the runtime dependency isn't present in the kernel? We have COMPILE_TEST to support build time regression testing. Thanks,

Alex

reviewer
#15 lore raw

RE: [RFC 5/5] vfio/qat: Add vfio_pci driver for Intel QAT VF devices

In reply to Xin Zeng
Quoted 9 lines

From: Xin Zeng <[email protected]>
Sent: Friday, June 30, 2023 9:13 PM

Add vfio pci driver for Intel QAT VF devices.

This driver uses vfio_pci_core to register to the VFIO subsystem. It acts as a vfio agent and interacts with the QAT PF driver to implement VF live migration.

this lacks of P2P support and .err_handler.

author
#16 lore raw

RE: [RFC 5/5] vfio/qat: Add vfio_pci driver for Intel QAT VF devices

On Friday, August 4, 2023 4:09 PM, Tian, Kevin <[email protected]> Wrote:

Quoted 8 lines
Quoted 6 lines

Add vfio pci driver for Intel QAT VF devices.

This driver uses vfio_pci_core to register to the VFIO subsystem. It acts as a vfio agent and interacts with the QAT PF driver to implement VF live migration.

this lacks of P2P support and .err_handler.

Thanks for the comment, will implement these in later version.

#17 lore raw

Re: [RFC 5/5] vfio/qat: Add vfio_pci driver for Intel QAT VF devices

In reply to Xin Zeng
Quoted 23 lines · Xin Zeng

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Add vfio pci driver for Intel QAT VF devices.

This driver uses vfio_pci_core to register to the VFIO subsystem. It acts as a vfio agent and interacts with the QAT PF driver to implement VF live migration.

---
  drivers/vfio/pci/Kconfig                 |   2 +
  drivers/vfio/pci/Makefile                |   1 +
  drivers/vfio/pci/qat/Kconfig             |  13 +
  drivers/vfio/pci/qat/Makefile            |   4 +
  drivers/vfio/pci/qat/qat_vfio_pci_main.c | 518 +++++++++++++++++++++++
  5 files changed, 538 insertions(+)

create mode 100644 drivers/vfio/pci/qat/Kconfig create mode 100644 drivers/vfio/pci/qat/Makefile create mode 100644 drivers/vfio/pci/qat/qat_vfio_pci_main.c

[...]

Quoted 6 lines
drivers/vfio/pci/qat/qat_vfio_pci_main.c new file +6 -0
+
+static int
+qat_vf_vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
+{
+       struct device *dev = &pdev->dev;
+       struct qat_vf_core_device *qat_vdev;

drivers/vfio/pci/qat/qat_vfio_pci_main.c line 471

Quoted 10 lines
drivers/vfio/pci/qat/qat_vfio_pci_main.c new file +10 -0
+       int ret;
+
+       qat_vdev = vfio_alloc_device(qat_vf_core_device, core_device.vdev, dev, &qat_vf_pci_ops);
+       if (IS_ERR(qat_vdev))
+               return PTR_ERR(qat_vdev);
+
+       qat_vdev->vf_id = pci_iov_vf_id(pdev);
+       qat_vdev->parent = pdev->physfn;
+       if (!qat_vdev->parent || qat_vdev->vf_id < 0)
+               return -EINVAL;

Since vfio_alloc_device() was successful you need to call vfio_put_device() in this error case as well.

Quoted 50 lines
drivers/vfio/pci/qat/qat_vfio_pci_main.c new file +47 -1
+
+       pci_set_drvdata(pdev, &qat_vdev->core_device);
+       ret = vfio_pci_core_register_device(&qat_vdev->core_device);
+       if (ret)
+               goto out_put_device;
+
+       return 0;
+
+out_put_device:
+       vfio_put_device(&qat_vdev->core_device.vdev);
+       return ret;
+}
+
+static struct qat_vf_core_device *qat_vf_drvdata(struct pci_dev *pdev)
+{
+       struct vfio_pci_core_device *core_device = pci_get_drvdata(pdev);
+
+       return container_of(core_device, struct qat_vf_core_device, core_device);
+}
+
+static void qat_vf_vfio_pci_remove(struct pci_dev *pdev)
+{
+       struct qat_vf_core_device *qat_vdev = qat_vf_drvdata(pdev);
+
+       vfio_pci_core_unregister_device(&qat_vdev->core_device);
+       vfio_put_device(&qat_vdev->core_device.vdev);
+}
+
+static const struct pci_device_id qat_vf_vfio_pci_table[] = {
+       /* Intel QAT GEN4 4xxx VF device */
+       { PCI_DRIVER_OVERRIDE_DEVICE_VFIO(PCI_VENDOR_ID_INTEL, 0x4941) },
+       {}
+};
+MODULE_DEVICE_TABLE(pci, qat_vf_vfio_pci_table);
+
+static struct pci_driver qat_vf_vfio_pci_driver = {
+       .name = "qat_vfio_pci",
+       .id_table = qat_vf_vfio_pci_table,
+       .probe = qat_vf_vfio_pci_probe,
+       .remove = qat_vf_vfio_pci_remove,
+       .driver_managed_dma = true,
+};
+module_pci_driver(qat_vf_vfio_pci_driver)
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Intel Corporation");
+MODULE_DESCRIPTION("QAT VFIO PCI - VFIO PCI driver with live migration support for Intel(R) QAT device family");
--

2.18.2

author
#18 lore raw

RE: [RFC 5/5] vfio/qat: Add vfio_pci driver for Intel QAT VF devices

Quoted 15 lines · Brett Creeley

drivers/vfio/pci/qat/qat_vfio_pci_main.c

Quoted 6 lines
drivers/vfio/pci/qat/qat_vfio_pci_main.c new file +6 -0
+{
+       struct device *dev = &pdev->dev;
+       struct qat_vf_core_device *qat_vdev;
+       int ret;
+
+       qat_vdev = vfio_alloc_device(qat_vf_core_device, core_device.vdev,

dev, &qat_vf_pci_ops);

drivers/vfio/pci/qat/qat_vfio_pci_main.c line 471

Quoted 7 lines
drivers/vfio/pci/qat/qat_vfio_pci_main.c new file +7 -0
+       if (IS_ERR(qat_vdev))
+               return PTR_ERR(qat_vdev);
+
+       qat_vdev->vf_id = pci_iov_vf_id(pdev);
+       qat_vdev->parent = pdev->physfn;
+       if (!qat_vdev->parent || qat_vdev->vf_id < 0)
+               return -EINVAL;
Quoted 2 lines

Since vfio_alloc_device() was successful you need to call vfio_put_device() in this error case as well.

Good catch, will fix it.
Thanks for the comment, Brett.

Quoted 30 lines
Quoted 8 lines
drivers/vfio/pci/qat/qat_vfio_pci_main.c new file +8 -0
+
+       pci_set_drvdata(pdev, &qat_vdev->core_device);
+       ret = vfio_pci_core_register_device(&qat_vdev->core_device);
+       if (ret)
+               goto out_put_device;
+
+       return 0;
+

drivers/vfio/pci/qat/qat_vfio_pci_main.c

Quoted 10 lines
drivers/vfio/pci/qat/qat_vfio_pci_main.c new file +10 -0
+out_put_device:
+       vfio_put_device(&qat_vdev->core_device.vdev);
+       return ret;
+}
+
+static struct qat_vf_core_device *qat_vf_drvdata(struct pci_dev *pdev)
+{
+       struct vfio_pci_core_device *core_device = pci_get_drvdata(pdev);
+
+       return container_of(core_device, struct qat_vf_core_device,

core_device);

drivers/vfio/pci/qat/qat_vfio_pci_main.c

Quoted 7 lines
drivers/vfio/pci/qat/qat_vfio_pci_main.c new file +6 -0
+}
+
+
 
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Intel Corporation");
+MODULE_DESCRIPTION("QAT VFIO PCI - VFIO PCI driver with live

migration support for Intel(R) QAT device family");

Quoted 3 lines

--
2.18.2