crypto: qat - enable QAT GEN4 SRIOV VF live migration for QAT GEN4
Subsystems (5)
v2 cover crypto: qat - enable QAT GEN4 SRIOV VF live migration for QAT GEN4
This set enables live migration for Intel QAT GEN4 SRIOV Virtual Functions (VFs).
It is composed of 10 patches. Patch 1~6 refactor the original QAT PF driver implementation which will be reused by the following patches. Patch 7 introduces the logic to the QAT PF driver that allows to save and restore the state of a bank (a QAT VF is a wrapper around banks) and drain a ring pair. Patch 8 adds the QAT PF driver a set of interfaces to allow to save and restore the state of a VF that will be called by the module qat_vfio_pci which will be introduced in the last patch. Patch 9 implements the defined device interfaces. The last one adds a vfio pci extension specific for QAT which intercepts the vfio device operations for a QAT VF to allow live migration.
Here are the steps required to test the live migration of a QAT GEN4 VF:
1. Bind one or more QAT GEN4 VF devices to the module qat_vfio_pci.ko
2. Assign the VFs to the virtual machine and enable device live migration
3. Run a workload using a QAT VF inside the VM, for example using qatlib
(https://github.com/intel/qatlib)
4. Migrate the VM from the source node to a destination node
Changes in v2 from v1:
https://lore.kernel.org/all/[email protected]
- Add VFIO_MIGRATION_PRE_COPY support (Alex) - Remove unnecessary module dependancy in Kconfig (Alex) - Use direct function calls instead of function pointers in qat vfio
variant driver (Jason)
- Address the comments including uncessary pointer check and kfree, missing lock and direct use of pci_iov_vf_id (Shameer)
- Change CHECK_STAT macro to avoid repeat comparison (Kamlesh)
Changes in v1 from RFC:
https://lore.kernel.org/all/[email protected]
- Address comments including the right module dependancy in Kconfig, source file name and module description (Alex)
- Added PCI error handler and P2P state handler (Suggested by Kevin) - Refactor the state check duing loading ring state (Kevin) - Fix missed call to vfio_put_device in the error case (Breet) - Migrate the shadow states in PF driver - Rebase on top of 6.8-rc1
Giovanni Cabiddu (2):
crypto: qat - adf_get_etr_base() helper
crypto: qat - relocate CSR access code
Siming Wan (3):
crypto: qat - rename get_sla_arr_of_type()
crypto: qat - expand CSR operations for QAT GEN4 devices
crypto: qat - add bank save and restore flows
Xin Zeng (5):
crypto: qat - relocate and rename 4xxx PF2VM definitions
crypto: qat - move PFVF compat checker to a function
crypto: qat - add interface for live migration
crypto: qat - implement interface for live migration vfio/qat: Add vfio_pci driver for Intel QAT VF devices
MAINTAINERS | 8 + .../intel/qat/qat_420xx/adf_420xx_hw_data.c | 3 + .../intel/qat/qat_4xxx/adf_4xxx_hw_data.c | 5 + .../intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c | 1 + .../qat/qat_c3xxxvf/adf_c3xxxvf_hw_data.c | 1 + .../intel/qat/qat_c62x/adf_c62x_hw_data.c | 1 + .../intel/qat/qat_c62xvf/adf_c62xvf_hw_data.c | 1 + drivers/crypto/intel/qat/qat_common/Makefile | 6 +- .../intel/qat/qat_common/adf_accel_devices.h | 88 ++ .../intel/qat/qat_common/adf_common_drv.h | 10 + .../qat/qat_common/adf_gen2_hw_csr_data.c | 101 ++ .../qat/qat_common/adf_gen2_hw_csr_data.h | 86 ++ .../intel/qat/qat_common/adf_gen2_hw_data.c | 97 -- .../intel/qat/qat_common/adf_gen2_hw_data.h | 76 -- .../qat/qat_common/adf_gen4_hw_csr_data.c | 231 ++++ .../qat/qat_common/adf_gen4_hw_csr_data.h | 188 +++ .../intel/qat/qat_common/adf_gen4_hw_data.c | 380 +++++-- .../intel/qat/qat_common/adf_gen4_hw_data.h | 127 +-- .../intel/qat/qat_common/adf_gen4_pfvf.c | 8 +- .../intel/qat/qat_common/adf_gen4_vf_mig.c | 1003 +++++++++++++++++ .../intel/qat/qat_common/adf_gen4_vf_mig.h | 10 + .../intel/qat/qat_common/adf_mstate_mgr.c | 318 ++++++ .../intel/qat/qat_common/adf_mstate_mgr.h | 89 ++ .../intel/qat/qat_common/adf_pfvf_pf_proto.c | 8 +- .../intel/qat/qat_common/adf_pfvf_utils.h | 11 + drivers/crypto/intel/qat/qat_common/adf_rl.c | 10 +- drivers/crypto/intel/qat/qat_common/adf_rl.h | 2 + .../crypto/intel/qat/qat_common/adf_sriov.c | 7 +- .../intel/qat/qat_common/adf_transport.c | 4 +- .../crypto/intel/qat/qat_common/qat_mig_dev.c | 130 +++ .../qat/qat_dh895xcc/adf_dh895xcc_hw_data.c | 1 + .../qat_dh895xccvf/adf_dh895xccvf_hw_data.c | 1 + drivers/vfio/pci/Kconfig | 2 + drivers/vfio/pci/Makefile | 2 + drivers/vfio/pci/intel/qat/Kconfig | 12 + drivers/vfio/pci/intel/qat/Makefile | 3 + drivers/vfio/pci/intel/qat/main.c | 688 +++++++++++ include/linux/qat/qat_mig_dev.h | 31 + 38 files changed, 3363 insertions(+), 387 deletions(-) create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen2_hw_csr_data.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen2_hw_csr_data.h create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_hw_csr_data.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_hw_csr_data.h create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.h create mode 100644 drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.h create mode 100644 drivers/crypto/intel/qat/qat_common/qat_mig_dev.c create mode 100644 drivers/vfio/pci/intel/qat/Kconfig create mode 100644 drivers/vfio/pci/intel/qat/Makefile create mode 100644 drivers/vfio/pci/intel/qat/main.c create mode 100644 include/linux/qat/qat_mig_dev.h
base-commit: a821f4cadc99272701016641bc821182fdfca289
Signature · 1 line
2.18.2
PATCH v2 1/10 crypto: qat - adf_get_etr_base() helper
From: Giovanni Cabiddu <[email protected]>
Add and use the new helper function adf_get_etr_base() which retrieves the virtual address of the ring bar.
This will be used extensively when adding support for Live Migration.
--- drivers/crypto/intel/qat/qat_common/adf_common_drv.h | 10 ++++++++++ drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c | 4 +--- drivers/crypto/intel/qat/qat_common/adf_transport.c | 4 +--- 3 files changed, 12 insertions(+), 6 deletions(-)
drivers/crypto/intel/qat/qat_common/adf_common_drv.h +10 -0
@@ -248,6 +248,16 @@ static inline void __iomem *adf_get_pmisc_base(struct adf_accel_dev *accel_dev)return pmisc->virt_addr; } +static inline void __iomem *adf_get_etr_base (struct adf_accel_dev *accel_dev) +{ +struct adf_hw_device_data *hw_data = accel_dev->hw_device; +struct adf_bar *etr; + + etr = &GET_BARS (accel_dev)[hw_data->get_etr_bar_id (hw_data)]; + +return etr->virt_addr; +} +static inline void __iomem *adf_get_aram_base (struct adf_accel_dev *accel_dev) {struct adf_hw_device_data *hw_data = accel_dev->hw_device;
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +1 -3
@@ -321,8 +321,7 @@ static int reset_ring_pair(void __iomem *csr, u32 bank_number)int adf_gen4_ring_pair_reset (struct adf_accel_dev *accel_dev,u32 bank_number) {struct adf_hw_device_data *hw_data = accel_dev->hw_device; -u32 etr_bar_id = hw_data->get_etr_bar_id (hw_data); -void __iomem *csr; +void __iomem *csr =adf_get_etr_base (accel_dev);int ret;if (bank_number >= hw_data->num_banks)
@@ -331,7 +330,6 @@ int adf_gen4_ring_pair_reset(struct adf_accel_dev *accel_dev, u32 bank_number)dev_dbg (&GET_DEV (accel_dev),"ring pair reset for bank:%d\n" , bank_number); - csr = (&GET_BARS (accel_dev)[etr_bar_id])->virt_addr; ret =reset_ring_pair (csr, bank_number);if (ret) dev_err(&GET_DEV(accel_dev),
drivers/crypto/intel/qat/qat_common/adf_transport.c +1 -3
@@ -474,7 +474,6 @@ static int adf_init_bank(struct adf_accel_dev *accel_dev,int adf_init_etr_data (struct adf_accel_dev *accel_dev) {struct adf_etr_data *etr_data; -struct adf_hw_device_data *hw_data = accel_dev->hw_device;void __iomem *csr_addr;u32 size;u32 num_banks =0 ;
@@ -495,8 +494,7 @@ int adf_init_etr_data(struct adf_accel_dev *accel_dev) } accel_dev->transport = etr_data; - i = hw_data->get_etr_bar_id (hw_data); - csr_addr = accel_dev->accel_pci_dev.pci_bars[i].virt_addr; + csr_addr =adf_get_etr_base (accel_dev);/* accel_dev->debugfs_dir should always be non-NULL here */ etr_data->debug = debugfs_create_dir("transport",
Signature · 1 line
2.18.2
PATCH v2 2/10 crypto: qat - relocate and rename 4xxx PF2VM definitions
In reply to Xin ZengMove and rename ADF_4XXX_PF2VM_OFFSET and ADF_4XXX_VM2PF_OFFSET to ADF_GEN4_PF2VM_OFFSET and ADF_GEN4_VM2PF_OFFSET respectively. These definitions are moved from adf_gen4_pfvf.c to adf_gen4_hw_data.h as they are specific to GEN4 and not just to qat_4xxx.
This change is made in anticipation of their use in live migration.
This does not introduce any functional change.
--- drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.h | 4 ++++ drivers/crypto/intel/qat/qat_common/adf_gen4_pfvf.c | 8 +++----- 2 files changed, 7 insertions(+), 5 deletions(-)
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.h +4 -0
@@ -197,6 +197,10 @@ do { \/* Arbiter threads mask with error value */ #define ADF_GEN4_ENA_THD_MASK_ERROR GENMASK(ADF_NUM_THREADS_PER_AE, 0) +/* PF2VM communication channel */ +#define ADF_GEN4_PF2VM_OFFSET (i) (0x40B010 + (i) * 0x20) +#define ADF_GEN4_VM2PF_OFFSET (i) (0x40B014 + (i) * 0x20) +void adf_gen4_set_ssm_wdtimer (struct adf_accel_dev *accel_dev);enum icp_qat_gen4_slice_mask {
drivers/crypto/intel/qat/qat_common/adf_gen4_pfvf.c +3 -5
@@ -6,12 +6,10 @@#include "adf_accel_devices.h" #include "adf_common_drv.h" #include "adf_gen4_pfvf.h" +#include "adf_gen4_hw_data.h" #include "adf_pfvf_pf_proto.h" #include "adf_pfvf_utils.h" -#define ADF_4XXX_PF2VM_OFFSET (i) (0x40B010 + ((i) * 0x20)) -#define ADF_4XXX_VM2PF_OFFSET (i) (0x40B014 + ((i) * 0x20)) -/* VF2PF interrupt source registers */ #define ADF_4XXX_VM2PF_SOU 0x41A180#define ADF_4XXX_VM2PF_MSK 0x41A1C0
@@ -29,12 +27,12 @@ static const struct pfvf_csr_format csr_gen4_fmt = {static u32 adf_gen4_pf_get_pf2vf_offset (u32 i) { -return ADF_4XXX_PF2VM_OFFSET (i); +return ADF_GEN4_PF2VM_OFFSET (i); }static u32 adf_gen4_pf_get_vf2pf_offset (u32 i) { -return ADF_4XXX_VM2PF_OFFSET (i); +return ADF_GEN4_VM2PF_OFFSET (i); }static void adf_gen4_enable_vf2pf_interrupts (void __iomem *pmisc_addr,u32 vf_mask)
Signature · 1 line
2.18.2
PATCH v2 3/10 crypto: qat - move PFVF compat checker to a function
In reply to Xin ZengMove the code that implements VF version compatibility on the PF side to a separate function so that it can be reused when doing VM live migration.
This does not introduce any functional change.
--- .../crypto/intel/qat/qat_common/adf_pfvf_pf_proto.c | 8 +------- drivers/crypto/intel/qat/qat_common/adf_pfvf_utils.h | 11 +++++++++++ 2 files changed, 12 insertions(+), 7 deletions(-)
drivers/crypto/intel/qat/qat_common/adf_pfvf_pf_proto.c +1 -7
@@ -242,13 +242,7 @@ static int adf_handle_vf2pf_msg(struct adf_accel_dev *accel_dev, u8 vf_nr,"VersionRequest received from VF%d (vers %d) to PF (vers %d)\n" , vf_nr, vf_compat_ver, ADF_PFVF_COMPAT_THIS_VERSION); -if (vf_compat_ver ==0 ) - compat =ADF_PF2VF_VF_INCOMPATIBLE ; -else if (vf_compat_ver <=ADF_PFVF_COMPAT_THIS_VERSION ) - compat =ADF_PF2VF_VF_COMPATIBLE ; -else - compat =ADF_PF2VF_VF_COMPAT_UNKNOWN ; - + compat =adf_vf_compat_checker (vf_compat_ver); vf_info->vf_compat_ver = vf_compat_ver; resp->type =ADF_PF2VF_MSGTYPE_VERSION_RESP ;
drivers/crypto/intel/qat/qat_common/adf_pfvf_utils.h +11 -0
@@ -28,4 +28,15 @@ u32 adf_pfvf_csr_msg_of(struct adf_accel_dev *accel_dev, struct pfvf_message msgstruct pfvf_message adf_pfvf_message_of (struct adf_accel_dev *accel_dev,u32 raw_msg,const struct pfvf_csr_format *fmt); +static inline u8 adf_vf_compat_checker (u8 vf_compat_ver) +{ +if (vf_compat_ver ==0 ) +return ADF_PF2VF_VF_INCOMPATIBLE ; + +if (vf_compat_ver <=ADF_PFVF_COMPAT_THIS_VERSION ) +return ADF_PF2VF_VF_COMPATIBLE ; + +return ADF_PF2VF_VF_COMPAT_UNKNOWN ; +} +#endif /* ADF_PFVF_UTILS_H */
Signature · 1 line
2.18.2
PATCH v2 4/10 crypto: qat - relocate CSR access code
In reply to Xin ZengFrom: Giovanni Cabiddu <[email protected]>
As the common hw_data files are growing and the adf_hw_csr_ops is going to be extended with new operations, move all logic related to ring CSRs to the newly created adf_gen[2|4]_hw_csr_data.[c|h] files.
This does not introduce any functional change.
--- .../intel/qat/qat_420xx/adf_420xx_hw_data.c | 1 + .../intel/qat/qat_4xxx/adf_4xxx_hw_data.c | 1 + .../intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c | 1 + .../qat/qat_c3xxxvf/adf_c3xxxvf_hw_data.c | 1 + .../intel/qat/qat_c62x/adf_c62x_hw_data.c | 1 + .../intel/qat/qat_c62xvf/adf_c62xvf_hw_data.c | 1 + drivers/crypto/intel/qat/qat_common/Makefile | 2 + .../qat/qat_common/adf_gen2_hw_csr_data.c | 101 ++++++++++++++++++ .../qat/qat_common/adf_gen2_hw_csr_data.h | 86 +++++++++++++++ .../intel/qat/qat_common/adf_gen2_hw_data.c | 97 ----------------- .../intel/qat/qat_common/adf_gen2_hw_data.h | 76 ------------- .../qat/qat_common/adf_gen4_hw_csr_data.c | 101 ++++++++++++++++++ .../qat/qat_common/adf_gen4_hw_csr_data.h | 97 +++++++++++++++++ .../intel/qat/qat_common/adf_gen4_hw_data.c | 97 ----------------- .../intel/qat/qat_common/adf_gen4_hw_data.h | 94 +--------------- .../qat/qat_dh895xcc/adf_dh895xcc_hw_data.c | 1 + .../qat_dh895xccvf/adf_dh895xccvf_hw_data.c | 1 + 17 files changed, 397 insertions(+), 362 deletions(-) create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen2_hw_csr_data.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen2_hw_csr_data.h create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_hw_csr_data.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_hw_csr_data.h
drivers/crypto/intel/qat/qat_420xx/adf_420xx_hw_data.c +1 -0
@@ -10,6 +10,7 @@#include <adf_fw_config.h> #include <adf_gen4_config.h> #include <adf_gen4_dc.h> +#include <adf_gen4_hw_csr_data.h> #include <adf_gen4_hw_data.h> #include <adf_gen4_pfvf.h> #include <adf_gen4_pm.h>
drivers/crypto/intel/qat/qat_4xxx/adf_4xxx_hw_data.c +1 -0
@@ -10,6 +10,7 @@#include <adf_fw_config.h> #include <adf_gen4_config.h> #include <adf_gen4_dc.h> +#include <adf_gen4_hw_csr_data.h> #include <adf_gen4_hw_data.h> #include <adf_gen4_pfvf.h> #include <adf_gen4_pm.h>
drivers/crypto/intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c +1 -0
@@ -6,6 +6,7 @@#include <adf_common_drv.h> #include <adf_gen2_config.h> #include <adf_gen2_dc.h> +#include <adf_gen2_hw_csr_data.h> #include <adf_gen2_hw_data.h> #include <adf_gen2_pfvf.h> #include "adf_c3xxx_hw_data.h"
drivers/crypto/intel/qat/qat_c3xxxvf/adf_c3xxxvf_hw_data.c +1 -0
@@ -4,6 +4,7 @@#include <adf_common_drv.h> #include <adf_gen2_config.h> #include <adf_gen2_dc.h> +#include <adf_gen2_hw_csr_data.h> #include <adf_gen2_hw_data.h> #include <adf_gen2_pfvf.h> #include <adf_pfvf_vf_msg.h>
drivers/crypto/intel/qat/qat_c62x/adf_c62x_hw_data.c +1 -0
@@ -6,6 +6,7 @@#include <adf_common_drv.h> #include <adf_gen2_config.h> #include <adf_gen2_dc.h> +#include <adf_gen2_hw_csr_data.h> #include <adf_gen2_hw_data.h> #include <adf_gen2_pfvf.h> #include "adf_c62x_hw_data.h"
drivers/crypto/intel/qat/qat_c62xvf/adf_c62xvf_hw_data.c +1 -0
@@ -4,6 +4,7 @@#include <adf_common_drv.h> #include <adf_gen2_config.h> #include <adf_gen2_dc.h> +#include <adf_gen2_hw_csr_data.h> #include <adf_gen2_hw_data.h> #include <adf_gen2_pfvf.h> #include <adf_pfvf_vf_msg.h>
drivers/crypto/intel/qat/qat_common/Makefile +2 -0
@@ -14,9 +14,11 @@ intel_qat-objs := adf_cfg.o \ adf_hw_arbiter.o \ adf_sysfs.o \ adf_sysfs_ras_counters.o \ + adf_gen2_hw_csr_data.o \ adf_gen2_hw_data.o \ adf_gen2_config.o \ adf_gen4_config.o \ + adf_gen4_hw_csr_data.o \ adf_gen4_hw_data.o \ adf_gen4_pm.o \ adf_gen2_dc.o \
drivers/crypto/intel/qat/qat_common/adf_gen2_hw_csr_data.c new file +101 -0
101 lines not on this page · raw message
drivers/crypto/intel/qat/qat_common/adf_gen2_hw_csr_data.h new file +86 -0
86 lines not on this page · raw message
drivers/crypto/intel/qat/qat_common/adf_gen2_hw_data.c +0 -97
103 lines not on this page · raw message
drivers/crypto/intel/qat/qat_common/adf_gen2_hw_data.h +0 -76
97 lines not on this page · raw message
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_csr_data.c new file +101 -0
101 lines not on this page · raw message
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_csr_data.h new file +97 -0
97 lines not on this page · raw message
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +0 -97
103 lines not on this page · raw message
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.h +2 -92
111 lines not on this page · raw message
drivers/crypto/intel/qat/qat_dh895xcc/adf_dh895xcc_hw_data.c +1 -0
@@ -5,6 +5,7 @@#include <adf_common_drv.h> #include <adf_gen2_config.h> #include <adf_gen2_dc.h> +#include <adf_gen2_hw_csr_data.h> #include <adf_gen2_hw_data.h> #include <adf_gen2_pfvf.h> #include "adf_dh895xcc_hw_data.h"
drivers/crypto/intel/qat/qat_dh895xccvf/adf_dh895xccvf_hw_data.c +1 -0
@@ -4,6 +4,7 @@#include <adf_common_drv.h> #include <adf_gen2_config.h> #include <adf_gen2_dc.h> +#include <adf_gen2_hw_csr_data.h> #include <adf_gen2_hw_data.h> #include <adf_gen2_pfvf.h> #include <adf_pfvf_vf_msg.h>
Signature · 1 line
2.18.2
PATCH v2 5/10 crypto: qat - rename get_sla_arr_of_type()
In reply to Xin ZengFrom: Siming Wan <[email protected]>
The function get_sla_arr_of_type() returns a pointer to an SLA type specific array.
Rename it and expose it as it will be used externally to this module.
This does not introduce any functional change.
--- drivers/crypto/intel/qat/qat_common/adf_rl.c | 10 +++++----- drivers/crypto/intel/qat/qat_common/adf_rl.h | 2 ++ 2 files changed, 7 insertions(+), 5 deletions(-)
drivers/crypto/intel/qat/qat_common/adf_rl.c +5 -5
@@ -183,14 +183,14 @@ static enum adf_cfg_service_type srv_to_cfg_svc_type(enum adf_base_services rl_s }/** -* get_sla_arr_of_type() - Returns a pointer to SLA type specific array +* adf_rl_get_sla_arr_of_type() - Returns a pointer to SLA type specific array * @rl_data: pointer to ratelimiting data * @type: SLA type * @sla_arr: pointer to variable where requested pointer will be stored * * Return: Max number of elements allowed for the returned array */ -static u32 get_sla_arr_of_type (struct adf_rl *rl_data,enum rl_node_type type, +u32 adf_rl_get_sla_arr_of_type (struct adf_rl *rl_data,enum rl_node_type type,struct rl_sla ***sla_arr) {switch (type) {
@@ -778,7 +778,7 @@ static void clear_sla(struct adf_rl *rl_data, struct rl_sla *sla) rp_in_use[sla->ring_pairs_ids[i]] = false;update_budget (sla, old_cir, true); -get_sla_arr_of_type (rl_data, sla->type, &sla_type_arr); +adf_rl_get_sla_arr_of_type (rl_data, sla->type, &sla_type_arr);assign_node_to_parent (rl_data->accel_dev, sla, true);adf_rl_send_admin_delete_msg (rl_data->accel_dev, node_id, sla->type);mark_rps_usage (sla, rl_data->rp_in_use, false);
@@ -875,7 +875,7 @@ static int add_update_sla(struct adf_accel_dev *accel_dev,if (!is_update) {mark_rps_usage (sla, rl_data->rp_in_use, true); -get_sla_arr_of_type (rl_data, sla->type, &sla_type_arr); +adf_rl_get_sla_arr_of_type (rl_data, sla->type, &sla_type_arr); sla_type_arr[sla->node_id] = sla; rl_data->sla[sla->sla_id] = sla; }
@@ -1065,7 +1065,7 @@ void adf_rl_remove_sla_all(struct adf_accel_dev *accel_dev, bool incl_default)/* Unregister and remove all SLAs */ for (j =RL_LEAF ; j >= end_type; j--) { - max_id =get_sla_arr_of_type (rl_data, j, &sla_type_arr); + max_id =adf_rl_get_sla_arr_of_type (rl_data, j, &sla_type_arr);for (i =0 ; i < max_id; i++) {if (!sla_type_arr[i])
drivers/crypto/intel/qat/qat_common/adf_rl.h +2 -0
@@ -151,6 +151,8 @@ struct rl_sla {u16 ring_pairs_cnt; }; +u32 adf_rl_get_sla_arr_of_type (struct adf_rl *rl_data,enum rl_node_type type, +struct rl_sla ***sla_arr);int adf_rl_add_sla (struct adf_accel_dev *accel_dev,struct adf_rl_sla_input_data *sla_in); int adf_rl_update_sla(struct adf_accel_dev *accel_dev,
Signature · 1 line
2.18.2
PATCH v2 6/10 crypto: qat - expand CSR operations for QAT GEN4 devices
In reply to Xin ZengFrom: Siming Wan <[email protected]>
Extend the CSR operations for QAT GEN4 devices to allow saving and restoring the rings state.
The new operations will be used as a building block for implementing the state save and restore of Virtual Functions necessary for VM live migration.
This adds the following operations:
- read ring status register
- read ring underflow/overflow status register
- read ring nearly empty status register
- read ring nearly full status register
- read ring full status register
- read ring complete status register
- read ring exception status register
- read/write ring exception interrupt mask register
- read ring configuration register
- read ring base register
- read/write ring interrupt enable register
- read ring interrupt flag register
- read/write ring interrupt source select register
- read ring coalesced interrupt enable register
- read ring coalesced interrupt control register
- read ring flag and coalesced interrupt enable register
- read ring service arbiter enable register
- get ring coalesced interrupt control enable mask
--- .../intel/qat/qat_common/adf_accel_devices.h | 27 ++++ .../qat/qat_common/adf_gen4_hw_csr_data.c | 130 ++++++++++++++++++ .../qat/qat_common/adf_gen4_hw_csr_data.h | 93 ++++++++++++- 3 files changed, 249 insertions(+), 1 deletion(-)
drivers/crypto/intel/qat/qat_common/adf_accel_devices.h +27 -0
@@ -150,22 +150,49 @@ struct adf_hw_csr_ops { u32 ring);void (*write_csr_ring_tail)(void __iomem *csr_base_addr,u32 bank,u32 ring,u32 value); +u32 (*read_csr_stat)(void __iomem *csr_base_addr,u32 bank); +u32 (*read_csr_uo_stat)(void __iomem *csr_base_addr,u32 bank);u32 (*read_csr_e_stat)(void __iomem *csr_base_addr,u32 bank); +u32 (*read_csr_ne_stat)(void __iomem *csr_base_addr,u32 bank); +u32 (*read_csr_nf_stat)(void __iomem *csr_base_addr,u32 bank); +u32 (*read_csr_f_stat)(void __iomem *csr_base_addr,u32 bank); +u32 (*read_csr_c_stat)(void __iomem *csr_base_addr,u32 bank); +u32 (*read_csr_exp_stat)(void __iomem *csr_base_addr,u32 bank); +u32 (*read_csr_exp_int_en)(void __iomem *csr_base_addr,u32 bank); +void (*write_csr_exp_int_en)(void __iomem *csr_base_addr,u32 bank, +u32 value); +u32 (*read_csr_ring_config)(void __iomem *csr_base_addr,u32 bank, +u32 ring);void (*write_csr_ring_config)(void __iomem *csr_base_addr,u32 bank,u32 ring,u32 value); +dma_addr_t (*read_csr_ring_base)(void __iomem *csr_base_addr,u32 bank, +u32 ring);void (*write_csr_ring_base)(void __iomem *csr_base_addr,u32 bank,u32 ring,dma_addr_t addr); +u32 (*read_csr_int_en)(void __iomem *csr_base_addr,u32 bank); +void (*write_csr_int_en)(void __iomem *csr_base_addr,u32 bank, +u32 value); +u32 (*read_csr_int_flag)(void __iomem *csr_base_addr,u32 bank);void (*write_csr_int_flag)(void __iomem *csr_base_addr,u32 bank,u32 value); +u32 (*read_csr_int_srcsel)(void __iomem *csr_base_addr,u32 bank);void (*write_csr_int_srcsel)(void __iomem *csr_base_addr,u32 bank); +void (*write_csr_int_srcsel_w_val)(void __iomem *csr_base_addr, +u32 bank,u32 value); +u32 (*read_csr_int_col_en)(void __iomem *csr_base_addr,u32 bank);void (*write_csr_int_col_en)(void __iomem *csr_base_addr,u32 bank,u32 value); +u32 (*read_csr_int_col_ctl)(void __iomem *csr_base_addr,u32 bank);void (*write_csr_int_col_ctl)(void __iomem *csr_base_addr,u32 bank,u32 value); +u32 (*read_csr_int_flag_and_col)(void __iomem *csr_base_addr, +u32 bank);void (*write_csr_int_flag_and_col)(void __iomem *csr_base_addr,u32 bank,u32 value); +u32 (*read_csr_ring_srv_arb_en)(void __iomem *csr_base_addr,u32 bank);void (*write_csr_ring_srv_arb_en)(void __iomem *csr_base_addr,u32 bank,u32 value); +u32 (*get_int_col_ctl_enable_mask)(void); };struct adf_cfg_device_data ;
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_csr_data.c +130 -0
201 lines not on this page · raw message
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_csr_data.h +92 -1
148 lines not on this page · raw message
Signature · 1 line
2.18.2
PATCH v2 7/10 crypto: qat - add bank save and restore flows
In reply to Xin ZengFrom: Siming Wan <[email protected]>
Add logic to save, restore, quiesce and drain a ring bank for QAT GEN4 devices.
This allows to save and restore the state of a Virtual Function (VF) and will be used to implement VM live migration.
--- .../intel/qat/qat_4xxx/adf_4xxx_hw_data.c | 2 + .../intel/qat/qat_common/adf_accel_devices.h | 38 +++ .../intel/qat/qat_common/adf_gen4_hw_data.c | 279 ++++++++++++++++++ .../intel/qat/qat_common/adf_gen4_hw_data.h | 19 ++ 4 files changed, 338 insertions(+)
drivers/crypto/intel/qat/qat_4xxx/adf_4xxx_hw_data.c +2 -0
@@ -455,6 +455,8 @@ void adf_init_hw_data_4xxx(struct adf_hw_device_data *hw_data, u32 dev_id) hw_data->get_ring_to_svc_map = adf_gen4_get_ring_to_svc_map; hw_data->disable_iov = adf_disable_sriov; hw_data->ring_pair_reset = adf_gen4_ring_pair_reset; + hw_data->bank_state_save = adf_gen4_bank_state_save; + hw_data->bank_state_restore = adf_gen4_bank_state_restore; hw_data->enable_pm = adf_gen4_enable_pm; hw_data->handle_pm_interrupt = adf_gen4_handle_pm_interrupt; hw_data->dev_config = adf_gen4_dev_config;
drivers/crypto/intel/qat/qat_common/adf_accel_devices.h +38 -0
@@ -140,6 +140,40 @@ struct admin_info {u32 mailbox_offset; }; +struct ring_config { +u64 base; +u32 config; +u32 head; +u32 tail; +u32 reserved0; +}; + +struct bank_state { +u32 ringstat0; +u32 ringstat1; +u32 ringuostat; +u32 ringestat; +u32 ringnestat; +u32 ringnfstat; +u32 ringfstat; +u32 ringcstat0; +u32 ringcstat1; +u32 ringcstat2; +u32 ringcstat3; +u32 iaintflagen; +u32 iaintflagreg; +u32 iaintflagsrcsel0; +u32 iaintflagsrcsel1; +u32 iaintcolen; +u32 iaintcolctl; +u32 iaintflagandcolen; +u32 ringexpstat; +u32 ringexpintenable; +u32 ringsrvarben; +u32 reserved0; +struct ring_config rings[ADF_ETR_MAX_RINGS_PER_BANK ]; +}; +struct adf_hw_csr_ops {u64 (*build_csr_ring_base_addr)(dma_addr_t addr,u32 size); u32 (*read_csr_ring_head)(void __iomem *csr_base_addr, u32 bank,
@@ -271,6 +305,10 @@ struct adf_hw_device_data {void (*enable_ints)(struct adf_accel_dev *accel_dev);void (*set_ssm_wdtimer)(struct adf_accel_dev *accel_dev);int (*ring_pair_reset)(struct adf_accel_dev *accel_dev,u32 bank_nr); +int (*bank_state_save)(struct adf_accel_dev *accel_dev,u32 bank_number, +struct bank_state *state); +int (*bank_state_restore)(struct adf_accel_dev *accel_dev, +u32 bank_number,struct bank_state *state);void (*reset_device)(struct adf_accel_dev *accel_dev);void (*set_msix_rttable)(struct adf_accel_dev *accel_dev);const char *(*uof_get_name)(struct adf_accel_dev *accel_dev,u32 obj_num);
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.c +279 -0
288 lines not on this page · raw message
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.h +19 -0
@@ -77,10 +77,19 @@#define ADF_RPRESET_POLL_TIMEOUT_US (5 * USEC_PER_SEC)#define ADF_RPRESET_POLL_DELAY_US 20#define ADF_WQM_CSR_RPRESETCTL_RESET BIT(0) +#define ADF_WQM_CSR_RPRESETCTL_DRAIN BIT(2)#define ADF_WQM_CSR_RPRESETCTL (bank) (0x6000 + ((bank) << 3))#define ADF_WQM_CSR_RPRESETSTS_STATUS BIT(0)#define ADF_WQM_CSR_RPRESETSTS (bank) (ADF_WQM_CSR_RPRESETCTL(bank) + 4) +/* Ring interrupt */ +#define ADF_RP_INT_SRC_SEL_F_RISE_MASK BIT(2) +#define ADF_RP_INT_SRC_SEL_F_FALL_MASK GENMASK(2, 0) +#define ADF_RP_INT_SRC_SEL_RANGE_WIDTH 4 +#define ADF_COALESCED_POLL_DELAY_US 1000 +#define ADF_WQM_CSR_RPINTSOU (bank) (0x200000 + ((bank) << 12)) +#define ADF_WQM_CSR_RP_IDX_RX 1 +/* Error source registers */ #define ADF_GEN4_ERRSOU0 (0x41A200)#define ADF_GEN4_ERRSOU1 (0x41A204)
@@ -150,5 +159,15 @@ void adf_gen4_set_msix_default_rttable(struct adf_accel_dev *accel_dev);void adf_gen4_set_ssm_wdtimer (struct adf_accel_dev *accel_dev);int adf_gen4_init_thd2arb_map (struct adf_accel_dev *accel_dev);u16 adf_gen4_get_ring_to_svc_map (struct adf_accel_dev *accel_dev); +int adf_gen4_bank_quiesce_coal_timer (struct adf_accel_dev *accel_dev, +u32 bank_idx,int timeout_ms); +int adf_gen4_bank_drain_start (struct adf_accel_dev *accel_dev, +u32 bank_number,int timeout_us); +void adf_gen4_bank_drain_finish (struct adf_accel_dev *accel_dev, +u32 bank_number); +int adf_gen4_bank_state_save (struct adf_accel_dev *accel_dev,u32 bank_number, +struct bank_state *state); +int adf_gen4_bank_state_restore (struct adf_accel_dev *accel_dev, +u32 bank_number,struct bank_state *state);#endif
Signature · 1 line
2.18.2
PATCH v2 8/10 crypto: qat - add interface for live migration
In reply to Xin ZengExtend the driver with a new interface to be used for VF live migration. This allows to create and destroy a qat_mig_dev object that contains a set of methods to allow to save and restore the state of QAT VF. This interface will be used by the qat-vfio-pci module.
--- drivers/crypto/intel/qat/qat_common/Makefile | 2 +- .../intel/qat/qat_common/adf_accel_devices.h | 17 +++ .../intel/qat/qat_common/adf_gen4_vf_mig.h | 10 ++ .../crypto/intel/qat/qat_common/qat_mig_dev.c | 130 ++++++++++++++++++ include/linux/qat/qat_mig_dev.h | 31 +++++ 5 files changed, 189 insertions(+), 1 deletion(-) create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.h create mode 100644 drivers/crypto/intel/qat/qat_common/qat_mig_dev.c create mode 100644 include/linux/qat/qat_mig_dev.h
drivers/crypto/intel/qat/qat_common/Makefile +1 -1
@@ -54,6 +54,6 @@ intel_qat-$(CONFIG_DEBUG_FS) += adf_transport_debug.o \ intel_qat-$(CONFIG_PCI_IOV) += adf_sriov.o adf_vf_isr.o adf_pfvf_utils.o \ adf_pfvf_pf_msg.o adf_pfvf_pf_proto.o \ adf_pfvf_vf_msg.o adf_pfvf_vf_proto.o \ - adf_gen2_pfvf.o adf_gen4_pfvf.o + adf_gen2_pfvf.o adf_gen4_pfvf.o qat_mig_dev.o intel_qat-$(CONFIG_CRYPTO_DEV_QAT_ERROR_INJECTION) += adf_heartbeat_inject.o
drivers/crypto/intel/qat/qat_common/adf_accel_devices.h +17 -0
@@ -9,6 +9,7 @@#include <linux/pci.h> #include <linux/ratelimit.h> #include <linux/types.h> +#include <linux/qat/qat_mig_dev.h> #include "adf_cfg_common.h" #include "adf_rl.h" #include "adf_telemetry.h"
@@ -258,6 +259,20 @@ struct adf_dc_ops {void (*build_deflate_ctx)(void *ctx); }; +struct qat_migdev_ops { +int (*init)(struct qat_mig_dev *mdev); +void (*cleanup)(struct qat_mig_dev *mdev); +void (*reset)(struct qat_mig_dev *mdev); +int (*open)(struct qat_mig_dev *mdev); +void (*close)(struct qat_mig_dev *mdev); +int (*suspend)(struct qat_mig_dev *mdev); +int (*resume)(struct qat_mig_dev *mdev); +int (*save_state)(struct qat_mig_dev *mdev); +int (*save_setup)(struct qat_mig_dev *mdev); +int (*load_state)(struct qat_mig_dev *mdev); +int (*load_setup)(struct qat_mig_dev *mdev,int size); +}; +struct adf_dev_err_mask {u32 cppagentcmdpar_mask;u32 parerr_ath_cph_mask;
@@ -325,6 +340,7 @@ struct adf_hw_device_data {struct adf_dev_err_mask dev_err_mask;struct adf_rl_hw_data rl_data;struct adf_tl_hw_data tl_data; +struct qat_migdev_ops vfmig_ops;const char *fw_name;const char *fw_mmp_name;u32 fuses;
@@ -381,6 +397,7 @@ struct adf_hw_device_data {#define GET_CSR_OPS (accel_dev) (&(accel_dev)->hw_device->csr_ops)#define GET_PFVF_OPS (accel_dev) (&(accel_dev)->hw_device->pfvf_ops)#define GET_DC_OPS (accel_dev) (&(accel_dev)->hw_device->dc_ops) +#define GET_VFMIG_OPS (accel_dev) (&(accel_dev)->hw_device->vfmig_ops)#define GET_TL_DATA (accel_dev) GET_HW_DATA(accel_dev)->tl_data#define accel_to_pci_dev (accel_ptr) accel_ptr->accel_pci_dev.pci_dev
drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.h new file +10 -0
@@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright(c) 2024 Intel Corporation */ +#ifndef ADF_GEN4_VF_MIG_H_ +#define ADF_GEN4_VF_MIG_H_ + +#include "adf_accel_devices.h" + +void adf_gen4_init_vf_mig_ops (struct qat_migdev_ops *vfmig_ops); + +#endif
drivers/crypto/intel/qat/qat_common/qat_mig_dev.c new file +130 -0
130 lines not on this page · raw message
include/linux/qat/qat_mig_dev.h new file +31 -0
@@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright(c) 2024 Intel Corporation */ +#ifndef QAT_MIG_DEV_H_ +#define QAT_MIG_DEV_H_ + +struct pci_dev ; + +struct qat_mig_dev { +void *parent_accel_dev; +u8 *state; +u32 setup_size; +u32 remote_setup_size; +u32 state_size; +s32 vf_id; +}; + +struct qat_mig_dev *qat_vfmig_create (struct pci_dev *pdev,int vf_id); +int qat_vfmig_init (struct qat_mig_dev *mdev); +void qat_vfmig_cleanup (struct qat_mig_dev *mdev); +void qat_vfmig_reset (struct qat_mig_dev *mdev); +int qat_vfmig_open (struct qat_mig_dev *mdev); +void qat_vfmig_close (struct qat_mig_dev *mdev); +int qat_vfmig_suspend (struct qat_mig_dev *mdev); +int qat_vfmig_resume (struct qat_mig_dev *mdev); +int qat_vfmig_save_state (struct qat_mig_dev *mdev); +int qat_vfmig_save_setup (struct qat_mig_dev *mdev); +int qat_vfmig_load_state (struct qat_mig_dev *mdev); +int qat_vfmig_load_setup (struct qat_mig_dev *mdev,int size); +void qat_vfmig_destroy (struct qat_mig_dev *mdev); + +#endif /*QAT_MIG_DEV_H_*/
Signature · 1 line
2.18.2
PATCH v2 9/10 crypto: qat - implement interface for live migration
In reply to Xin ZengAdd logic to implement the interface for live migration defined in qat/qat_mig_dev.h. This is specific for QAT GEN4 Virtual Functions (VFs).
This introduces a migration data manager which is used to handle the device state during migration. The manager ensures that the device state is stored in a format that can be restored in the destination node.
The VF state is organized into a hierarchical structure that includes a preamble, a general state section, a MISC bar section and an ETR bar section. The latter contains the state of the 4 ring pairs contained on a VF. Here is a graphical representation of the state:
preamble | general state section | leaf state
| MISC bar state section| leaf state
| ETR bar state section | bank0 state section | leaf state
| bank1 state section | leaf state
| bank2 state section | leaf state
| bank3 state section | leaf stateIn addition to the implementation of the qat_migdev_ops interface and the state manager framework, add a mutex in pfvf to avoid pf2vf messages during migration.
--- .../intel/qat/qat_420xx/adf_420xx_hw_data.c | 2 + .../intel/qat/qat_4xxx/adf_4xxx_hw_data.c | 2 + drivers/crypto/intel/qat/qat_common/Makefile | 2 + .../intel/qat/qat_common/adf_accel_devices.h | 6 + .../intel/qat/qat_common/adf_gen4_hw_data.h | 10 + .../intel/qat/qat_common/adf_gen4_vf_mig.c | 1003 +++++++++++++++++ .../intel/qat/qat_common/adf_mstate_mgr.c | 318 ++++++ .../intel/qat/qat_common/adf_mstate_mgr.h | 89 ++ .../crypto/intel/qat/qat_common/adf_sriov.c | 7 +- 9 files changed, 1438 insertions(+), 1 deletion(-) create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.h
drivers/crypto/intel/qat/qat_420xx/adf_420xx_hw_data.c +2 -0
@@ -17,6 +17,7 @@#include <adf_gen4_ras.h> #include <adf_gen4_timer.h> #include <adf_gen4_tl.h> +#include <adf_gen4_vf_mig.h> #include "adf_420xx_hw_data.h" #include "icp_qat_hw.h"
@@ -488,6 +489,7 @@ void adf_init_hw_data_420xx(struct adf_hw_device_data *hw_data, u32 dev_id)adf_gen4_init_dc_ops (&hw_data->dc_ops);adf_gen4_init_ras_ops (&hw_data->ras_ops);adf_gen4_init_tl_data (&hw_data->tl_data); +adf_gen4_init_vf_mig_ops (&hw_data->vfmig_ops);adf_init_rl_data (&hw_data->rl_data); }
drivers/crypto/intel/qat/qat_4xxx/adf_4xxx_hw_data.c +2 -0
@@ -17,6 +17,7 @@#include "adf_gen4_ras.h" #include <adf_gen4_timer.h> #include <adf_gen4_tl.h> +#include <adf_gen4_vf_mig.h> #include "adf_4xxx_hw_data.h" #include "icp_qat_hw.h"
@@ -472,6 +473,7 @@ void adf_init_hw_data_4xxx(struct adf_hw_device_data *hw_data, u32 dev_id)adf_gen4_init_dc_ops (&hw_data->dc_ops);adf_gen4_init_ras_ops (&hw_data->ras_ops);adf_gen4_init_tl_data (&hw_data->tl_data); +adf_gen4_init_vf_mig_ops (&hw_data->vfmig_ops);adf_init_rl_data (&hw_data->rl_data); }
drivers/crypto/intel/qat/qat_common/Makefile +2 -0
@@ -20,12 +20,14 @@ intel_qat-objs := adf_cfg.o \ adf_gen4_config.o \ adf_gen4_hw_csr_data.o \ adf_gen4_hw_data.o \ + adf_gen4_vf_mig.o \ adf_gen4_pm.o \ adf_gen2_dc.o \ adf_gen4_dc.o \ adf_gen4_ras.o \ adf_gen4_timer.o \ adf_clock.o \ + adf_mstate_mgr.o \ qat_crypto.o \ qat_compression.o \ qat_comp_algs.o \
drivers/crypto/intel/qat/qat_common/adf_accel_devices.h +6 -0
@@ -412,11 +412,17 @@ struct adf_fw_loader_data {struct adf_accel_vf_info {struct adf_accel_dev *accel_dev;struct mutex pf2vf_lock;/* protect CSR access for PF2VF messages */ +struct mutex pfvf_mig_lock;/* protects PFVF state for migration */ struct ratelimit_state vf2pf_ratelimit;u32 vf_nr;bool init;bool restarting;u8 vf_compat_ver; +/* +* Private area used for device migration. +* Memory allocation and free is managed by migration driver. +*/ +void *mig_priv; };struct adf_dc_data {
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.h +10 -0
@@ -86,6 +86,7 @@#define ADF_RP_INT_SRC_SEL_F_RISE_MASK BIT(2)#define ADF_RP_INT_SRC_SEL_F_FALL_MASK GENMASK(2, 0)#define ADF_RP_INT_SRC_SEL_RANGE_WIDTH 4 +#define ADF_COALESCED_POLL_TIMEOUT_US (1 * USEC_PER_SEC)#define ADF_COALESCED_POLL_DELAY_US 1000#define ADF_WQM_CSR_RPINTSOU (bank) (0x200000 + ((bank) << 12))#define ADF_WQM_CSR_RP_IDX_RX 1
@@ -120,6 +121,15 @@/* PF2VM communication channel */ #define ADF_GEN4_PF2VM_OFFSET (i) (0x40B010 + (i) * 0x20)#define ADF_GEN4_VM2PF_OFFSET (i) (0x40B014 + (i) * 0x20) +#define ADF_GEN4_VINTMSKPF2VM_OFFSET (i) (0x40B00C + (i) * 0x20) +#define ADF_GEN4_VINTSOUPF2VM_OFFSET (i) (0x40B008 + (i) * 0x20) +#define ADF_GEN4_VINTMSK_OFFSET (i) (0x40B004 + (i) * 0x20) +#define ADF_GEN4_VINTSOU_OFFSET (i) (0x40B000 + (i) * 0x20) + +struct adf_gen4_vfmig { +struct adf_mstate_mgr *mstate_mgr; +bool bank_stopped[ADF_GEN4_NUM_BANKS_PER_VF ]; +};void adf_gen4_set_ssm_wdtimer (struct adf_accel_dev *accel_dev);
drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.c new file +1003 -0
1,003 lines not on this page · raw message
drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.c new file +318 -0
318 lines not on this page · raw message
drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.h new file +89 -0
89 lines not on this page · raw message
drivers/crypto/intel/qat/qat_common/adf_sriov.c +6 -1
@@ -26,10 +26,12 @@ static void adf_iov_send_resp(struct work_struct *work)u32 vf_nr = vf_info->vf_nr;bool ret; +mutex_lock (&vf_info->pfvf_mig_lock); ret =adf_recv_and_handle_vf2pf_msg (accel_dev, vf_nr);if (ret)/* re-enable interrupt on PF from this VF */ adf_enable_vf2pf_interrupts (accel_dev,1 << vf_nr); +mutex_unlock (&vf_info->pfvf_mig_lock);kfree (pf2vf_resp); }
@@ -62,6 +64,7 @@ static int adf_enable_sriov(struct adf_accel_dev *accel_dev) vf_info->vf_nr = i;mutex_init (&vf_info->pf2vf_lock); +mutex_init (&vf_info->pfvf_mig_lock);ratelimit_state_init (&vf_info->vf2pf_ratelimit,ADF_VF2PF_RATELIMIT_INTERVAL ,ADF_VF2PF_RATELIMIT_BURST );
@@ -138,8 +141,10 @@ void adf_disable_sriov(struct adf_accel_dev *accel_dev)if (hw_data->configure_iov_threads) hw_data->configure_iov_threads (accel_dev, false); -for (i =0 , vf = accel_dev->pf.vf_info; i < totalvfs; i++, vf++) +for (i =0 , vf = accel_dev->pf.vf_info; i < totalvfs; i++, vf++) {mutex_destroy (&vf->pf2vf_lock); +mutex_destroy (&vf->pfvf_mig_lock); + }if (!test_bit (ADF_STATUS_RESTARTING , &accel_dev->status)) {kfree (accel_dev->pf.vf_info);
Signature · 1 line
2.18.2
PATCH v2 10/10 vfio/qat: Add vfio_pci driver for Intel QAT VF devices
In reply to Xin ZengAdd vfio pci driver for Intel QAT VF devices.
This driver uses vfio_pci_core to register to the VFIO subsystem. It acts as a vfio agent and interacts with the QAT PF driver to implement VF live migration.
--- MAINTAINERS | 8 + drivers/vfio/pci/Kconfig | 2 + drivers/vfio/pci/Makefile | 2 + drivers/vfio/pci/intel/qat/Kconfig | 12 + drivers/vfio/pci/intel/qat/Makefile | 3 + drivers/vfio/pci/intel/qat/main.c | 688 ++++++++++++++++++++++++++++ 6 files changed, 715 insertions(+) create mode 100644 drivers/vfio/pci/intel/qat/Kconfig create mode 100644 drivers/vfio/pci/intel/qat/Makefile create mode 100644 drivers/vfio/pci/intel/qat/main.c
MAINTAINERS +8 -0
@@ -23099,6 +23099,14 @@ S: Maintained F: Documentation/networking/device_drivers/ethernet/amd/pds_vfio_pci.rst F: drivers/vfio/pci/pds/ +VFIO QAT PCI DRIVER +M: Xin Zeng <[email protected]> +M: Giovanni Cabiddu <[email protected]> +L: [email protected] +L: [email protected] +S: Supported +F: drivers/vfio/pci/intel/qat/ + VFIO PLATFORM DRIVER M: Eric Auger <[email protected]> L: [email protected]
drivers/vfio/pci/Kconfig +2 -0
@@ -67,4 +67,6 @@ source "drivers/vfio/pci/pds/Kconfig" source "drivers/vfio/pci/virtio/Kconfig" +source "drivers/vfio/pci/intel/qat/Kconfig" + endmenu
drivers/vfio/pci/Makefile +2 -0
@@ -15,3 +15,5 @@ obj-$(CONFIG_HISI_ACC_VFIO_PCI) += hisilicon/ obj-$(CONFIG_PDS_VFIO_PCI) += pds/ obj-$(CONFIG_VIRTIO_VFIO_PCI) += virtio/ + +obj-$(CONFIG_QAT_VFIO_PCI) += intel/qat/
drivers/vfio/pci/intel/qat/Kconfig new file +12 -0
@@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0-only +config QAT_VFIO_PCI + tristate "VFIO support for QAT VF PCI devices" + select VFIO_PCI_CORE + depends on CRYPTO_DEV_QAT_4XXX + help + This provides migration support for Intel(R) QAT Virtual Function + using the VFIO framework. + + To compile this as a module, choose M here: the module + will be called qat_vfio_pci. If you don't know what to do here, + say N.
drivers/vfio/pci/intel/qat/Makefile new file +3 -0
@@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0-only +obj-$(CONFIG_QAT_VFIO_PCI) += qat_vfio_pci.o +qat_vfio_pci-y := main.o
drivers/vfio/pci/intel/qat/main.c new file +688 -0
688 lines not on this page · raw message
Signature · 1 line
2.18.2