crypto: qat - fix rate limiting and telemetry for QAT GEN6 wireless
Subsystems (2)
cover crypto: qat - fix rate limiting and telemetry for QAT GEN6 wireless
This set contains fixes for the QAT GEN6 driver (qat_6xxx), mostly affecting devices running in wireless mode, where only the symmetric service is available and the firmware uses WCP (wireless cipher) slices and the adf_wcy_fw_config object layout.
Patch 1 relaxes the rate limiting initialization so that throughput limits are validated only for enabled services. Without it, devices that support only a subset of services fail to initialize rate limiting.
Patch 2 fixes the rate limiting parameters for wireless mode: it uses the firmware-reported WCP slice count as the symmetric slice count, sets only the symmetric maximum throughput (with a wireless-specific value) and computes the symmetric accel engine count from adf_wcy_fw_config, where the wireless object is loaded on two accel engine groups. It depends on patch 1.
Patch 3 fixes the number of command queue counters reported by telemetry on QAT GEN6. The per-slice multipliers are replaced with the total number of command queues for each slice type, reported only when the slice type is present, and the queue count constants are corrected. It is independent of patches 1 and 2.
George Abraham P (3):
crypto: qat - validate rate limiting only for enabled services
crypto: qat - fix rate limiting for QAT GEN6 wireless
crypto: qat - fix telemetry command queue counts
.../intel/qat/qat_6xxx/adf_6xxx_hw_data.c | 38 +++++++++++++++---- .../intel/qat/qat_6xxx/adf_6xxx_hw_data.h | 1 + .../crypto/intel/qat/qat_common/adf_gen6_tl.c | 24 ++++++------ drivers/crypto/intel/qat/qat_common/adf_rl.c | 15 +++++--- drivers/crypto/intel/qat/qat_common/adf_rl.h | 1 + .../intel/qat/qat_common/adf_rl_admin.c | 1 + .../intel/qat/qat_common/adf_telemetry.c | 14 +++---- .../intel/qat/qat_common/adf_telemetry.h | 2 +- 8 files changed, 63 insertions(+), 33 deletions(-)
base-commit: 19a2e99ff8eb57bd3d5cb5d52f9eb77bf3b1540a
Signature · 1 line
2.40.1
PATCH 1/3 crypto: qat - validate rate limiting only for enabled services
Rate limiting validates the throughput limits for asymmetric, symmetric and compression services unconditionally. Devices that support only a subset of these services (for example, QAT GEN6 wireless) can leave unused throughput limits unset, causing rate limiting initialization to fail.
Validate the throughput limit for each enabled base service instead, skipping services that are not enabled. The check also covers the decompression service.
--- drivers/crypto/intel/qat/qat_common/adf_rl.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-)
drivers/crypto/intel/qat/qat_common/adf_rl.c +10 -5
@@ -1050,14 +1050,19 @@ int adf_rl_init(struct adf_accel_dev *accel_dev) {struct adf_hw_device_data *hw_data =GET_HW_DATA (accel_dev);struct adf_rl_hw_data *rl_hw_data = &hw_data->rl_data; +enum adf_base_services svc;struct adf_rl *rl;int ret =0 ; -/* Validate device parameters */ -if (RL_VALIDATE_NON_ZERO (rl_hw_data->max_tp[SVC_ASYM ]) || -RL_VALIDATE_NON_ZERO (rl_hw_data->max_tp[SVC_SYM ]) || -RL_VALIDATE_NON_ZERO (rl_hw_data->max_tp[SVC_DC ]) || -RL_VALIDATE_NON_ZERO (rl_hw_data->scan_interval) || +for (svc =0 ; svc <SVC_BASE_COUNT ; svc++) { +if (adf_is_service_enabled (accel_dev, svc) && +RL_VALIDATE_NON_ZERO (rl_hw_data->max_tp[svc])) { + ret = -EOPNOTSUPP ; + goto err_ret; + } + } + +if (RL_VALIDATE_NON_ZERO (rl_hw_data->scan_interval) ||RL_VALIDATE_NON_ZERO (rl_hw_data->pcie_scale_div) ||RL_VALIDATE_NON_ZERO (rl_hw_data->pcie_scale_mul) ||RL_VALIDATE_NON_ZERO (rl_hw_data->scale_ref)) {
Signature · 1 line
2.40.1
PATCH 2/3 crypto: qat - fix rate limiting for QAT GEN6 wireless
In reply to George Abraham PRate limiting initializes QAT GEN6 wireless devices with the same parameters used for the default configuration. Wireless mode supports only the symmetric service, uses WCP (wireless ciphers) rather than CPH (cipher) slices and loads firmware objects from adf_wcy_fw_config instead of adf_default_fw_config. This results in incorrect rate limiting parameters: the symmetric slice count is taken from the CPH slices, and throughput limits are set for services that are not available. In addition, init_num_svc_aes() derives the symmetric accel engine count from adf_default_fw_config, where ADF_FW_CY_OBJ is loaded on a single accel engine group, instead of the two groups used by ADF_FW_WCY_OBJ in wireless mode, undercounting the AE cycle budget for symmetric SLAs.
Store the firmware-reported WCP slice count in rl_slice_cnt and use it as the symmetric slice count in wireless mode. In wireless mode, set only the symmetric maximum throughput, using a wireless-specific value, and compute the number of accel engines for the symmetric service with the new function init_num_wcy_svc_aes(). This looks up ADF_FW_WCY_OBJ in adf_wcy_fw_config and accumulates the ae_mask across all matching entries, since that object is loaded on two accel engine groups (ADF_AE_GROUP_0 and ADF_AE_GROUP_1).
Pass hw_data to adf_gen6_init_rl_data() so that it can detect wireless mode, and report zero slices for services other than symmetric in that mode.
--- .../intel/qat/qat_6xxx/adf_6xxx_hw_data.c | 38 +++++++++++++++---- .../intel/qat/qat_6xxx/adf_6xxx_hw_data.h | 1 + drivers/crypto/intel/qat/qat_common/adf_rl.h | 1 + .../intel/qat/qat_common/adf_rl_admin.c | 1 + 4 files changed, 33 insertions(+), 8 deletions(-)
drivers/crypto/intel/qat/qat_6xxx/adf_6xxx_hw_data.c +30 -8
@@ -573,6 +573,19 @@ static int adf_gen6_init_thd2arb_map(struct adf_accel_dev *accel_dev)return 0 ; } +static void init_num_wcy_svc_aes (struct adf_rl_hw_data *device_data) +{ +u32 ae_mask =0 ; +unsigned int i; + +for (i =0 ; i <ARRAY_SIZE (adf_wcy_fw_config); i++) { +if (adf_wcy_fw_config[i].obj ==ADF_FW_WCY_OBJ ) + ae_mask |= adf_wcy_fw_config[i].ae_mask; + } + + device_data->svc_ae_mask[SVC_SYM ] =hweight32 (ae_mask); +} +static void init_num_svc_aes (struct adf_rl_hw_data *device_data) {enum adf_fw_objs obj_type, obj_iter;
@@ -608,6 +621,9 @@ static u32 adf_gen6_get_svc_slice_cnt(struct adf_accel_dev *accel_dev, {struct adf_rl_hw_data *device_data = &accel_dev->hw_device->rl_data; +if (adf_6xxx_is_wcy (GET_HW_DATA (accel_dev))) +return (svc ==SVC_SYM ) ? device_data->slices.wcp_cnt :0 ; +switch (svc) {case SVC_SYM :return device_data->slices.cph_cnt;
@@ -973,8 +989,10 @@ static void adf_gen6_init_kpt(struct adf_kpt_hw_data *kpt_data) kpt_data->user_input.swk_cnt_per_pasid =ADF_6XXX_KPT_DEFAULT_SWK_CNT_PER_PASID ; } -static void adf_gen6_init_rl_data (struct adf_rl_hw_data *rl_data) +static void adf_gen6_init_rl_data (struct adf_hw_device_data *hw_data) { +struct adf_rl_hw_data *rl_data = &hw_data->rl_data; + rl_data->pciout_tb_offset =ADF_GEN6_RL_TOKEN_PCIEOUT_BUCKET_OFFSET ; rl_data->pciin_tb_offset =ADF_GEN6_RL_TOKEN_PCIEIN_BUCKET_OFFSET ; rl_data->r2l_offset =ADF_GEN6_RL_R2L_OFFSET ;
@@ -982,14 +1000,18 @@ static void adf_gen6_init_rl_data(struct adf_rl_hw_data *rl_data) rl_data->c2s_offset =ADF_GEN6_RL_C2S_OFFSET ; rl_data->pcie_scale_div =ADF_6XXX_RL_PCIE_SCALE_FACTOR_DIV ; rl_data->pcie_scale_mul =ADF_6XXX_RL_PCIE_SCALE_FACTOR_MUL ; - rl_data->max_tp[SVC_ASYM ] =ADF_6XXX_RL_MAX_TP_ASYM ; - rl_data->max_tp[SVC_SYM ] =ADF_6XXX_RL_MAX_TP_SYM ; - rl_data->max_tp[SVC_DC ] =ADF_6XXX_RL_MAX_TP_DC ; - rl_data->max_tp[SVC_DECOMP ] =ADF_6XXX_RL_MAX_TP_DECOMP ; rl_data->scan_interval =ADF_6XXX_RL_SCANS_PER_SEC ; rl_data->scale_ref =ADF_6XXX_RL_SLICE_REF ; - -init_num_svc_aes (rl_data); +if (adf_6xxx_is_wcy (hw_data)) { + rl_data->max_tp[SVC_SYM ] =ADF_6XXX_RL_MAX_TP_WCY_SYM ; +init_num_wcy_svc_aes (rl_data); + }else { + rl_data->max_tp[SVC_ASYM ] =ADF_6XXX_RL_MAX_TP_ASYM ; + rl_data->max_tp[SVC_SYM ] =ADF_6XXX_RL_MAX_TP_SYM ; + rl_data->max_tp[SVC_DC ] =ADF_6XXX_RL_MAX_TP_DC ; + rl_data->max_tp[SVC_DECOMP ] =ADF_6XXX_RL_MAX_TP_DECOMP ; +init_num_svc_aes (rl_data); + } }static void adf_gen6_init_services_supported (struct adf_hw_device_data *hw_data)
@@ -1067,7 +1089,7 @@ void adf_init_hw_data_6xxx(struct adf_hw_device_data *hw_data)adf_gen6_init_vf_mig_ops (&hw_data->vfmig_ops);adf_gen6_init_ras_ops (&hw_data->ras_ops);adf_gen6_init_tl_data (&hw_data->tl_data); -adf_gen6_init_rl_data (&hw_data->rl_data); +adf_gen6_init_rl_data (hw_data);adf_gen6_init_anti_rb (&hw_data->anti_rb_data);adf_gen6_init_kpt (&hw_data->kpt_data); }
drivers/crypto/intel/qat/qat_6xxx/adf_6xxx_hw_data.h +1 -0
@@ -158,6 +158,7 @@#define ADF_6XXX_RL_SCANS_PER_SEC 954#define ADF_6XXX_RL_MAX_TP_ASYM 173750UL#define ADF_6XXX_RL_MAX_TP_SYM 95000UL +#define ADF_6XXX_RL_MAX_TP_WCY_SYM 61000UL#define ADF_6XXX_RL_MAX_TP_DC 40000UL#define ADF_6XXX_RL_MAX_TP_DECOMP 40000UL#define ADF_6XXX_RL_SLICE_REF 1000UL
drivers/crypto/intel/qat/qat_common/adf_rl.h +1 -0
@@ -69,6 +69,7 @@ struct rl_slice_cnt {u8 pke_cnt;u8 cph_cnt;u8 cpr_cnt; +u8 wcp_cnt; };struct adf_rl_interface_data {
drivers/crypto/intel/qat/qat_common/adf_rl_admin.c +1 -0
@@ -64,6 +64,7 @@ int adf_rl_send_admin_init_msg(struct adf_accel_dev *accel_dev,/* For symmetric crypto, slice tokens are relative to the UCS slice */ slices_int->cph_cnt = slices_resp.ucs_cnt; slices_int->cpr_cnt = slices_resp.cpr_cnt; + slices_int->wcp_cnt = slices_resp.wcp_cnt;return 0 ; }
Signature · 1 line
2.40.1
PATCH 3/3 crypto: qat - fix telemetry command queue counts
In reply to George Abraham PQAT GEN6 telemetry derives the number of command queues for each slice type by multiplying the slice count reported by firmware by a per-slice multiplier. However, the number of command queues for a slice type is fixed and does not scale with the number of slices. For example, WAT (wireless auth) and WCP (wireless cipher) slices have 18 command queues across four slices. In addition, the per-slice queue counts for the PKE (public key encryption), WAT, WCP, UCS (crypto) and ATH (auth) slices do not match the hardware (for example, PKE has 18 command queues, not 1). This results in an incorrect number of command queue counters being reported.
Replace the per-slice multipliers with the total number of command queues for each slice type, and report that count only when the corresponding slice type is present. Update the PKE, WAT, WCP, UCS and ATH queue count constants, rename the multiplier field in struct adf_tl_hw_data to cmdq_cnt, and fix the USC_QUEUE_COUNT typo to UCS_QUEUE_COUNT.
--- .../crypto/intel/qat/qat_common/adf_gen6_tl.c | 24 +++++++++---------- .../intel/qat/qat_common/adf_telemetry.c | 14 +++++------ .../intel/qat/qat_common/adf_telemetry.h | 2 +- 3 files changed, 20 insertions(+), 20 deletions(-)
drivers/crypto/intel/qat/qat_common/adf_gen6_tl.c +12 -12
@@ -34,11 +34,11 @@#define CPR_QUEUE_COUNT 5#define DCPR_QUEUE_COUNT 3 -#define PKE_QUEUE_COUNT 1 -#define WAT_QUEUE_COUNT 7 -#define WCP_QUEUE_COUNT 7 -#define USC_QUEUE_COUNT 3 -#define ATH_QUEUE_COUNT 2 +#define PKE_QUEUE_COUNT 18 +#define WAT_QUEUE_COUNT 18 +#define WCP_QUEUE_COUNT 18 +#define UCS_QUEUE_COUNT 6 +#define ATH_QUEUE_COUNT 8/* Device level counters. */ static const struct adf_tl_dbg_counter dev_counters[] = {
@@ -247,12 +247,12 @@ void adf_gen6_init_tl_data(struct adf_tl_hw_data *tl_data) tl_data->num_rp_counters =ARRAY_SIZE (rp_counters); tl_data->max_sl_cnt =ADF_GEN6_TL_MAX_SLICES_PER_TYPE ; - tl_data->multiplier.cpr_cnt =CPR_QUEUE_COUNT ; - tl_data->multiplier.dcpr_cnt =DCPR_QUEUE_COUNT ; - tl_data->multiplier.pke_cnt =PKE_QUEUE_COUNT ; - tl_data->multiplier.wat_cnt =WAT_QUEUE_COUNT ; - tl_data->multiplier.wcp_cnt =WCP_QUEUE_COUNT ; - tl_data->multiplier.ucs_cnt =USC_QUEUE_COUNT ; - tl_data->multiplier.ath_cnt =ATH_QUEUE_COUNT ; + tl_data->cmdq_cnt.cpr_cnt =CPR_QUEUE_COUNT ; + tl_data->cmdq_cnt.dcpr_cnt =DCPR_QUEUE_COUNT ; + tl_data->cmdq_cnt.pke_cnt =PKE_QUEUE_COUNT ; + tl_data->cmdq_cnt.wat_cnt =WAT_QUEUE_COUNT ; + tl_data->cmdq_cnt.wcp_cnt =WCP_QUEUE_COUNT ; + tl_data->cmdq_cnt.ucs_cnt =UCS_QUEUE_COUNT ; + tl_data->cmdq_cnt.ath_cnt =ATH_QUEUE_COUNT ; }EXPORT_SYMBOL_GPL (adf_gen6_init_tl_data);
drivers/crypto/intel/qat/qat_common/adf_telemetry.c +7 -7
@@ -219,13 +219,13 @@ static void adf_set_cmdq_cnt(struct adf_accel_dev *accel_dev, slice_cnt = &accel_dev->telemetry->slice_cnt; cmdq_cnt = &accel_dev->telemetry->cmdq_cnt; - cmdq_cnt->cpr_cnt = slice_cnt->cpr_cnt * tl_data->multiplier.cpr_cnt; - cmdq_cnt->dcpr_cnt = slice_cnt->dcpr_cnt * tl_data->multiplier.dcpr_cnt; - cmdq_cnt->pke_cnt = slice_cnt->pke_cnt * tl_data->multiplier.pke_cnt; - cmdq_cnt->wat_cnt = slice_cnt->wat_cnt * tl_data->multiplier.wat_cnt; - cmdq_cnt->wcp_cnt = slice_cnt->wcp_cnt * tl_data->multiplier.wcp_cnt; - cmdq_cnt->ucs_cnt = slice_cnt->ucs_cnt * tl_data->multiplier.ucs_cnt; - cmdq_cnt->ath_cnt = slice_cnt->ath_cnt * tl_data->multiplier.ath_cnt; + cmdq_cnt->cpr_cnt = slice_cnt->cpr_cnt ? tl_data->cmdq_cnt.cpr_cnt :0 ; + cmdq_cnt->dcpr_cnt = slice_cnt->dcpr_cnt ? tl_data->cmdq_cnt.dcpr_cnt :0 ; + cmdq_cnt->pke_cnt = slice_cnt->pke_cnt ? tl_data->cmdq_cnt.pke_cnt :0 ; + cmdq_cnt->wat_cnt = slice_cnt->wat_cnt ? tl_data->cmdq_cnt.wat_cnt :0 ; + cmdq_cnt->wcp_cnt = slice_cnt->wcp_cnt ? tl_data->cmdq_cnt.wcp_cnt :0 ; + cmdq_cnt->ucs_cnt = slice_cnt->ucs_cnt ? tl_data->cmdq_cnt.ucs_cnt :0 ; + cmdq_cnt->ath_cnt = slice_cnt->ath_cnt ? tl_data->cmdq_cnt.ath_cnt :0 ; }int adf_tl_run (struct adf_accel_dev *accel_dev,int state)
drivers/crypto/intel/qat/qat_common/adf_telemetry.h +1 -1
@@ -44,7 +44,7 @@ struct adf_tl_hw_data {u8 num_cmdq_counters;u8 max_rp;u8 max_sl_cnt; -struct icp_qat_fw_init_admin_slice_cnt multiplier; +struct icp_qat_fw_init_admin_slice_cnt cmdq_cnt; };struct adf_telemetry {
Signature · 1 line
2.40.1
Herbert Xu
maintainerQuoted 38 lines · George Abraham P
This set contains fixes for the QAT GEN6 driver (
qat_6xxx), mostly affecting devices running in wireless mode, where only the symmetric service is available and the firmware uses WCP (wireless cipher) slices and theadf_wcy_fw_configobject layout.Patch 1 relaxes the rate limiting initialization so that throughput limits are validated only for enabled services. Without it, devices that support only a subset of services fail to initialize rate limiting.
Patch 2 fixes the rate limiting parameters for wireless mode: it uses the firmware-reported WCP slice count as the symmetric slice count, sets only the symmetric maximum throughput (with a wireless-specific value) and computes the symmetric accel engine count from
adf_wcy_fw_config, where the wireless object is loaded on two accel engine groups. It depends on patch 1.Patch 3 fixes the number of command queue counters reported by telemetry on QAT GEN6. The per-slice multipliers are replaced with the total number of command queues for each slice type, reported only when the slice type is present, and the queue count constants are corrected. It is independent of patches 1 and 2.
George Abraham P (3):
crypto: qat - validate rate limiting only for enabled services
crypto: qat - fix rate limiting for QAT GEN6 wireless
crypto: qat - fix telemetry command queue counts.../intel/qat/qat_6xxx/adf_6xxx_hw_data.c | 38 +++++++++++++++---- .../intel/qat/qat_6xxx/adf_6xxx_hw_data.h | 1 + .../crypto/intel/qat/qat_common/adf_gen6_tl.c | 24 ++++++------ drivers/crypto/intel/qat/qat_common/adf_rl.c | 15 +++++--- drivers/crypto/intel/qat/qat_common/adf_rl.h | 1 + .../intel/qat/qat_common/adf_rl_admin.c | 1 + .../intel/qat/qat_common/adf_telemetry.c | 14 +++---- .../intel/qat/qat_common/adf_telemetry.h | 2 +- 8 files changed, 63 insertions(+), 33 deletions(-)
Quoted 3 lines
base-commit:
19a2e99ff8eb57bd3d5cb5d52f9eb77bf3b1540aSignature · 1 line
2.40.1
All applied. Thanks.
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt