crypto: qat - add heartbeat feature
Subsystems (2)
v4 cover crypto: qat - add heartbeat feature
This set introduces support for the QAT heartbeat feature. It allows detection whenever device firmware or acceleration unit will hang. We're adding this feature to allow our clients having a tool with they could verify if all of the Quick Assist hardware resources are healthy and operational.
QAT device firmware periodically writes counters to a specified physical memory location. A pair of counters per thread is incremented at the start and end of the main processing loop within the firmware. Checking for Heartbeat consists of checking the validity of the pair of counter values for each thread. Stagnant counters indicate a firmware hang.
The first patch adds timestamp synchronization to the firmware. The second patch removes historical and never used HB definitions. Patch no. 3 is implementing the hardware clock frequency measuring interface.
The fourth introduces the main heartbeat implementation with the debugfs interface.
The last patch implements an algorithm that allows the code to detect which version of heartbeat API is used at the currently loaded firmware.
Changes since v3:
- improved comment in measure_clock() as suggested by Andy Shevchenko
- changed release date and version for 6.6 in interface documentation
Changes since v2:
- fixed build error on a few of architectures - reduced unnecessary 64bit division.
Changes since v1:
- fixed build errors on a few of architectures - replaced macro DIV_ROUND_CLOSEST with DIV_ROUND_CLOSEST_ULL
- included prerequisite patch "add internal timer for qat 4xxx" which initially was sent separately as this patchset was still in development.
- timer patch reworked to use delayed work as suggested by Herbert Xu
Damian Muszynski (5):
crypto: qat - add internal timer for qat 4xxx
crypto: qat - drop obsolete heartbeat interface
crypto: qat - add measure clock frequency
crypto: qat - add heartbeat feature
crypto: qat - add heartbeat counters check
Documentation/ABI/testing/debugfs-driver-qat | 51 +++ .../intel/qat/qat_4xxx/adf_4xxx_hw_data.c | 14 + .../intel/qat/qat_4xxx/adf_4xxx_hw_data.h | 4 + drivers/crypto/intel/qat/qat_4xxx/adf_drv.c | 3 + .../intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c | 28 ++ .../intel/qat/qat_c3xxx/adf_c3xxx_hw_data.h | 7 + .../intel/qat/qat_c62x/adf_c62x_hw_data.c | 28 ++ .../intel/qat/qat_c62x/adf_c62x_hw_data.h | 7 + drivers/crypto/intel/qat/qat_common/Makefile | 4 + .../intel/qat/qat_common/adf_accel_devices.h | 13 + .../crypto/intel/qat/qat_common/adf_admin.c | 43 +++ .../intel/qat/qat_common/adf_cfg_strings.h | 2 + .../crypto/intel/qat/qat_common/adf_clock.c | 131 +++++++ .../crypto/intel/qat/qat_common/adf_clock.h | 14 + .../intel/qat/qat_common/adf_common_drv.h | 5 + .../crypto/intel/qat/qat_common/adf_dbgfs.c | 9 +- .../intel/qat/qat_common/adf_gen2_config.c | 7 + .../intel/qat/qat_common/adf_gen2_hw_data.h | 3 + .../intel/qat/qat_common/adf_gen4_hw_data.h | 3 + .../intel/qat/qat_common/adf_gen4_timer.c | 70 ++++ .../intel/qat/qat_common/adf_gen4_timer.h | 21 ++ .../intel/qat/qat_common/adf_heartbeat.c | 336 ++++++++++++++++++ .../intel/qat/qat_common/adf_heartbeat.h | 79 ++++ .../qat/qat_common/adf_heartbeat_dbgfs.c | 194 ++++++++++ .../qat/qat_common/adf_heartbeat_dbgfs.h | 12 + .../crypto/intel/qat/qat_common/adf_init.c | 28 ++ drivers/crypto/intel/qat/qat_common/adf_isr.c | 6 + .../qat/qat_common/icp_qat_fw_init_admin.h | 23 +- .../qat/qat_dh895xcc/adf_dh895xcc_hw_data.c | 13 + .../qat/qat_dh895xcc/adf_dh895xcc_hw_data.h | 5 + 30 files changed, 1147 insertions(+), 16 deletions(-) create mode 100644 drivers/crypto/intel/qat/qat_common/adf_clock.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_clock.h create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_timer.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_timer.h create mode 100644 drivers/crypto/intel/qat/qat_common/adf_heartbeat.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_heartbeat.h create mode 100644 drivers/crypto/intel/qat/qat_common/adf_heartbeat_dbgfs.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_heartbeat_dbgfs.h
base-commit: 67b9bc0df80cfa241fe7a9c2b857c3e3efde982a
Signature · 1 line
2.40.1
PATCH v4 1/5 crypto: qat - add internal timer for qat 4xxx
The power management feature in QAT 4xxx devices can disable clock sources used to implement timers. Because of that, the firmware needs to get an external reliable source of time.
Add a kernel delayed work that periodically sends an event to the firmware. This is triggered every 200ms. At each execution, the driver sends a sync request to the firmware reporting the current timestamp counter value.
This is a pre-requisite for enabling the heartbeat, telemetry and rate limiting features.
--- .../intel/qat/qat_4xxx/adf_4xxx_hw_data.c | 3 + drivers/crypto/intel/qat/qat_common/Makefile | 1 + .../intel/qat/qat_common/adf_accel_devices.h | 3 + .../crypto/intel/qat/qat_common/adf_admin.c | 12 ++++ .../intel/qat/qat_common/adf_common_drv.h | 3 + .../intel/qat/qat_common/adf_gen4_timer.c | 70 +++++++++++++++++++ .../intel/qat/qat_common/adf_gen4_timer.h | 21 ++++++ .../crypto/intel/qat/qat_common/adf_init.c | 13 ++++ drivers/crypto/intel/qat/qat_common/adf_isr.c | 6 ++ .../qat/qat_common/icp_qat_fw_init_admin.h | 5 ++ 10 files changed, 137 insertions(+) create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_timer.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_timer.h
drivers/crypto/intel/qat/qat_4xxx/adf_4xxx_hw_data.c +3 -0
@@ -8,6 +8,7 @@#include <adf_gen4_hw_data.h> #include <adf_gen4_pfvf.h> #include <adf_gen4_pm.h> +#include <adf_gen4_timer.h> #include "adf_4xxx_hw_data.h" #include "icp_qat_hw.h"
@@ -508,6 +509,8 @@ void adf_init_hw_data_4xxx(struct adf_hw_device_data *hw_data, u32 dev_id) hw_data->enable_pm = adf_gen4_enable_pm; hw_data->handle_pm_interrupt = adf_gen4_handle_pm_interrupt; hw_data->dev_config = adf_gen4_dev_config; + hw_data->start_timer = adf_gen4_timer_start; + hw_data->stop_timer = adf_gen4_timer_stop;adf_gen4_init_hw_csr_ops (&hw_data->csr_ops);adf_gen4_init_pf_pfvf_ops (&hw_data->pfvf_ops);
drivers/crypto/intel/qat/qat_common/Makefile +1 -0
@@ -17,6 +17,7 @@ intel_qat-objs := adf_cfg.o \ adf_gen4_pm.o \ adf_gen2_dc.o \ adf_gen4_dc.o \ + adf_gen4_timer.o \ qat_crypto.o \ qat_compression.o \ qat_comp_algs.o \
drivers/crypto/intel/qat/qat_common/adf_accel_devices.h +3 -0
@@ -188,6 +188,8 @@ struct adf_hw_device_data {int (*init_admin_comms)(struct adf_accel_dev *accel_dev);void (*exit_admin_comms)(struct adf_accel_dev *accel_dev);int (*send_admin_init)(struct adf_accel_dev *accel_dev); +int (*start_timer)(struct adf_accel_dev *accel_dev); +void (*stop_timer)(struct adf_accel_dev *accel_dev);int (*init_arb)(struct adf_accel_dev *accel_dev);void (*exit_arb)(struct adf_accel_dev *accel_dev);const u32 *(*get_arb_mapping)(struct adf_accel_dev *accel_dev);
@@ -296,6 +298,7 @@ struct adf_accel_dev {struct list_head list;struct module *owner;struct adf_accel_pci accel_pci_dev; +struct adf_timer *timer; union { struct { /* protects VF2PF interrupts access */
drivers/crypto/intel/qat/qat_common/adf_admin.c +12 -0
@@ -241,6 +241,18 @@ int adf_get_ae_fw_counters(struct adf_accel_dev *accel_dev, u16 ae, u64 *reqs, ureturn 0 ; } +int adf_send_admin_tim_sync (struct adf_accel_dev *accel_dev,u32 cnt) +{ +u32 ae_mask = accel_dev->hw_device->ae_mask; +struct icp_qat_fw_init_admin_req req = { }; +struct icp_qat_fw_init_admin_resp resp = { }; + + req.cmd_id =ICP_QAT_FW_SYNC ; + req.int_timer_ticks = cnt; + +return adf_send_admin (accel_dev, &req, &resp, ae_mask); +} +/** * adf_send_admin_init() - Function sends init message to FW * @accel_dev: Pointer to acceleration device.
drivers/crypto/intel/qat/qat_common/adf_common_drv.h +3 -0
@@ -96,6 +96,7 @@ void adf_exit_admin_comms(struct adf_accel_dev *accel_dev);int adf_send_admin_init (struct adf_accel_dev *accel_dev);int adf_get_ae_fw_counters (struct adf_accel_dev *accel_dev,u16 ae,u64 *reqs,u64 *resps);int adf_init_admin_pm (struct adf_accel_dev *accel_dev,u32 idle_delay); +int adf_send_admin_tim_sync (struct adf_accel_dev *accel_dev,u32 cnt);int adf_init_arb (struct adf_accel_dev *accel_dev);void adf_exit_arb (struct adf_accel_dev *accel_dev);void adf_update_ring_arb (struct adf_etr_ring_data *ring);
@@ -194,6 +195,8 @@ int qat_uclo_set_cfg_ae_mask(struct icp_qat_fw_loader_handle *handle,int adf_init_misc_wq (void );void adf_exit_misc_wq (void );bool adf_misc_wq_queue_work (struct work_struct *work); +bool adf_misc_wq_queue_delayed_work (struct delayed_work *work, +unsigned long delay);#if defined(CONFIG_PCI_IOV )int adf_sriov_configure (struct pci_dev *pdev,int numvfs);void adf_disable_sriov (struct adf_accel_dev *accel_dev);
drivers/crypto/intel/qat/qat_common/adf_gen4_timer.c new file +70 -0
70 lines not on this page · raw message
drivers/crypto/intel/qat/qat_common/adf_gen4_timer.h new file +21 -0
@@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright(c) 2023 Intel Corporation */ + +#ifndef ADF_GEN4_TIMER_H_ +#define ADF_GEN4_TIMER_H_ + +#include <linux/ktime.h> +#include <linux/workqueue.h> + +struct adf_accel_dev ; + +struct adf_timer { +struct adf_accel_dev *accel_dev; +struct delayed_work work_ctx; +ktime_t initial_ktime; +}; + +int adf_gen4_timer_start (struct adf_accel_dev *accel_dev); +void adf_gen4_timer_stop (struct adf_accel_dev *accel_dev); + +#endif /* ADF_GEN4_TIMER_H_ */
drivers/crypto/intel/qat/qat_common/adf_init.c +13 -0
@@ -163,6 +163,7 @@ static int adf_dev_start(struct adf_accel_dev *accel_dev)struct adf_hw_device_data *hw_data = accel_dev->hw_device;struct service_hndl *service;struct list_head *list_itr; +int ret;set_bit (ADF_STATUS_STARTING , &accel_dev->status);
@@ -187,6 +188,14 @@ static int adf_dev_start(struct adf_accel_dev *accel_dev)return -EFAULT ; } +if (hw_data->start_timer) { + ret = hw_data->start_timer (accel_dev); +if (ret) { +dev_err (&GET_DEV (accel_dev),"Failed to start internal sync timer\n" ); +return ret; + } + } +list_for_each (list_itr, &service_table) { service = list_entry(list_itr, struct service_hndl, list);if (service->event_hld (accel_dev,ADF_EVENT_START )) {
@@ -235,6 +244,7 @@ static int adf_dev_start(struct adf_accel_dev *accel_dev)*/ static void adf_dev_stop (struct adf_accel_dev *accel_dev) { +struct adf_hw_device_data *hw_data = accel_dev->hw_device;struct service_hndl *service;struct list_head *list_itr;bool wait = false;
@@ -270,6 +280,9 @@ static void adf_dev_stop(struct adf_accel_dev *accel_dev) } } +if (hw_data->stop_timer) + hw_data->stop_timer (accel_dev); +if (wait)msleep (100 );
drivers/crypto/intel/qat/qat_common/adf_isr.c +6 -0
@@ -380,3 +380,9 @@ bool adf_misc_wq_queue_work(struct work_struct *work) {return queue_work (adf_misc_wq, work); } + +bool adf_misc_wq_queue_delayed_work (struct delayed_work *work, +unsigned long delay) +{ +return queue_delayed_work (adf_misc_wq, work, delay); +}
drivers/crypto/intel/qat/qat_common/icp_qat_fw_init_admin.h +5 -0
@@ -37,6 +37,9 @@ struct icp_qat_fw_init_admin_req {__u16 ibuf_size_in_kb;__u16 resrvd3; }; +struct { +__u32 int_timer_ticks; + };__u32 idle_filter; };
@@ -97,6 +100,8 @@ struct icp_qat_fw_init_admin_resp { }; } __packed; +#define ICP_QAT_FW_SYNC ICP_QAT_FW_HEARTBEAT_SYNC +#define ICP_QAT_FW_COMN_HEARTBEAT_OK 0#define ICP_QAT_FW_COMN_HEARTBEAT_BLOCKED 1#define ICP_QAT_FW_COMN_HEARTBEAT_FLAG_BITPOS 0
Signature · 1 line
2.40.1
PATCH v4 2/5 crypto: qat - drop obsolete heartbeat interface
In reply to Damian MuszynskiDrop legacy heartbeat interface from FW API as it is no longer used.
--- .../intel/qat/qat_common/icp_qat_fw_init_admin.h | 15 --------------- 1 file changed, 15 deletions(-)
drivers/crypto/intel/qat/qat_common/icp_qat_fw_init_admin.h +0 -15
@@ -102,19 +102,4 @@ struct icp_qat_fw_init_admin_resp {#define ICP_QAT_FW_SYNC ICP_QAT_FW_HEARTBEAT_SYNC -#define ICP_QAT_FW_COMN_HEARTBEAT_OK 0 -#define ICP_QAT_FW_COMN_HEARTBEAT_BLOCKED 1 -#define ICP_QAT_FW_COMN_HEARTBEAT_FLAG_BITPOS 0 -#define ICP_QAT_FW_COMN_HEARTBEAT_FLAG_MASK 0x1 -#define ICP_QAT_FW_COMN_STATUS_RESRVD_FLD_MASK 0xFE -#define ICP_QAT_FW_COMN_HEARTBEAT_HDR_FLAG_GET (hdr_t) \ - ICP_QAT_FW_COMN_HEARTBEAT_FLAG_GET(hdr_t.flags) - -#define ICP_QAT_FW_COMN_HEARTBEAT_HDR_FLAG_SET (hdr_t, val) \ - ICP_QAT_FW_COMN_HEARTBEAT_FLAG_SET(hdr_t, val) - -#define ICP_QAT_FW_COMN_HEARTBEAT_FLAG_GET (flags) \ - QAT_FIELD_GET(flags, \ - ICP_QAT_FW_COMN_HEARTBEAT_FLAG_BITPOS, \ - ICP_QAT_FW_COMN_HEARTBEAT_FLAG_MASK)#endif
Signature · 1 line
2.40.1
PATCH v4 3/5 crypto: qat - add measure clock frequency
In reply to Damian MuszynskiThe QAT hardware does not expose a mechanism to report its clock frequency. This is required to implement the Heartbeat feature.
Add a clock measuring algorithm that estimates the frequency by comparing the internal timestamp counter incremented by the firmware with the time measured by the kernel.
The frequency value is only used internally and not exposed to the user.
--- .../intel/qat/qat_4xxx/adf_4xxx_hw_data.c | 10 ++ .../intel/qat/qat_4xxx/adf_4xxx_hw_data.h | 4 + .../intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c | 25 ++++ .../intel/qat/qat_c3xxx/adf_c3xxx_hw_data.h | 7 + .../intel/qat/qat_c62x/adf_c62x_hw_data.c | 25 ++++ .../intel/qat/qat_c62x/adf_c62x_hw_data.h | 7 + drivers/crypto/intel/qat/qat_common/Makefile | 1 + .../intel/qat/qat_common/adf_accel_devices.h | 2 + .../crypto/intel/qat/qat_common/adf_admin.c | 17 +++ .../crypto/intel/qat/qat_common/adf_clock.c | 131 ++++++++++++++++++ .../crypto/intel/qat/qat_common/adf_clock.h | 14 ++ .../intel/qat/qat_common/adf_common_drv.h | 1 + .../crypto/intel/qat/qat_common/adf_init.c | 8 ++ .../qat/qat_common/icp_qat_fw_init_admin.h | 1 + .../qat/qat_dh895xcc/adf_dh895xcc_hw_data.c | 10 ++ .../qat/qat_dh895xcc/adf_dh895xcc_hw_data.h | 5 + 16 files changed, 268 insertions(+) create mode 100644 drivers/crypto/intel/qat/qat_common/adf_clock.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_clock.h
drivers/crypto/intel/qat/qat_4xxx/adf_4xxx_hw_data.c +10 -0
@@ -3,6 +3,7 @@#include <linux/iopoll.h> #include <adf_accel_devices.h> #include <adf_cfg.h> +#include <adf_clock.h> #include <adf_common_drv.h> #include <adf_gen4_dc.h> #include <adf_gen4_hw_data.h>
@@ -318,6 +319,14 @@ static void get_admin_info(struct admin_info *admin_csrs_info) admin_csrs_info->admin_msg_lr =ADF_4XXX_ADMINMSGLR_OFFSET ; } +static u32 get_heartbeat_clock (struct adf_hw_device_data *self) +{ +/* +* 4XXX uses KPT counter for HB +*/ +return ADF_4XXX_KPT_COUNTER_FREQ ; +} +static void adf_enable_error_correction (struct adf_accel_dev *accel_dev) {struct adf_bar *misc_bar = &GET_BARS (accel_dev)[ADF_4XXX_PMISC_BAR ];
@@ -511,6 +520,7 @@ void adf_init_hw_data_4xxx(struct adf_hw_device_data *hw_data, u32 dev_id) hw_data->dev_config = adf_gen4_dev_config; hw_data->start_timer = adf_gen4_timer_start; hw_data->stop_timer = adf_gen4_timer_stop; + hw_data->get_hb_clock = get_heartbeat_clock;adf_gen4_init_hw_csr_ops (&hw_data->csr_ops);adf_gen4_init_pf_pfvf_ops (&hw_data->pfvf_ops);
drivers/crypto/intel/qat/qat_4xxx/adf_4xxx_hw_data.h +4 -0
@@ -3,6 +3,7 @@#ifndef ADF_4XXX_HW_DATA_H_ #define ADF_4XXX_HW_DATA_H_ +#include <linux/units.h> #include <adf_accel_devices.h> /* PCIe configuration space */
@@ -64,6 +65,9 @@#define ADF_402XX_ASYM_OBJ "qat_402xx_asym.bin"#define ADF_402XX_ADMIN_OBJ "qat_402xx_admin.bin" +/* Clocks frequency */ +#define ADF_4XXX_KPT_COUNTER_FREQ (100 * HZ_PER_MHZ) +/* qat_4xxx fuse bits are different from old GENs, redefine them */ enum icp_qat_4xxx_slice_mask {ICP_ACCEL_4XXX_MASK_CIPHER_SLICE =BIT (0 ),
drivers/crypto/intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c +25 -0
@@ -1,6 +1,7 @@// SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0-only) /* Copyright(c) 2014 - 2021 Intel Corporation */ #include <adf_accel_devices.h> +#include <adf_clock.h> #include <adf_common_drv.h> #include <adf_gen2_config.h> #include <adf_gen2_dc.h>
@@ -50,6 +51,28 @@ static u32 get_ae_mask(struct adf_hw_device_data *self)return ~(fuses | straps) &ADF_C3XXX_ACCELENGINES_MASK ; } +static u32 get_ts_clock (struct adf_hw_device_data *self) +{ +/* +* Timestamp update interval is 16 AE clock ticks for c3xxx. +*/ +return self->clock_frequency /16 ; +} + +static int measure_clock (struct adf_accel_dev *accel_dev) +{ +u32 frequency; +int ret; + + ret =adf_dev_measure_clock (accel_dev, &frequency,ADF_C3XXX_MIN_AE_FREQ , +ADF_C3XXX_MAX_AE_FREQ ); +if (ret) +return ret; + + accel_dev->hw_device->clock_frequency = frequency; +return 0 ; +} +static u32 get_misc_bar_id (struct adf_hw_device_data *self) {return ADF_C3XXX_PMISC_BAR ;
@@ -127,6 +150,8 @@ void adf_init_hw_data_c3xxx(struct adf_hw_device_data *hw_data) hw_data->set_ssm_wdtimer = adf_gen2_set_ssm_wdtimer; hw_data->disable_iov = adf_disable_sriov; hw_data->dev_config = adf_gen2_dev_config; + hw_data->measure_clock = measure_clock; + hw_data->get_hb_clock = get_ts_clock;adf_gen2_init_pf_pfvf_ops (&hw_data->pfvf_ops);adf_gen2_init_hw_csr_ops (&hw_data->csr_ops);
drivers/crypto/intel/qat/qat_c3xxx/adf_c3xxx_hw_data.h +7 -0
@@ -3,6 +3,8 @@#ifndef ADF_C3XXX_HW_DATA_H_ #define ADF_C3XXX_HW_DATA_H_ +#include <linux/units.h> +/* PCIe configuration space */ #define ADF_C3XXX_PMISC_BAR 0#define ADF_C3XXX_ETR_BAR 1
@@ -19,6 +21,11 @@#define ADF_C3XXX_AE2FUNC_MAP_GRP_A_NUM_REGS 48#define ADF_C3XXX_AE2FUNC_MAP_GRP_B_NUM_REGS 6 +/* Clocks frequency */ +#define ADF_C3XXX_AE_FREQ (685 * HZ_PER_MHZ) +#define ADF_C3XXX_MIN_AE_FREQ (533 * HZ_PER_MHZ) +#define ADF_C3XXX_MAX_AE_FREQ (685 * HZ_PER_MHZ) +/* Firmware Binary */ #define ADF_C3XXX_FW "qat_c3xxx.bin"#define ADF_C3XXX_MMP "qat_c3xxx_mmp.bin"
drivers/crypto/intel/qat/qat_c62x/adf_c62x_hw_data.c +25 -0
@@ -1,6 +1,7 @@// SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0-only) /* Copyright(c) 2014 - 2021 Intel Corporation */ #include <adf_accel_devices.h> +#include <adf_clock.h> #include <adf_common_drv.h> #include <adf_gen2_config.h> #include <adf_gen2_dc.h>
@@ -50,6 +51,28 @@ static u32 get_ae_mask(struct adf_hw_device_data *self)return ~(fuses | straps) &ADF_C62X_ACCELENGINES_MASK ; } +static u32 get_ts_clock (struct adf_hw_device_data *self) +{ +/* +* Timestamp update interval is 16 AE clock ticks for c62x. +*/ +return self->clock_frequency /16 ; +} + +static int measure_clock (struct adf_accel_dev *accel_dev) +{ +u32 frequency; +int ret; + + ret =adf_dev_measure_clock (accel_dev, &frequency,ADF_C62X_MIN_AE_FREQ , +ADF_C62X_MAX_AE_FREQ ); +if (ret) +return ret; + + accel_dev->hw_device->clock_frequency = frequency; +return 0 ; +} +static u32 get_misc_bar_id (struct adf_hw_device_data *self) {return ADF_C62X_PMISC_BAR ;
@@ -129,6 +152,8 @@ void adf_init_hw_data_c62x(struct adf_hw_device_data *hw_data) hw_data->set_ssm_wdtimer = adf_gen2_set_ssm_wdtimer; hw_data->disable_iov = adf_disable_sriov; hw_data->dev_config = adf_gen2_dev_config; + hw_data->measure_clock = measure_clock; + hw_data->get_hb_clock = get_ts_clock;adf_gen2_init_pf_pfvf_ops (&hw_data->pfvf_ops);adf_gen2_init_hw_csr_ops (&hw_data->csr_ops);
drivers/crypto/intel/qat/qat_c62x/adf_c62x_hw_data.h +7 -0
@@ -3,6 +3,8 @@#ifndef ADF_C62X_HW_DATA_H_ #define ADF_C62X_HW_DATA_H_ +#include <linux/units.h> +/* PCIe configuration space */ #define ADF_C62X_SRAM_BAR 0#define ADF_C62X_PMISC_BAR 1
@@ -19,6 +21,11 @@#define ADF_C62X_AE2FUNC_MAP_GRP_A_NUM_REGS 80#define ADF_C62X_AE2FUNC_MAP_GRP_B_NUM_REGS 10 +/* Clocks frequency */ +#define ADF_C62X_AE_FREQ (685 * HZ_PER_MHZ) +#define ADF_C62X_MIN_AE_FREQ (533 * HZ_PER_MHZ) +#define ADF_C62X_MAX_AE_FREQ (800 * HZ_PER_MHZ) +/* Firmware Binary */ #define ADF_C62X_FW "qat_c62x.bin"#define ADF_C62X_MMP "qat_c62x_mmp.bin"
drivers/crypto/intel/qat/qat_common/Makefile +1 -0
@@ -18,6 +18,7 @@ intel_qat-objs := adf_cfg.o \ adf_gen2_dc.o \ adf_gen4_dc.o \ adf_gen4_timer.o \ + adf_clock.o \ qat_crypto.o \ qat_compression.o \ qat_comp_algs.o \
drivers/crypto/intel/qat/qat_common/adf_accel_devices.h +2 -0
@@ -190,6 +190,8 @@ struct adf_hw_device_data {int (*send_admin_init)(struct adf_accel_dev *accel_dev);int (*start_timer)(struct adf_accel_dev *accel_dev);void (*stop_timer)(struct adf_accel_dev *accel_dev); +uint32_t (*get_hb_clock)(struct adf_hw_device_data *self); +int (*measure_clock)(struct adf_accel_dev *accel_dev);int (*init_arb)(struct adf_accel_dev *accel_dev);void (*exit_arb)(struct adf_accel_dev *accel_dev);const u32 *(*get_arb_mapping)(struct adf_accel_dev *accel_dev);
drivers/crypto/intel/qat/qat_common/adf_admin.c +17 -0
@@ -15,6 +15,7 @@#define ADF_CONST_TABLE_SIZE 1024#define ADF_ADMIN_POLL_DELAY_US 20#define ADF_ADMIN_POLL_TIMEOUT_US (5 * USEC_PER_SEC) +#define ADF_ONE_AE 1static const u8 const_tab[1024 ]__aligned(1024) = {0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,
@@ -194,6 +195,22 @@ static int adf_set_fw_constants(struct adf_accel_dev *accel_dev)return adf_send_admin (accel_dev, &req, &resp, ae_mask); } +int adf_get_fw_timestamp (struct adf_accel_dev *accel_dev,u64 *timestamp) +{ +struct icp_qat_fw_init_admin_req req = { }; +struct icp_qat_fw_init_admin_resp resp; +unsigned int ae_mask =ADF_ONE_AE ; +int ret; + + req.cmd_id =ICP_QAT_FW_TIMER_GET ; + ret =adf_send_admin (accel_dev, &req, &resp, ae_mask); +if (ret) +return ret; + + *timestamp = resp.timestamp; +return 0 ; +} +static int adf_get_dc_capabilities (struct adf_accel_dev *accel_dev,u32 *capabilities) {
drivers/crypto/intel/qat/qat_common/adf_clock.c new file +131 -0
131 lines not on this page · raw message
drivers/crypto/intel/qat/qat_common/adf_clock.h new file +14 -0
@@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright(c) 2023 Intel Corporation */ +#ifndef ADF_CLOCK_H +#define ADF_CLOCK_H + +#include <linux/types.h> + +struct adf_accel_dev ; + +int adf_dev_measure_clock (struct adf_accel_dev *accel_dev,u32 *frequency, +u32 min,u32 max); +u64 adf_clock_get_current_time (void ); + +#endif
drivers/crypto/intel/qat/qat_common/adf_common_drv.h +1 -0
@@ -97,6 +97,7 @@ int adf_send_admin_init(struct adf_accel_dev *accel_dev);int adf_get_ae_fw_counters (struct adf_accel_dev *accel_dev,u16 ae,u64 *reqs,u64 *resps);int adf_init_admin_pm (struct adf_accel_dev *accel_dev,u32 idle_delay);int adf_send_admin_tim_sync (struct adf_accel_dev *accel_dev,u32 cnt); +int adf_get_fw_timestamp (struct adf_accel_dev *accel_dev,u64 *timestamp);int adf_init_arb (struct adf_accel_dev *accel_dev);void adf_exit_arb (struct adf_accel_dev *accel_dev);void adf_update_ring_arb (struct adf_etr_ring_data *ring);
drivers/crypto/intel/qat/qat_common/adf_init.c +8 -0
@@ -178,6 +178,14 @@ static int adf_dev_start(struct adf_accel_dev *accel_dev)return -EFAULT ; } +if (hw_data->measure_clock) { + ret = hw_data->measure_clock (accel_dev); +if (ret) { +dev_err (&GET_DEV (accel_dev),"Failed measure device clock\n" ); +return ret; + } + } +/* Set ssm watch dog timer */ if (hw_data->set_ssm_wdtimer) hw_data->set_ssm_wdtimer (accel_dev);
drivers/crypto/intel/qat/qat_common/icp_qat_fw_init_admin.h +1 -0
@@ -16,6 +16,7 @@ enum icp_qat_fw_init_admin_cmd_id {ICP_QAT_FW_HEARTBEAT_SYNC =7 ,ICP_QAT_FW_HEARTBEAT_GET =8 ,ICP_QAT_FW_COMP_CAPABILITY_GET =9 , +ICP_QAT_FW_TIMER_GET =19 , ICP_QAT_FW_PM_STATE_CONFIG = 128, };
drivers/crypto/intel/qat/qat_dh895xcc/adf_dh895xcc_hw_data.c +10 -0
@@ -44,6 +44,14 @@ static u32 get_misc_bar_id(struct adf_hw_device_data *self)return ADF_DH895XCC_PMISC_BAR ; } +static u32 get_ts_clock (struct adf_hw_device_data *self) +{ +/* +* Timestamp update interval is 16 AE clock ticks for dh895xcc. +*/ +return self->clock_frequency /16 ; +} +static u32 get_etr_bar_id (struct adf_hw_device_data *self) {return ADF_DH895XCC_ETR_BAR ;
@@ -237,6 +245,8 @@ void adf_init_hw_data_dh895xcc(struct adf_hw_device_data *hw_data) hw_data->reset_device = adf_reset_sbr; hw_data->disable_iov = adf_disable_sriov; hw_data->dev_config = adf_gen2_dev_config; + hw_data->clock_frequency =ADF_DH895X_AE_FREQ ; + hw_data->get_hb_clock = get_ts_clock;adf_gen2_init_pf_pfvf_ops (&hw_data->pfvf_ops); hw_data->pfvf_ops.enable_vf2pf_interrupts = enable_vf2pf_interrupts;
drivers/crypto/intel/qat/qat_dh895xcc/adf_dh895xcc_hw_data.h +5 -0
@@ -3,6 +3,8 @@#ifndef ADF_DH895x_HW_DATA_H_#define ADF_DH895x_HW_DATA_H_ +#include <linux/units.h> +/* PCIe configuration space */ #define ADF_DH895XCC_SRAM_BAR 0#define ADF_DH895XCC_PMISC_BAR 1
@@ -30,6 +32,9 @@#define ADF_DH895XCC_AE2FUNC_MAP_GRP_A_NUM_REGS 96#define ADF_DH895XCC_AE2FUNC_MAP_GRP_B_NUM_REGS 12 +/* Clocks frequency */ +#define ADF_DH895X_AE_FREQ (933 * HZ_PER_MHZ) +/* FW names */ #define ADF_DH895XCC_FW "qat_895xcc.bin"#define ADF_DH895XCC_MMP "qat_895xcc_mmp.bin"
Signature · 1 line
2.40.1
PATCH v4 4/5 crypto: qat - add heartbeat feature
In reply to Damian MuszynskiUnder some circumstances, firmware in the QAT devices could become unresponsive. The Heartbeat feature provides a mechanism to detect unresponsive devices.
The QAT FW periodically writes to memory a set of counters that allow to detect the liveness of a device. This patch adds logic to enable the reporting of those counters, analyze them and report if a device is alive or not.
In particular this adds
(1) heartbeat enabling, reading and detection logic
(2) reporting of heartbeat status and configuration via debugfs
(3) documentation for the newly created sysfs entries
(4) configuration of FW settings related to heartbeat, e.g. tick period
(5) logic to convert time in ms (provided by the user) to clock ticks
This patch introduces a new folder in debugfs called heartbeat with the following attributes:
- status
- queries_sent
- queries_failed
- config
All attributes except config are reading only. In particular:
- status file returns 0 when device is operational and -1 otherwise.
- queries_sent returns the total number of heartbeat queries sent.
- queries_failed returns the total number of heartbeat queries failed.
- config allows to adjust the frequency at which the firmware writes counters to memory. This period is given in milliseconds and it is fixed for GEN4 devices.
--- Documentation/ABI/testing/debugfs-driver-qat | 51 ++++ .../intel/qat/qat_4xxx/adf_4xxx_hw_data.c | 1 + drivers/crypto/intel/qat/qat_4xxx/adf_drv.c | 3 + .../intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c | 1 + .../intel/qat/qat_c62x/adf_c62x_hw_data.c | 1 + drivers/crypto/intel/qat/qat_common/Makefile | 2 + .../intel/qat/qat_common/adf_accel_devices.h | 7 + .../crypto/intel/qat/qat_common/adf_admin.c | 14 + .../intel/qat/qat_common/adf_cfg_strings.h | 2 + .../intel/qat/qat_common/adf_common_drv.h | 1 + .../crypto/intel/qat/qat_common/adf_dbgfs.c | 9 +- .../intel/qat/qat_common/adf_gen2_config.c | 7 + .../intel/qat/qat_common/adf_gen2_hw_data.h | 3 + .../intel/qat/qat_common/adf_gen4_hw_data.h | 3 + .../intel/qat/qat_common/adf_heartbeat.c | 268 ++++++++++++++++++ .../intel/qat/qat_common/adf_heartbeat.h | 73 +++++ .../qat/qat_common/adf_heartbeat_dbgfs.c | 194 +++++++++++++ .../qat/qat_common/adf_heartbeat_dbgfs.h | 12 + .../crypto/intel/qat/qat_common/adf_init.c | 7 + .../qat/qat_common/icp_qat_fw_init_admin.h | 4 + .../qat/qat_dh895xcc/adf_dh895xcc_hw_data.c | 1 + 21 files changed, 662 insertions(+), 2 deletions(-) create mode 100644 drivers/crypto/intel/qat/qat_common/adf_heartbeat.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_heartbeat.h create mode 100644 drivers/crypto/intel/qat/qat_common/adf_heartbeat_dbgfs.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_heartbeat_dbgfs.h
Documentation/ABI/testing/debugfs-driver-qat +51 -0
@@ -8,3 +8,54 @@ Description: (RO) Read returns the number of requests sent to the FW and the num <N>: Number of requests sent from Acceleration Engine N to FW and responses Acceleration Engine N received from FW + +What: /sys/kernel/debug/qat_<device>_<BDF>/heartbeat/config +Date: November 2023 +KernelVersion: 6.6 +Contact: [email protected] +Description: (RW) Read returns value of the Heartbeat update period. + Write to the file changes this period value. + + This period should reflect planned polling interval of device + health status. High frequency Heartbeat monitoring wastes CPU cycles + but minimizes the customer’s system downtime. Also, if there are + large service requests that take some time to complete, high frequency + Heartbeat monitoring could result in false reports of unresponsiveness + and in those cases, period needs to be increased. + + This parameter is effective only for c3xxx, c62x, dh895xcc devices. + 4xxx has this value internally fixed to 200ms. + + Default value is set to 500. Minimal allowed value is 200. + All values are expressed in milliseconds. + +What: /sys/kernel/debug/qat_<device>_<BDF>/heartbeat/queries_failed +Date: November 2023 +KernelVersion: 6.6 +Contact: [email protected] +Description: (RO) Read returns the number of times the device became unresponsive. + + Attribute returns value of the counter which is incremented when + status query results negative. + +What: /sys/kernel/debug/qat_<device>_<BDF>/heartbeat/queries_sent +Date: November 2023 +KernelVersion: 6.6 +Contact: [email protected] +Description: (RO) Read returns the number of times the control process checked + if the device is responsive. + + Attribute returns value of the counter which is incremented on + every status query. + +What: /sys/kernel/debug/qat_<device>_<BDF>/heartbeat/status +Date: November 2023 +KernelVersion: 6.6 +Contact: [email protected] +Description: (RO) Read returns the device health status. + + Returns 0 when device is healthy or -1 when is unresponsive + or the query failed to send. + + The driver does not monitor for Heartbeat. It is left for a user + to poll the status periodically.
drivers/crypto/intel/qat/qat_4xxx/adf_4xxx_hw_data.c +1 -0
@@ -521,6 +521,7 @@ void adf_init_hw_data_4xxx(struct adf_hw_device_data *hw_data, u32 dev_id) hw_data->start_timer = adf_gen4_timer_start; hw_data->stop_timer = adf_gen4_timer_stop; hw_data->get_hb_clock = get_heartbeat_clock; + hw_data->num_hb_ctrs =ADF_NUM_HB_CNT_PER_AE ;adf_gen4_init_hw_csr_ops (&hw_data->csr_ops);adf_gen4_init_pf_pfvf_ops (&hw_data->pfvf_ops);
drivers/crypto/intel/qat/qat_4xxx/adf_drv.c +3 -0
@@ -8,6 +8,7 @@#include <adf_cfg.h> #include <adf_common_drv.h> #include <adf_dbgfs.h> +#include <adf_heartbeat.h> #include "adf_4xxx_hw_data.h" #include "qat_compression.h"
@@ -77,6 +78,8 @@ static int adf_cfg_dev_init(struct adf_accel_dev *accel_dev)if (ret)return ret; +adf_heartbeat_save_cfg_param (accel_dev,ADF_CFG_HB_TIMER_MIN_MS ); +return 0 ; }
drivers/crypto/intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c +1 -0
@@ -152,6 +152,7 @@ void adf_init_hw_data_c3xxx(struct adf_hw_device_data *hw_data) hw_data->dev_config = adf_gen2_dev_config; hw_data->measure_clock = measure_clock; hw_data->get_hb_clock = get_ts_clock; + hw_data->num_hb_ctrs =ADF_NUM_HB_CNT_PER_AE ;adf_gen2_init_pf_pfvf_ops (&hw_data->pfvf_ops);adf_gen2_init_hw_csr_ops (&hw_data->csr_ops);
drivers/crypto/intel/qat/qat_c62x/adf_c62x_hw_data.c +1 -0
@@ -154,6 +154,7 @@ void adf_init_hw_data_c62x(struct adf_hw_device_data *hw_data) hw_data->dev_config = adf_gen2_dev_config; hw_data->measure_clock = measure_clock; hw_data->get_hb_clock = get_ts_clock; + hw_data->num_hb_ctrs =ADF_NUM_HB_CNT_PER_AE ;adf_gen2_init_pf_pfvf_ops (&hw_data->pfvf_ops);adf_gen2_init_hw_csr_ops (&hw_data->csr_ops);
drivers/crypto/intel/qat/qat_common/Makefile +2 -0
@@ -31,6 +31,8 @@ intel_qat-objs := adf_cfg.o \ intel_qat-$(CONFIG_DEBUG_FS) += adf_transport_debug.o \ adf_fw_counters.o \ + adf_heartbeat.o \ + adf_heartbeat_dbgfs.o \ adf_dbgfs.o intel_qat-$(CONFIG_PCI_IOV) += adf_sriov.o adf_vf_isr.o adf_pfvf_utils.o \
drivers/crypto/intel/qat/qat_common/adf_accel_devices.h +7 -0
@@ -233,6 +233,7 @@ struct adf_hw_device_data {u8 num_accel;u8 num_logical_accel;u8 num_engines; +u32 num_hb_ctrs; };/* CSR write macro */
@@ -245,6 +246,11 @@ struct adf_hw_device_data {#define ADF_CFG_NUM_SERVICES 4#define ADF_SRV_TYPE_BIT_LEN 3#define ADF_SRV_TYPE_MASK 0x7 +#define ADF_AE_ADMIN_THREAD 7 +#define ADF_NUM_THREADS_PER_AE 8 +#define ADF_NUM_PKE_STRAND 2 +#define ADF_AE_STRAND0_THREAD 8 +#define ADF_AE_STRAND1_THREAD 9#define GET_DEV (accel_dev) ((accel_dev)->accel_pci_dev.pci_dev->dev)#define GET_BARS (accel_dev) ((accel_dev)->accel_pci_dev.pci_bars)
@@ -301,6 +307,7 @@ struct adf_accel_dev {struct module *owner;struct adf_accel_pci accel_pci_dev;struct adf_timer *timer; +struct adf_heartbeat *heartbeat; union { struct { /* protects VF2PF interrupts access */
drivers/crypto/intel/qat/qat_common/adf_admin.c +14 -0
@@ -8,6 +8,7 @@#include <linux/dma-mapping.h> #include "adf_accel_devices.h" #include "adf_common_drv.h" +#include "adf_heartbeat.h" #include "icp_qat_fw_init_admin.h" #define ADF_ADMIN_MAILBOX_STRIDE 0x1000
@@ -270,6 +271,19 @@ int adf_send_admin_tim_sync(struct adf_accel_dev *accel_dev, u32 cnt)return adf_send_admin (accel_dev, &req, &resp, ae_mask); } +int adf_send_admin_hb_timer (struct adf_accel_dev *accel_dev,uint32_t ticks) +{ +u32 ae_mask = accel_dev->hw_device->ae_mask; +struct icp_qat_fw_init_admin_req req = { }; +struct icp_qat_fw_init_admin_resp resp; + + req.cmd_id =ICP_QAT_FW_HEARTBEAT_TIMER_SET ; + req.init_cfg_ptr = accel_dev->heartbeat->dma.phy_addr; + req.heartbeat_ticks = ticks; + +return adf_send_admin (accel_dev, &req, &resp, ae_mask); +} +/** * adf_send_admin_init() - Function sends init message to FW * @accel_dev: Pointer to acceleration device.
drivers/crypto/intel/qat/qat_common/adf_cfg_strings.h +2 -0
@@ -47,4 +47,6 @@#define ADF_ETRMGR_CORE_AFFINITY_FORMAT \ ADF_ETRMGR_BANK "%d" ADF_ETRMGR_CORE_AFFINITY#define ADF_ACCEL_STR "Accelerator%d" +#define ADF_HEARTBEAT_TIMER "HeartbeatTimer" +#endif
drivers/crypto/intel/qat/qat_common/adf_common_drv.h +1 -0
@@ -97,6 +97,7 @@ int adf_send_admin_init(struct adf_accel_dev *accel_dev);int adf_get_ae_fw_counters (struct adf_accel_dev *accel_dev,u16 ae,u64 *reqs,u64 *resps);int adf_init_admin_pm (struct adf_accel_dev *accel_dev,u32 idle_delay);int adf_send_admin_tim_sync (struct adf_accel_dev *accel_dev,u32 cnt); +int adf_send_admin_hb_timer (struct adf_accel_dev *accel_dev,uint32_t ticks);int adf_get_fw_timestamp (struct adf_accel_dev *accel_dev,u64 *timestamp);int adf_init_arb (struct adf_accel_dev *accel_dev);void adf_exit_arb (struct adf_accel_dev *accel_dev);
drivers/crypto/intel/qat/qat_common/adf_dbgfs.c +7 -2
@@ -7,6 +7,7 @@#include "adf_common_drv.h" #include "adf_dbgfs.h" #include "adf_fw_counters.h" +#include "adf_heartbeat_dbgfs.h" /** * adf_dbgfs_init() - add persistent debugfs entries
@@ -58,8 +59,10 @@ void adf_dbgfs_add(struct adf_accel_dev *accel_dev)if (!accel_dev->debugfs_dir)return ; -if (!accel_dev->is_vf) +if (!accel_dev->is_vf) {adf_fw_counters_dbgfs_add (accel_dev); +adf_heartbeat_dbgfs_add (accel_dev); + } }/**
@@ -71,6 +74,8 @@ void adf_dbgfs_rm(struct adf_accel_dev *accel_dev)if (!accel_dev->debugfs_dir)return ; -if (!accel_dev->is_vf) +if (!accel_dev->is_vf) { +adf_heartbeat_dbgfs_rm (accel_dev);adf_fw_counters_dbgfs_rm (accel_dev); + } }
drivers/crypto/intel/qat/qat_common/adf_gen2_config.c +7 -0
@@ -7,6 +7,7 @@#include "adf_common_drv.h" #include "qat_crypto.h" #include "qat_compression.h" +#include "adf_heartbeat.h" #include "adf_transport_access_macros.h" static int adf_gen2_crypto_dev_config (struct adf_accel_dev *accel_dev)
@@ -195,6 +196,12 @@ int adf_gen2_dev_config(struct adf_accel_dev *accel_dev)if (ret) goto err; + ret =adf_cfg_section_add (accel_dev,ADF_GENERAL_SEC ); +if (ret) + goto err; + +adf_heartbeat_save_cfg_param (accel_dev,ADF_CFG_HB_TIMER_DEFAULT_MS ); +set_bit (ADF_STATUS_CONFIGURED , &accel_dev->status);return ret;
drivers/crypto/intel/qat/qat_common/adf_gen2_hw_data.h +3 -0
@@ -145,6 +145,9 @@ do { \#define ADF_GEN2_CERRSSMSH (i) ((i) * 0x4000 + 0x10)#define ADF_GEN2_ERRSSMSH_EN BIT(3) +/* Number of heartbeat counter pairs */ +#define ADF_NUM_HB_CNT_PER_AE ADF_NUM_THREADS_PER_AE +/* Interrupts */ #define ADF_GEN2_SMIAPF0_MASK_OFFSET (0x3A000 + 0x28)#define ADF_GEN2_SMIAPF1_MASK_OFFSET (0x3A000 + 0x30)
drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.h +3 -0
@@ -136,6 +136,9 @@ do { \#define ADF_GEN4_VFLNOTIFY BIT(7) +/* Number of heartbeat counter pairs */ +#define ADF_NUM_HB_CNT_PER_AE ADF_NUM_THREADS_PER_AE +void adf_gen4_set_ssm_wdtimer (struct adf_accel_dev *accel_dev);void adf_gen4_init_hw_csr_ops (struct adf_hw_csr_ops *csr_ops);int adf_gen4_ring_pair_reset (struct adf_accel_dev *accel_dev,u32 bank_number);
drivers/crypto/intel/qat/qat_common/adf_heartbeat.c new file +268 -0
268 lines not on this page · raw message
drivers/crypto/intel/qat/qat_common/adf_heartbeat.h new file +73 -0
73 lines not on this page · raw message
drivers/crypto/intel/qat/qat_common/adf_heartbeat_dbgfs.c new file +194 -0
194 lines not on this page · raw message
drivers/crypto/intel/qat/qat_common/adf_heartbeat_dbgfs.h new file +12 -0
@@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright(c) 2023 Intel Corporation */ + +#ifndef ADF_HEARTBEAT_DBGFS_H_ +#define ADF_HEARTBEAT_DBGFS_H_ + +struct adf_accel_dev ; + +void adf_heartbeat_dbgfs_add (struct adf_accel_dev *accel_dev); +void adf_heartbeat_dbgfs_rm (struct adf_accel_dev *accel_dev); + +#endif /* ADF_HEARTBEAT_DBGFS_H_ */
drivers/crypto/intel/qat/qat_common/adf_init.c +7 -0
@@ -8,6 +8,7 @@#include "adf_cfg.h" #include "adf_common_drv.h" #include "adf_dbgfs.h" +#include "adf_heartbeat.h" static LIST_HEAD (service_table);static DEFINE_MUTEX (service_lock);
@@ -129,6 +130,8 @@ static int adf_dev_init(struct adf_accel_dev *accel_dev)return -EFAULT ; } +adf_heartbeat_init (accel_dev); +/* * Subservice initialisation is divided into two stages: init and start. * This is to facilitate any ordering dependencies between services
@@ -204,6 +207,8 @@ static int adf_dev_start(struct adf_accel_dev *accel_dev) } } +adf_heartbeat_start (accel_dev); +list_for_each (list_itr, &service_table) { service = list_entry(list_itr, struct service_hndl, list);if (service->event_hld (accel_dev,ADF_EVENT_START )) {
@@ -347,6 +352,8 @@ static void adf_dev_shutdown(struct adf_accel_dev *accel_dev)clear_bit (accel_dev->accel_id, service->init_status); } +adf_heartbeat_shutdown (accel_dev); + hw_data->disable_iov (accel_dev);if (test_bit (ADF_STATUS_IRQ_ALLOCATED , &accel_dev->status)) {
drivers/crypto/intel/qat/qat_common/icp_qat_fw_init_admin.h +4 -0
@@ -16,6 +16,7 @@ enum icp_qat_fw_init_admin_cmd_id {ICP_QAT_FW_HEARTBEAT_SYNC =7 ,ICP_QAT_FW_HEARTBEAT_GET =8 ,ICP_QAT_FW_COMP_CAPABILITY_GET =9 , +ICP_QAT_FW_HEARTBEAT_TIMER_SET =13 ,ICP_QAT_FW_TIMER_GET =19 , ICP_QAT_FW_PM_STATE_CONFIG = 128, };
@@ -41,6 +42,9 @@ struct icp_qat_fw_init_admin_req {struct {__u32 int_timer_ticks; }; +struct { +__u32 heartbeat_ticks; + };__u32 idle_filter; };
drivers/crypto/intel/qat/qat_dh895xcc/adf_dh895xcc_hw_data.c +1 -0
@@ -247,6 +247,7 @@ void adf_init_hw_data_dh895xcc(struct adf_hw_device_data *hw_data) hw_data->dev_config = adf_gen2_dev_config; hw_data->clock_frequency =ADF_DH895X_AE_FREQ ; hw_data->get_hb_clock = get_ts_clock; + hw_data->num_hb_ctrs =ADF_NUM_HB_CNT_PER_AE ;adf_gen2_init_pf_pfvf_ops (&hw_data->pfvf_ops); hw_data->pfvf_ops.enable_vf2pf_interrupts = enable_vf2pf_interrupts;
Signature · 1 line
2.40.1
PATCH v4 5/5 crypto: qat - add heartbeat counters check
In reply to Damian MuszynskiA firmware update for QAT GEN2 changed the format of a data structure used to report the heartbeat counters.
To support all firmware versions, extend the heartbeat logic with an algorithm that detects the number of counters returned by firmware. The algorithm detects the number of counters to be used (and size of the corresponding data structure) by the comparison the expected size of the data in memory, with the data which was written by the firmware.
Firmware detection is done one time during the first read of heartbeat debugfs file to avoid increasing the time needed to load the module.
--- .../intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c | 2 + .../intel/qat/qat_c62x/adf_c62x_hw_data.c | 2 + .../intel/qat/qat_common/adf_accel_devices.h | 1 + .../intel/qat/qat_common/adf_heartbeat.c | 68 +++++++++++++++++++ .../intel/qat/qat_common/adf_heartbeat.h | 6 ++ .../qat/qat_dh895xcc/adf_dh895xcc_hw_data.c | 2 + 6 files changed, 81 insertions(+)
drivers/crypto/intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c +2 -0
@@ -8,6 +8,7 @@#include <adf_gen2_hw_data.h> #include <adf_gen2_pfvf.h> #include "adf_c3xxx_hw_data.h" +#include "adf_heartbeat.h" #include "icp_qat_hw.h" /* Worker thread to service arbiter mappings */
@@ -153,6 +154,7 @@ void adf_init_hw_data_c3xxx(struct adf_hw_device_data *hw_data) hw_data->measure_clock = measure_clock; hw_data->get_hb_clock = get_ts_clock; hw_data->num_hb_ctrs =ADF_NUM_HB_CNT_PER_AE ; + hw_data->check_hb_ctrs = adf_heartbeat_check_ctrs;adf_gen2_init_pf_pfvf_ops (&hw_data->pfvf_ops);adf_gen2_init_hw_csr_ops (&hw_data->csr_ops);
drivers/crypto/intel/qat/qat_c62x/adf_c62x_hw_data.c +2 -0
@@ -8,6 +8,7 @@#include <adf_gen2_hw_data.h> #include <adf_gen2_pfvf.h> #include "adf_c62x_hw_data.h" +#include "adf_heartbeat.h" #include "icp_qat_hw.h" /* Worker thread to service arbiter mappings */
@@ -155,6 +156,7 @@ void adf_init_hw_data_c62x(struct adf_hw_device_data *hw_data) hw_data->measure_clock = measure_clock; hw_data->get_hb_clock = get_ts_clock; hw_data->num_hb_ctrs =ADF_NUM_HB_CNT_PER_AE ; + hw_data->check_hb_ctrs = adf_heartbeat_check_ctrs;adf_gen2_init_pf_pfvf_ops (&hw_data->pfvf_ops);adf_gen2_init_hw_csr_ops (&hw_data->csr_ops);
drivers/crypto/intel/qat/qat_common/adf_accel_devices.h +1 -0
@@ -190,6 +190,7 @@ struct adf_hw_device_data {int (*send_admin_init)(struct adf_accel_dev *accel_dev);int (*start_timer)(struct adf_accel_dev *accel_dev);void (*stop_timer)(struct adf_accel_dev *accel_dev); +void (*check_hb_ctrs)(struct adf_accel_dev *accel_dev);uint32_t (*get_hb_clock)(struct adf_hw_device_data *self);int (*measure_clock)(struct adf_accel_dev *accel_dev);int (*init_arb)(struct adf_accel_dev *accel_dev);
drivers/crypto/intel/qat/qat_common/adf_heartbeat.c +68 -0
98 lines not on this page · raw message
drivers/crypto/intel/qat/qat_common/adf_heartbeat.h +6 -0
@@ -24,6 +24,7 @@ struct adf_heartbeat {unsigned int hb_failed_counter;unsigned int hb_timer;u64 last_hb_check_time; +bool ctrs_cnt_checked;struct hb_dma_addr {dma_addr_t phy_addr;void *virt_addr;
@@ -48,6 +49,7 @@ int adf_heartbeat_save_cfg_param(struct adf_accel_dev *accel_dev, unsigned int timer_ms);void adf_heartbeat_status (struct adf_accel_dev *accel_dev,enum adf_device_heartbeat_status *hb_status); +void adf_heartbeat_check_ctrs (struct adf_accel_dev *accel_dev);#else static inline int adf_heartbeat_init (struct adf_accel_dev *accel_dev)
@@ -69,5 +71,9 @@ static inline int adf_heartbeat_save_cfg_param(struct adf_accel_dev *accel_dev, {return 0 ; } + +static inline void adf_heartbeat_check_ctrs (struct adf_accel_dev *accel_dev) +{ +}#endif #endif /* ADF_HEARTBEAT_H_ */
drivers/crypto/intel/qat/qat_dh895xcc/adf_dh895xcc_hw_data.c +2 -0
@@ -7,6 +7,7 @@#include <adf_gen2_hw_data.h> #include <adf_gen2_pfvf.h> #include "adf_dh895xcc_hw_data.h" +#include "adf_heartbeat.h" #include "icp_qat_hw.h" #define ADF_DH895XCC_VF_MSK 0xFFFFFFFF
@@ -248,6 +249,7 @@ void adf_init_hw_data_dh895xcc(struct adf_hw_device_data *hw_data) hw_data->clock_frequency =ADF_DH895X_AE_FREQ ; hw_data->get_hb_clock = get_ts_clock; hw_data->num_hb_ctrs =ADF_NUM_HB_CNT_PER_AE ; + hw_data->check_hb_ctrs = adf_heartbeat_check_ctrs;adf_gen2_init_pf_pfvf_ops (&hw_data->pfvf_ops); hw_data->pfvf_ops.enable_vf2pf_interrupts = enable_vf2pf_interrupts;
Signature · 1 line
2.40.1
Herbert Xu
maintainerQuoted 85 lines · Damian Muszynski
This set introduces support for the QAT heartbeat feature. It allows detection whenever device firmware or acceleration unit will hang. We're adding this feature to allow our clients having a tool with they could verify if all of the Quick Assist hardware resources are healthy and operational.
QAT device firmware periodically writes counters to a specified physical memory location. A pair of counters per thread is incremented at the start and end of the main processing loop within the firmware. Checking for Heartbeat consists of checking the validity of the pair of counter values for each thread. Stagnant counters indicate a firmware hang.
The first patch adds timestamp synchronization to the firmware. The second patch removes historical and never used HB definitions. Patch no. 3 is implementing the hardware clock frequency measuring interface.
The fourth introduces the main heartbeat implementation with the debugfs interface.
The last patch implements an algorithm that allows the code to detect which version of heartbeat API is used at the currently loaded firmware.Changes since v3:
- improved comment inmeasure_clock()as suggested by Andy Shevchenko
- changed release date and version for 6.6 in interface documentationChanges since v2:
- fixed build error on a few of architectures - reduced unnecessary 64bit division.Changes since v1:
- fixed build errors on a few of architectures - replaced macroDIV_ROUND_CLOSESTwithDIV_ROUND_CLOSEST_ULL
- included prerequisite patch "add internal timer for qat 4xxx" which initially was sent separately as this patchset was still in development.
- timer patch reworked to use delayed work as suggested by Herbert XuDamian Muszynski (5):
crypto: qat - add internal timer for qat 4xxx
crypto: qat - drop obsolete heartbeat interface
crypto: qat - add measure clock frequency
crypto: qat - add heartbeat feature
crypto: qat - add heartbeat counters checkDocumentation/ABI/testing/debugfs-driver-qat | 51 +++ .../intel/qat/qat_4xxx/adf_4xxx_hw_data.c | 14 + .../intel/qat/qat_4xxx/adf_4xxx_hw_data.h | 4 + drivers/crypto/intel/qat/qat_4xxx/adf_drv.c | 3 + .../intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c | 28 ++ .../intel/qat/qat_c3xxx/adf_c3xxx_hw_data.h | 7 + .../intel/qat/qat_c62x/adf_c62x_hw_data.c | 28 ++ .../intel/qat/qat_c62x/adf_c62x_hw_data.h | 7 + drivers/crypto/intel/qat/qat_common/Makefile | 4 + .../intel/qat/qat_common/adf_accel_devices.h | 13 + .../crypto/intel/qat/qat_common/adf_admin.c | 43 +++ .../intel/qat/qat_common/adf_cfg_strings.h | 2 + .../crypto/intel/qat/qat_common/adf_clock.c | 131 +++++++ .../crypto/intel/qat/qat_common/adf_clock.h | 14 + .../intel/qat/qat_common/adf_common_drv.h | 5 + .../crypto/intel/qat/qat_common/adf_dbgfs.c | 9 +- .../intel/qat/qat_common/adf_gen2_config.c | 7 + .../intel/qat/qat_common/adf_gen2_hw_data.h | 3 + .../intel/qat/qat_common/adf_gen4_hw_data.h | 3 + .../intel/qat/qat_common/adf_gen4_timer.c | 70 ++++ .../intel/qat/qat_common/adf_gen4_timer.h | 21 ++ .../intel/qat/qat_common/adf_heartbeat.c | 336 ++++++++++++++++++ .../intel/qat/qat_common/adf_heartbeat.h | 79 ++++ .../qat/qat_common/adf_heartbeat_dbgfs.c | 194 ++++++++++ .../qat/qat_common/adf_heartbeat_dbgfs.h | 12 + .../crypto/intel/qat/qat_common/adf_init.c | 28 ++ drivers/crypto/intel/qat/qat_common/adf_isr.c | 6 + .../qat/qat_common/icp_qat_fw_init_admin.h | 23 +- .../qat/qat_dh895xcc/adf_dh895xcc_hw_data.c | 13 + .../qat/qat_dh895xcc/adf_dh895xcc_hw_data.h | 5 + 30 files changed, 1147 insertions(+), 16 deletions(-) create mode 100644 drivers/crypto/intel/qat/qat_common/adf_clock.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_clock.h create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_timer.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_timer.h create mode 100644 drivers/crypto/intel/qat/qat_common/adf_heartbeat.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_heartbeat.h create mode 100644 drivers/crypto/intel/qat/qat_common/adf_heartbeat_dbgfs.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_heartbeat_dbgfs.h
Quoted 3 lines
base-commit:
67b9bc0df80cfa241fe7a9c2b857c3e3efde982aSignature · 1 line
2.40.1
All applied. Thanks.
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt