Skip to contentSkip to conversation

crypto: qat - improve recovery flows

PATCH In review Mun Chun Yep 2 participants 11 messages Feb 2, 2024 – Feb 9, 2024

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v2 cover crypto: qat - improve recovery flows

This set improves the error recovery flows in the QAT drivers and adds a mechanism to test it through an heartbeat simulator.

When a QAT device reports either a fatal error, or an AER fatal error, or fails an heartbeat check, the PF driver sends an error notification to the VFs through PFVF comms and if auto_reset is enabled then the device goes through reset flows for error recovery.
If SRIOV is enabled when an error is encountered, this is re-enabled after the reset cycle is done.

Changed in v2:
- Removed redundant default value in Kconfig
- Removed ccflags define, use the CONFIG option directly in the code
- Reworked the AER reset and recovery flow

Damian Muszynski (2):
crypto: qat - add heartbeat error simulator
crypto: qat - add auto reset on error

Furong Zhou (3):
crypto: qat - add fatal error notify method
crypto: qat - disable arbitration before reset
crypto: qat - limit heartbeat notifications

Mun Chun Yep (4):
crypto: qat - update PFVF protocol for recovery
crypto: qat - re-enable sriov after pf reset
crypto: qat - add fatal error notification
crypto: qat - improve aer error reset handling

 Documentation/ABI/testing/debugfs-driver-qat  |  26 ++++
 Documentation/ABI/testing/sysfs-driver-qat    |  20 +++
 drivers/crypto/intel/qat/Kconfig              |  14 +++
 drivers/crypto/intel/qat/qat_common/Makefile  |   2 +
 .../intel/qat/qat_common/adf_accel_devices.h  |   2 +
 drivers/crypto/intel/qat/qat_common/adf_aer.c | 116 +++++++++++++++++-
 .../intel/qat/qat_common/adf_cfg_strings.h    |   1 +
 .../intel/qat/qat_common/adf_common_drv.h     |  10 ++
 .../intel/qat/qat_common/adf_heartbeat.c      |  20 ++-
 .../intel/qat/qat_common/adf_heartbeat.h      |  21 ++++
 .../qat/qat_common/adf_heartbeat_dbgfs.c      |  52 ++++++++
 .../qat/qat_common/adf_heartbeat_inject.c     |  76 ++++++++++++
 .../intel/qat/qat_common/adf_hw_arbiter.c     |  25 ++++
 .../crypto/intel/qat/qat_common/adf_init.c    |  12 ++
 drivers/crypto/intel/qat/qat_common/adf_isr.c |   7 +-
 .../intel/qat/qat_common/adf_pfvf_msg.h       |   7 +-
 .../intel/qat/qat_common/adf_pfvf_pf_msg.c    |  64 +++++++++-
 .../intel/qat/qat_common/adf_pfvf_pf_msg.h    |  21 ++++
 .../intel/qat/qat_common/adf_pfvf_pf_proto.c  |   8 ++
 .../intel/qat/qat_common/adf_pfvf_vf_proto.c  |   6 +
 .../crypto/intel/qat/qat_common/adf_sriov.c   |  38 +++++-
 .../crypto/intel/qat/qat_common/adf_sysfs.c   |  37 ++++++
 22 files changed, 571 insertions(+), 14 deletions(-)
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_heartbeat_inject.c
Signature · 1 line
2.34.1
#1 lore raw

PATCH v2 1/9 crypto: qat - add heartbeat error simulator

From: Damian Muszynski <[email protected]>

Add a mechanism that allows to inject a heartbeat error for testing purposes.
A new attribute inject_error is added to debugfs for each QAT device. Upon a write on this attribute, the driver will inject an error on the device which can then be detected by the heartbeat feature. Errors are breaking the device functionality thus they require a device reset in order to be recovered.

This functionality is not compiled by default, to enable it CRYPTO_DEV_QAT_ERROR_INJECTION must be set.

---
 Documentation/ABI/testing/debugfs-driver-qat  | 26 +++++++
 drivers/crypto/intel/qat/Kconfig              | 14 ++++
 drivers/crypto/intel/qat/qat_common/Makefile  |  2 +
 .../intel/qat/qat_common/adf_common_drv.h     |  1 +
 .../intel/qat/qat_common/adf_heartbeat.c      |  6 --
 .../intel/qat/qat_common/adf_heartbeat.h      | 18 +++++
 .../qat/qat_common/adf_heartbeat_dbgfs.c      | 52 +++++++++++++
 .../qat/qat_common/adf_heartbeat_inject.c     | 76 +++++++++++++++++++
 .../intel/qat/qat_common/adf_hw_arbiter.c     | 25 ++++++
 9 files changed, 214 insertions(+), 6 deletions(-)
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_heartbeat_inject.c
Documentation/ABI/testing/debugfs-driver-qat +26 -0
index b2db010d851e..bd6793760f29 100644
@@ -81,3 +81,29 @@ Description:	(RO) Read returns, for each Acceleration Engine (AE), the number
 			<N>: Number of Compress and Verify (CnV) errors and type
 			     of the last CnV error detected by Acceleration
 			     Engine N.
+
+What:		/sys/kernel/debug/qat_<device>_<BDF>/heartbeat/inject_error
+Date:		March 2024
+KernelVersion:	6.8
+Contact:	[email protected]
+Description:	(WO) Write to inject an error that simulates an heartbeat
+		failure. This is to be used for testing purposes.
+
+		After writing this file, the driver stops arbitration on a
+		random engine and disables the fetching of heartbeat counters.
+		If a workload is running on the device, a job submitted to the
+		accelerator might not get a response and a read of the
+		`heartbeat/status` attribute might report -1, i.e. device
+		unresponsive.
+		The error is unrecoverable thus the device must be restarted to
+		restore its functionality.
+
+		This attribute is available only when the kernel is built with
+		CONFIG_CRYPTO_DEV_QAT_ERROR_INJECTION=y.
+
+		A write of 1 enables error injection.
+
+		The following example shows how to enable error injection::
+
+			# cd /sys/kernel/debug/qat_<device>_<BDF>
+			# echo 1 > heartbeat/inject_error
drivers/crypto/intel/qat/Kconfig +14 -0
index c120f6715a09..02fb8abe4e6e 100644
@@ -106,3 +106,17 @@ config CRYPTO_DEV_QAT_C62XVF
 
 	  To compile this as a module, choose M here: the module
 	  will be called qat_c62xvf.
+
+config CRYPTO_DEV_QAT_ERROR_INJECTION
+	bool "Support for Intel(R) QAT Devices Heartbeat Error Injection"
+	depends on CRYPTO_DEV_QAT
+	depends on DEBUG_FS
+	help
+	  Enables a mechanism that allows to inject a heartbeat error on
+	  Intel(R) QuickAssist devices for testing purposes.
+
+	  This is intended for developer use only.
+	  If unsure, say N.
+
+	  This functionality is available via debugfs entry of the Intel(R)
+	  QuickAssist device
drivers/crypto/intel/qat/qat_common/Makefile +2 -0
index 6908727bff3b..5915cde8a7aa 100644
@@ -53,3 +53,5 @@ intel_qat-$(CONFIG_PCI_IOV) += adf_sriov.o adf_vf_isr.o adf_pfvf_utils.o \
 			       adf_pfvf_pf_msg.o adf_pfvf_pf_proto.o \
 			       adf_pfvf_vf_msg.o adf_pfvf_vf_proto.o \
 			       adf_gen2_pfvf.o adf_gen4_pfvf.o
+
+intel_qat-$(CONFIG_CRYPTO_DEV_QAT_ERROR_INJECTION) += adf_heartbeat_inject.o
drivers/crypto/intel/qat/qat_common/adf_common_drv.h +1 -0
index f06188033a93..0baae42deb3a 100644
@@ -90,6 +90,7 @@ void adf_exit_aer(void);
 int adf_init_arb(struct adf_accel_dev *accel_dev);
 void adf_exit_arb(struct adf_accel_dev *accel_dev);
 void adf_update_ring_arb(struct adf_etr_ring_data *ring);
+int adf_disable_arb_thd(struct adf_accel_dev *accel_dev, u32 ae, u32 thr);
 
 int adf_dev_get(struct adf_accel_dev *accel_dev);
 void adf_dev_put(struct adf_accel_dev *accel_dev);
drivers/crypto/intel/qat/qat_common/adf_heartbeat.c +0 -6
index 13f48d2f6da8..f88b1bc6857e 100644
@@ -23,12 +23,6 @@
 
 #define ADF_HB_EMPTY_SIG 0xA5A5A5A5
 
-/* Heartbeat counter pair */
-struct hb_cnt_pair {
-	__u16 resp_heartbeat_cnt;
-	__u16 req_heartbeat_cnt;
-};
-
 static int adf_hb_check_polling_freq(struct adf_accel_dev *accel_dev)
 {
 	u64 curr_time = adf_clock_get_current_time();
drivers/crypto/intel/qat/qat_common/adf_heartbeat.h +18 -0
index b22e3cb29798..24c3f4f24c86 100644
@@ -19,6 +19,12 @@ enum adf_device_heartbeat_status {
 	HB_DEV_UNSUPPORTED,
 };
 
+/* Heartbeat counter pair */
+struct hb_cnt_pair {
+	__u16 resp_heartbeat_cnt;
+	__u16 req_heartbeat_cnt;
+};
+
 struct adf_heartbeat {
 	unsigned int hb_sent_counter;
 	unsigned int hb_failed_counter;
@@ -35,6 +41,9 @@ struct adf_heartbeat {
 		struct dentry *cfg;
 		struct dentry *sent;
 		struct dentry *failed;
+#ifdef CONFIG_CRYPTO_DEV_QAT_ERROR_INJECTION
+		struct dentry *inject_error;
+#endif
 	} dbgfs;
 };
 
@@ -51,6 +60,15 @@ void adf_heartbeat_status(struct adf_accel_dev *accel_dev,
 			  enum adf_device_heartbeat_status *hb_status);
 void adf_heartbeat_check_ctrs(struct adf_accel_dev *accel_dev);
 
+#ifdef CONFIG_CRYPTO_DEV_QAT_ERROR_INJECTION
+int adf_heartbeat_inject_error(struct adf_accel_dev *accel_dev);
+#else
+static inline int adf_heartbeat_inject_error(struct adf_accel_dev *accel_dev)
+{
+	return -EPERM;
+}
+#endif
+
 #else
 static inline int adf_heartbeat_init(struct adf_accel_dev *accel_dev)
 {
drivers/crypto/intel/qat/qat_common/adf_heartbeat_dbgfs.c +52 -0

70 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/adf_heartbeat_inject.c new file +76 -0

76 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/adf_hw_arbiter.c +25 -0
index da6956699246..65bd26b25abc 100644
@@ -103,3 +103,28 @@ void adf_exit_arb(struct adf_accel_dev *accel_dev)
 		csr_ops->write_csr_ring_srv_arb_en(csr, i, 0);
 }
 EXPORT_SYMBOL_GPL(adf_exit_arb);
+
+int adf_disable_arb_thd(struct adf_accel_dev *accel_dev, u32 ae, u32 thr)
+{
+	void __iomem *csr = accel_dev->transport->banks[0].csr_addr;
+	struct adf_hw_device_data *hw_data = accel_dev->hw_device;
+	const u32 *thd_2_arb_cfg;
+	struct arb_info info;
+	u32 ae_thr_map;
+
+	if (ADF_AE_STRAND0_THREAD == thr || ADF_AE_STRAND1_THREAD == thr)
+		thr = ADF_AE_ADMIN_THREAD;
+
+	hw_data->get_arb_info(&info);
+	thd_2_arb_cfg = hw_data->get_arb_mapping(accel_dev);
+	if (!thd_2_arb_cfg)
+		return -EFAULT;
+
+	/* Disable scheduling for this particular AE and thread */
+	ae_thr_map = *(thd_2_arb_cfg + ae);
+	ae_thr_map &= ~(GENMASK(3, 0) << (thr * BIT(2)));
+
+	WRITE_CSR_ARB_WT2SAM(csr, info.arb_offset, info.wt2sam_offset, ae,
+			     ae_thr_map);
+	return 0;
+}
Signature · 1 line
2.34.1
#2 lore raw

PATCH v2 2/9 crypto: qat - add fatal error notify method

In reply to Mun Chun Yep

From: Furong Zhou <[email protected]>

Add error notify method to report a fatal error event to all the subsystems registered. In addition expose an API, adf_notify_fatal_error(), that allows to trigger a fatal error notification asynchronously in the context of a workqueue.

This will be invoked when a fatal error is detected by the ISR or through Heartbeat.

---
 drivers/crypto/intel/qat/qat_common/adf_aer.c | 30 +++++++++++++++++++
 .../intel/qat/qat_common/adf_common_drv.h     |  3 ++
 .../crypto/intel/qat/qat_common/adf_init.c    | 12 ++++++++
 3 files changed, 45 insertions(+)
drivers/crypto/intel/qat/qat_common/adf_aer.c +30 -0
index a39e70bd4b21..22a43b4b8315 100644
@@ -8,6 +8,11 @@
 #include "adf_accel_devices.h"
 #include "adf_common_drv.h"
 
+struct adf_fatal_error_data {
+	struct adf_accel_dev *accel_dev;
+	struct work_struct work;
+};
+
 static struct workqueue_struct *device_reset_wq;
 
 static pci_ers_result_t adf_error_detected(struct pci_dev *pdev,
@@ -171,6 +176,31 @@ const struct pci_error_handlers adf_err_handler = {
 };
 EXPORT_SYMBOL_GPL(adf_err_handler);
 
+static void adf_notify_fatal_error_worker(struct work_struct *work)
+{
+	struct adf_fatal_error_data *wq_data =
+			container_of(work, struct adf_fatal_error_data, work);
+	struct adf_accel_dev *accel_dev = wq_data->accel_dev;
+
+	adf_error_notifier(accel_dev);
+	kfree(wq_data);
+}
+
+int adf_notify_fatal_error(struct adf_accel_dev *accel_dev)
+{
+	struct adf_fatal_error_data *wq_data;
+
+	wq_data = kzalloc(sizeof(*wq_data), GFP_ATOMIC);
+	if (!wq_data)
+		return -ENOMEM;
+
+	wq_data->accel_dev = accel_dev;
+	INIT_WORK(&wq_data->work, adf_notify_fatal_error_worker);
+	adf_misc_wq_queue_work(&wq_data->work);
+
+	return 0;
+}
+
 int adf_init_aer(void)
 {
 	device_reset_wq = alloc_workqueue("qat_device_reset_wq",
drivers/crypto/intel/qat/qat_common/adf_common_drv.h +3 -0
index 0baae42deb3a..8c062d5a8db2 100644
@@ -40,6 +40,7 @@ enum adf_event {
 	ADF_EVENT_SHUTDOWN,
 	ADF_EVENT_RESTARTING,
 	ADF_EVENT_RESTARTED,
+	ADF_EVENT_FATAL_ERROR,
 };
 
 struct service_hndl {
@@ -60,6 +61,8 @@ int adf_dev_restart(struct adf_accel_dev *accel_dev);
 
 void adf_devmgr_update_class_index(struct adf_hw_device_data *hw_data);
 void adf_clean_vf_map(bool);
+int adf_notify_fatal_error(struct adf_accel_dev *accel_dev);
+void adf_error_notifier(struct adf_accel_dev *accel_dev);
 int adf_devmgr_add_dev(struct adf_accel_dev *accel_dev,
 		       struct adf_accel_dev *pf);
 void adf_devmgr_rm_dev(struct adf_accel_dev *accel_dev,
drivers/crypto/intel/qat/qat_common/adf_init.c +12 -0
index f43ae9111553..74f0818c0703 100644
@@ -433,6 +433,18 @@ int adf_dev_restarted_notify(struct adf_accel_dev *accel_dev)
 	return 0;
 }
 
+void adf_error_notifier(struct adf_accel_dev *accel_dev)
+{
+	struct service_hndl *service;
+
+	list_for_each_entry(service, &service_table, list) {
+		if (service->event_hld(accel_dev, ADF_EVENT_FATAL_ERROR))
+			dev_err(&GET_DEV(accel_dev),
+				"Failed to send error event to %s.\n",
+				service->name);
+	}
+}
+
 static int adf_dev_shutdown_cache_cfg(struct adf_accel_dev *accel_dev)
 {
 	char services[ADF_CFG_MAX_VAL_LEN_IN_BYTES] = {0};
Signature · 1 line
2.34.1
#3 lore raw

PATCH v2 3/9 crypto: qat - disable arbitration before reset

In reply to Mun Chun Yep

From: Furong Zhou <[email protected]>

Disable arbitration to avoid new requests to be processed before resetting a device.

This is needed so that new requests are not fetched when an error is detected.

---
 drivers/crypto/intel/qat/qat_common/adf_aer.c | 8 ++++++++
 1 file changed, 8 insertions(+)
drivers/crypto/intel/qat/qat_common/adf_aer.c +8 -0
index 22a43b4b8315..acbbd32bd815 100644
@@ -181,8 +181,16 @@ static void adf_notify_fatal_error_worker(struct work_struct *work)
 	struct adf_fatal_error_data *wq_data =
 			container_of(work, struct adf_fatal_error_data, work);
 	struct adf_accel_dev *accel_dev = wq_data->accel_dev;
+	struct adf_hw_device_data *hw_device = accel_dev->hw_device;
 
 	adf_error_notifier(accel_dev);
+
+	if (!accel_dev->is_vf) {
+		/* Disable arbitration to stop processing of new requests */
+		if (hw_device->exit_arb)
+			hw_device->exit_arb(accel_dev);
+	}
+
 	kfree(wq_data);
 }
 
Signature · 1 line
2.34.1
#4 lore raw

PATCH v2 4/9 crypto: qat - update PFVF protocol for recovery

In reply to Mun Chun Yep

Update the PFVF logic to handle restart and recovery. This adds the following functions:

* adf_pf2vf_notify_fatal_error(): allows the PF to notify VFs that the device detected a fatal error and requires a reset. This sends to VF the event ADF_PF2VF_MSGTYPE_FATAL_ERROR.
* adf_pf2vf_wait_for_restarting_complete(): allows the PF to wait for ADF_VF2PF_MSGTYPE_RESTARTING_COMPLETE events from active VFs before proceeding with a reset.
* adf_pf2vf_notify_restarted(): enables the PF to notify VFs with an ADF_PF2VF_MSGTYPE_RESTARTED event after recovery, indicating that the device is back to normal. This prompts VF drivers switch back to use the accelerator for workload processing.

These changes improve the communication and synchronization between PF and VF drivers during system restart and recovery processes.

---
 .../intel/qat/qat_common/adf_accel_devices.h  |  1 +
 drivers/crypto/intel/qat/qat_common/adf_aer.c |  3 +
 .../intel/qat/qat_common/adf_pfvf_msg.h       |  7 +-
 .../intel/qat/qat_common/adf_pfvf_pf_msg.c    | 64 ++++++++++++++++++-
 .../intel/qat/qat_common/adf_pfvf_pf_msg.h    | 21 ++++++
 .../intel/qat/qat_common/adf_pfvf_pf_proto.c  |  8 +++
 .../intel/qat/qat_common/adf_pfvf_vf_proto.c  |  6 ++
 .../crypto/intel/qat/qat_common/adf_sriov.c   |  1 +
 8 files changed, 109 insertions(+), 2 deletions(-)
drivers/crypto/intel/qat/qat_common/adf_accel_devices.h +1 -0
index a16c7e6edc65..4a3c36aaa7ca 100644
@@ -332,6 +332,7 @@ struct adf_accel_vf_info {
 	struct ratelimit_state vf2pf_ratelimit;
 	u32 vf_nr;
 	bool init;
+	bool restarting;
 	u8 vf_compat_ver;
 };
 
drivers/crypto/intel/qat/qat_common/adf_aer.c +3 -0
index acbbd32bd815..ecb114e1b59f 100644
@@ -7,6 +7,7 @@
 #include <linux/delay.h>
 #include "adf_accel_devices.h"
 #include "adf_common_drv.h"
+#include "adf_pfvf_pf_msg.h"
 
 struct adf_fatal_error_data {
 	struct adf_accel_dev *accel_dev;
@@ -189,6 +190,8 @@ static void adf_notify_fatal_error_worker(struct work_struct *work)
 		/* Disable arbitration to stop processing of new requests */
 		if (hw_device->exit_arb)
 			hw_device->exit_arb(accel_dev);
+		if (accel_dev->pf.vf_info)
+			adf_pf2vf_notify_fatal_error(accel_dev);
 	}
 
 	kfree(wq_data);
drivers/crypto/intel/qat/qat_common/adf_pfvf_msg.h +6 -1
index 204a42438992..d1b3ef9cadac 100644
@@ -99,6 +99,8 @@ enum pf2vf_msgtype {
 	ADF_PF2VF_MSGTYPE_RESTARTING		= 0x01,
 	ADF_PF2VF_MSGTYPE_VERSION_RESP		= 0x02,
 	ADF_PF2VF_MSGTYPE_BLKMSG_RESP		= 0x03,
+	ADF_PF2VF_MSGTYPE_FATAL_ERROR		= 0x04,
+	ADF_PF2VF_MSGTYPE_RESTARTED		= 0x05,
 /* Values from 0x10 are Gen4 specific, message type is only 4 bits in Gen2 devices. */
 	ADF_PF2VF_MSGTYPE_RP_RESET_RESP		= 0x10,
 };
@@ -112,6 +114,7 @@ enum vf2pf_msgtype {
 	ADF_VF2PF_MSGTYPE_LARGE_BLOCK_REQ	= 0x07,
 	ADF_VF2PF_MSGTYPE_MEDIUM_BLOCK_REQ	= 0x08,
 	ADF_VF2PF_MSGTYPE_SMALL_BLOCK_REQ	= 0x09,
+	ADF_VF2PF_MSGTYPE_RESTARTING_COMPLETE	= 0x0a,
 /* Values from 0x10 are Gen4 specific, message type is only 4 bits in Gen2 devices. */
 	ADF_VF2PF_MSGTYPE_RP_RESET		= 0x10,
 };
@@ -124,8 +127,10 @@ enum pfvf_compatibility_version {
 	ADF_PFVF_COMPAT_FAST_ACK		= 0x03,
 	/* Ring to service mapping support for non-standard mappings */
 	ADF_PFVF_COMPAT_RING_TO_SVC_MAP		= 0x04,
+	/* Fallback compat */
+	ADF_PFVF_COMPAT_FALLBACK		= 0x05,
 	/* Reference to the latest version */
-	ADF_PFVF_COMPAT_THIS_VERSION		= 0x04,
+	ADF_PFVF_COMPAT_THIS_VERSION		= 0x05,
 };
 
 /* PF->VF Version Response */
drivers/crypto/intel/qat/qat_common/adf_pfvf_pf_msg.c +63 -1

84 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/adf_pfvf_pf_msg.h +21 -0
index e8982d1ac896..f203d88c919c 100644
@@ -5,7 +5,28 @@
 
 #include "adf_accel_devices.h"
 
+#if defined(CONFIG_PCI_IOV)
 void adf_pf2vf_notify_restarting(struct adf_accel_dev *accel_dev);
+void adf_pf2vf_wait_for_restarting_complete(struct adf_accel_dev *accel_dev);
+void adf_pf2vf_notify_restarted(struct adf_accel_dev *accel_dev);
+void adf_pf2vf_notify_fatal_error(struct adf_accel_dev *accel_dev);
+#else
+static inline void adf_pf2vf_notify_restarting(struct adf_accel_dev *accel_dev)
+{
+}
+
+static inline void adf_pf2vf_wait_for_restarting_complete(struct adf_accel_dev *accel_dev)
+{
+}
+
+static inline void adf_pf2vf_notify_restarted(struct adf_accel_dev *accel_dev)
+{
+}
+
+static inline void adf_pf2vf_notify_fatal_error(struct adf_accel_dev *accel_dev)
+{
+}
+#endif
 
 typedef int (*adf_pf2vf_blkmsg_provider)(struct adf_accel_dev *accel_dev,
 					 u8 *buffer, u8 compat);
drivers/crypto/intel/qat/qat_common/adf_pfvf_pf_proto.c +8 -0
index 388e58bcbcaf..9ab93fbfefde 100644
@@ -291,6 +291,14 @@ static int adf_handle_vf2pf_msg(struct adf_accel_dev *accel_dev, u8 vf_nr,
 		vf_info->init = false;
 		}
 		break;
+	case ADF_VF2PF_MSGTYPE_RESTARTING_COMPLETE:
+		{
+		dev_dbg(&GET_DEV(accel_dev),
+			"Restarting Complete received from VF%d\n", vf_nr);
+		vf_info->restarting = false;
+		vf_info->init = false;
+		}
+		break;
 	case ADF_VF2PF_MSGTYPE_LARGE_BLOCK_REQ:
 	case ADF_VF2PF_MSGTYPE_MEDIUM_BLOCK_REQ:
 	case ADF_VF2PF_MSGTYPE_SMALL_BLOCK_REQ:
drivers/crypto/intel/qat/qat_common/adf_pfvf_vf_proto.c +6 -0
index 1015155b6374..dc284a089c88 100644
@@ -308,6 +308,12 @@ static bool adf_handle_pf2vf_msg(struct adf_accel_dev *accel_dev,
 
 		adf_pf2vf_handle_pf_restarting(accel_dev);
 		return false;
+	case ADF_PF2VF_MSGTYPE_RESTARTED:
+		dev_dbg(&GET_DEV(accel_dev), "Restarted message received from PF\n");
+		return true;
+	case ADF_PF2VF_MSGTYPE_FATAL_ERROR:
+		dev_err(&GET_DEV(accel_dev), "Fatal error received from PF\n");
+		return true;
 	case ADF_PF2VF_MSGTYPE_VERSION_RESP:
 	case ADF_PF2VF_MSGTYPE_BLKMSG_RESP:
 	case ADF_PF2VF_MSGTYPE_RP_RESET_RESP:
drivers/crypto/intel/qat/qat_common/adf_sriov.c +1 -0
index f44025bb6f99..cb2a9830f192 100644
@@ -103,6 +103,7 @@ void adf_disable_sriov(struct adf_accel_dev *accel_dev)
 		return;
 
 	adf_pf2vf_notify_restarting(accel_dev);
+	adf_pf2vf_wait_for_restarting_complete(accel_dev);
 	pci_disable_sriov(accel_to_pci_dev(accel_dev));
 
 	/* Disable VF to PF interrupts */
Signature · 1 line
2.34.1
#5 lore raw

PATCH v2 5/9 crypto: qat - re-enable sriov after pf reset

In reply to Mun Chun Yep

When a Physical Function (PF) is reset, SR-IOV gets disabled, making the associated Virtual Functions (VFs) unavailable. Even after reset and using pci_restore_state, VFs remain uncreated because the numvfs still at 0. Therefore, it's necessary to reconfigure SR-IOV to re-enable VFs.

This commit introduces the ADF_SRIOV_ENABLED configuration flag to cache the SR-IOV enablement state. SR-IOV is only re-enabled if it was previously configured.

This commit also introduces a dedicated workqueue without WQ_MEM_RECLAIM flag for enabling SR-IOV during Heartbeat and CPM error resets, preventing workqueue flushing warning.

This patch is based on earlier work done by Shashank Gupta.

---
 drivers/crypto/intel/qat/qat_common/adf_aer.c | 40 ++++++++++++++++++-
 .../intel/qat/qat_common/adf_cfg_strings.h    |  1 +
 .../intel/qat/qat_common/adf_common_drv.h     |  5 +++
 .../crypto/intel/qat/qat_common/adf_sriov.c   | 37 +++++++++++++++--
 4 files changed, 79 insertions(+), 4 deletions(-)
drivers/crypto/intel/qat/qat_common/adf_aer.c +39 -1
index ecb114e1b59f..cd273b31db0e 100644
@@ -15,6 +15,7 @@ struct adf_fatal_error_data {
 };
 
 static struct workqueue_struct *device_reset_wq;
+static struct workqueue_struct *device_sriov_wq;
 
 static pci_ers_result_t adf_error_detected(struct pci_dev *pdev,
 					   pci_channel_state_t state)
@@ -43,6 +44,13 @@ struct adf_reset_dev_data {
 	struct work_struct reset_work;
 };
 
+/* sriov dev data */
+struct adf_sriov_dev_data {
+	struct adf_accel_dev *accel_dev;
+	struct completion compl;
+	struct work_struct sriov_work;
+};
+
 void adf_reset_sbr(struct adf_accel_dev *accel_dev)
 {
 	struct pci_dev *pdev = accel_to_pci_dev(accel_dev);
@@ -88,11 +96,22 @@ void adf_dev_restore(struct adf_accel_dev *accel_dev)
 	}
 }
 
+static void adf_device_sriov_worker(struct work_struct *work)
+{
+	struct adf_sriov_dev_data *sriov_data =
+		container_of(work, struct adf_sriov_dev_data, sriov_work);
+
+	adf_reenable_sriov(sriov_data->accel_dev);
+	complete(&sriov_data->compl);
+}
+
 static void adf_device_reset_worker(struct work_struct *work)
 {
 	struct adf_reset_dev_data *reset_data =
 		  container_of(work, struct adf_reset_dev_data, reset_work);
 	struct adf_accel_dev *accel_dev = reset_data->accel_dev;
+	unsigned long wait_jiffies = msecs_to_jiffies(10000);
+	struct adf_sriov_dev_data sriov_data;
 
 	adf_dev_restarting_notify(accel_dev);
 	if (adf_dev_restart(accel_dev)) {
@@ -103,6 +122,14 @@ static void adf_device_reset_worker(struct work_struct *work)
 		WARN(1, "QAT: device restart failed. Device is unusable\n");
 		return;
 	}
+
+	sriov_data.accel_dev = accel_dev;
+	init_completion(&sriov_data.compl);
+	INIT_WORK(&sriov_data.sriov_work, adf_device_sriov_worker);
+	queue_work(device_sriov_wq, &sriov_data.sriov_work);
+	if (wait_for_completion_timeout(&sriov_data.compl, wait_jiffies))
+		adf_pf2vf_notify_restarted(accel_dev);
+
 	adf_dev_restarted_notify(accel_dev);
 	clear_bit(ADF_STATUS_RESTARTING, &accel_dev->status);
 
@@ -216,7 +243,14 @@ int adf_init_aer(void)
 {
 	device_reset_wq = alloc_workqueue("qat_device_reset_wq",
 					  WQ_MEM_RECLAIM, 0);
-	return !device_reset_wq ? -EFAULT : 0;
+	if (!device_reset_wq)
+		return -EFAULT;
+
+	device_sriov_wq = alloc_workqueue("qat_device_sriov_wq", 0, 0);
+	if (!device_sriov_wq)
+		return -EFAULT;
+
+	return 0;
 }
 
 void adf_exit_aer(void)
@@ -224,4 +258,8 @@ void adf_exit_aer(void)
 	if (device_reset_wq)
 		destroy_workqueue(device_reset_wq);
 	device_reset_wq = NULL;
+
+	if (device_sriov_wq)
+		destroy_workqueue(device_sriov_wq);
+	device_sriov_wq = NULL;
 }
drivers/crypto/intel/qat/qat_common/adf_cfg_strings.h +1 -0
index 322b76903a73..e015ad6cace2 100644
@@ -49,5 +49,6 @@
 	ADF_ETRMGR_BANK "%d" ADF_ETRMGR_CORE_AFFINITY
 #define ADF_ACCEL_STR "Accelerator%d"
 #define ADF_HEARTBEAT_TIMER  "HeartbeatTimer"
+#define ADF_SRIOV_ENABLED "SriovEnabled"
 
 #endif
drivers/crypto/intel/qat/qat_common/adf_common_drv.h +5 -0
index 8c062d5a8db2..10891c9da6e7 100644
@@ -192,6 +192,7 @@ bool adf_misc_wq_queue_delayed_work(struct delayed_work *work,
 #if defined(CONFIG_PCI_IOV)
 int adf_sriov_configure(struct pci_dev *pdev, int numvfs);
 void adf_disable_sriov(struct adf_accel_dev *accel_dev);
+void adf_reenable_sriov(struct adf_accel_dev *accel_dev);
 void adf_enable_vf2pf_interrupts(struct adf_accel_dev *accel_dev, u32 vf_mask);
 void adf_disable_all_vf2pf_interrupts(struct adf_accel_dev *accel_dev);
 bool adf_recv_and_handle_pf2vf_msg(struct adf_accel_dev *accel_dev);
@@ -212,6 +213,10 @@ static inline void adf_disable_sriov(struct adf_accel_dev *accel_dev)
 {
 }
 
+static inline void adf_reenable_sriov(struct adf_accel_dev *accel_dev)
+{
+}
+
 static inline int adf_init_pf_wq(void)
 {
 	return 0;
drivers/crypto/intel/qat/qat_common/adf_sriov.c +34 -3
index cb2a9830f192..87a70c00c41e 100644
@@ -60,7 +60,6 @@ static int adf_enable_sriov(struct adf_accel_dev *accel_dev)
 		/* This ptr will be populated when VFs will be created */
 		vf_info->accel_dev = accel_dev;
 		vf_info->vf_nr = i;
-		vf_info->vf_compat_ver = 0;
 
 		mutex_init(&vf_info->pf2vf_lock);
 		ratelimit_state_init(&vf_info->vf2pf_ratelimit,
@@ -84,6 +83,32 @@ static int adf_enable_sriov(struct adf_accel_dev *accel_dev)
 	return pci_enable_sriov(pdev, totalvfs);
 }
 
+void adf_reenable_sriov(struct adf_accel_dev *accel_dev)
+{
+	struct pci_dev *pdev = accel_to_pci_dev(accel_dev);
+	char cfg[ADF_CFG_MAX_VAL_LEN_IN_BYTES] = {0};
+	unsigned long val = 0;
+
+	if (adf_cfg_get_param_value(accel_dev, ADF_GENERAL_SEC,
+				    ADF_SRIOV_ENABLED, cfg))
+		return;
+
+	if (!accel_dev->pf.vf_info)
+		return;
+
+	if (adf_cfg_add_key_value_param(accel_dev, ADF_KERNEL_SEC, ADF_NUM_CY,
+					&val, ADF_DEC))
+		return;
+
+	if (adf_cfg_add_key_value_param(accel_dev, ADF_KERNEL_SEC, ADF_NUM_DC,
+					&val, ADF_DEC))
+		return;
+
+	set_bit(ADF_STATUS_CONFIGURED, &accel_dev->status);
+	dev_dbg(&pdev->dev, "Re-enabling SRIOV\n");
+	adf_enable_sriov(accel_dev);
+}
+
 /**
  * adf_disable_sriov() - Disable SRIOV for the device
  * @accel_dev:  Pointer to accel device.
@@ -116,8 +141,10 @@ void adf_disable_sriov(struct adf_accel_dev *accel_dev)
 	for (i = 0, vf = accel_dev->pf.vf_info; i < totalvfs; i++, vf++)
 		mutex_destroy(&vf->pf2vf_lock);
 
-	kfree(accel_dev->pf.vf_info);
-	accel_dev->pf.vf_info = NULL;
+	if (!test_bit(ADF_STATUS_RESTARTING, &accel_dev->status)) {
+		kfree(accel_dev->pf.vf_info);
+		accel_dev->pf.vf_info = NULL;
+	}
 }
 EXPORT_SYMBOL_GPL(adf_disable_sriov);
 
@@ -195,6 +222,10 @@ int adf_sriov_configure(struct pci_dev *pdev, int numvfs)
 	if (ret)
 		return ret;
 
+	val = 1;
+	adf_cfg_add_key_value_param(accel_dev, ADF_GENERAL_SEC, ADF_SRIOV_ENABLED,
+				    &val, ADF_DEC);
+
 	return numvfs;
 }
 EXPORT_SYMBOL_GPL(adf_sriov_configure);
Signature · 1 line
2.34.1
#6 lore raw

PATCH v2 6/9 crypto: qat - add fatal error notification

In reply to Mun Chun Yep

Notify a fatal error condition and optionally reset the device in the following cases:
* if the device reports an uncorrectable fatal error through an interrupt
* if the heartbeat feature detects that the device is not responding

This patch is based on earlier work done by Shashank Gupta.

---
 drivers/crypto/intel/qat/qat_common/adf_heartbeat.c | 3 +++
 drivers/crypto/intel/qat/qat_common/adf_isr.c       | 7 ++++++-
 2 files changed, 9 insertions(+), 1 deletion(-)
drivers/crypto/intel/qat/qat_common/adf_heartbeat.c +3 -0
index f88b1bc6857e..fe8428d4ff39 100644
@@ -229,6 +229,9 @@ void adf_heartbeat_status(struct adf_accel_dev *accel_dev,
 			"Heartbeat ERROR: QAT is not responding.\n");
 		*hb_status = HB_DEV_UNRESPONSIVE;
 		hb->hb_failed_counter++;
+		if (adf_notify_fatal_error(accel_dev))
+			dev_err(&GET_DEV(accel_dev),
+				"Failed to notify fatal error\n");
 		return;
 	}
 
drivers/crypto/intel/qat/qat_common/adf_isr.c +6 -1
index a13d9885d60f..020d213f4c99 100644
@@ -139,8 +139,13 @@ static bool adf_handle_ras_int(struct adf_accel_dev *accel_dev)
 
 	if (ras_ops->handle_interrupt &&
 	    ras_ops->handle_interrupt(accel_dev, &reset_required)) {
-		if (reset_required)
+		if (reset_required) {
 			dev_err(&GET_DEV(accel_dev), "Fatal error, reset required\n");
+			if (adf_notify_fatal_error(accel_dev))
+				dev_err(&GET_DEV(accel_dev),
+					"Failed to notify fatal error\n");
+		}
+
 		return true;
 	}
 
Signature · 1 line
2.34.1
#7 lore raw

PATCH v2 7/9 crypto: qat - add auto reset on error

In reply to Mun Chun Yep

From: Damian Muszynski <[email protected]>

Expose the auto_reset sysfs attribute to configure the driver to reset the device when a fatal error is detected.

When auto reset is enabled, the driver resets the device when it detects either an heartbeat failure or a fatal error through an interrupt.

This patch is based on earlier work done by Shashank Gupta.

---
 Documentation/ABI/testing/sysfs-driver-qat    | 20 ++++++++++
 .../intel/qat/qat_common/adf_accel_devices.h  |  1 +
 drivers/crypto/intel/qat/qat_common/adf_aer.c | 11 +++++-
 .../intel/qat/qat_common/adf_common_drv.h     |  1 +
 .../crypto/intel/qat/qat_common/adf_sysfs.c   | 37 +++++++++++++++++++
 5 files changed, 69 insertions(+), 1 deletion(-)
Documentation/ABI/testing/sysfs-driver-qat +20 -0
index bbf329cf0d67..6778f1fea874 100644
@@ -141,3 +141,23 @@ Description:
 			64
 
 		This attribute is only available for qat_4xxx devices.
+
+What:		/sys/bus/pci/devices/<BDF>/qat/auto_reset
+Date:		March 2024
+KernelVersion:	6.8
+Contact:	[email protected]
+Description:	(RW) Reports the current state of the autoreset feature
+		for a QAT device
+
+		Write to the attribute to enable or disable device auto reset.
+
+		Device auto reset is disabled by default.
+
+		The values are::
+
+		* 1/Yy/on: auto reset enabled. If the device encounters an
+		  unrecoverable error, it will be reset automatically.
+		* 0/Nn/off: auto reset disabled. If the device encounters an
+		  unrecoverable error, it will not be reset.
+
+		This attribute is only available for qat_4xxx devices.
drivers/crypto/intel/qat/qat_common/adf_accel_devices.h +1 -0
index 4a3c36aaa7ca..0f26aa976c8c 100644
@@ -402,6 +402,7 @@ struct adf_accel_dev {
 	struct adf_error_counters ras_errors;
 	struct mutex state_lock; /* protect state of the device */
 	bool is_vf;
+	bool autoreset_on_error;
 	u32 accel_id;
 };
 #endif
drivers/crypto/intel/qat/qat_common/adf_aer.c +10 -1
index cd273b31db0e..b3d4b6b99c65 100644
@@ -204,6 +204,14 @@ const struct pci_error_handlers adf_err_handler = {
 };
 EXPORT_SYMBOL_GPL(adf_err_handler);
 
+int adf_dev_autoreset(struct adf_accel_dev *accel_dev)
+{
+	if (accel_dev->autoreset_on_error)
+		return adf_dev_aer_schedule_reset(accel_dev, ADF_DEV_RESET_ASYNC);
+
+	return 0;
+}
+
 static void adf_notify_fatal_error_worker(struct work_struct *work)
 {
 	struct adf_fatal_error_data *wq_data =
@@ -215,10 +223,11 @@ static void adf_notify_fatal_error_worker(struct work_struct *work)
 
 	if (!accel_dev->is_vf) {
 		/* Disable arbitration to stop processing of new requests */
-		if (hw_device->exit_arb)
+		if (accel_dev->autoreset_on_error && hw_device->exit_arb)
 			hw_device->exit_arb(accel_dev);
 		if (accel_dev->pf.vf_info)
 			adf_pf2vf_notify_fatal_error(accel_dev);
+		adf_dev_autoreset(accel_dev);
 	}
 
 	kfree(wq_data);
drivers/crypto/intel/qat/qat_common/adf_common_drv.h +1 -0
index 10891c9da6e7..57328249c89e 100644
@@ -87,6 +87,7 @@ int adf_ae_stop(struct adf_accel_dev *accel_dev);
 extern const struct pci_error_handlers adf_err_handler;
 void adf_reset_sbr(struct adf_accel_dev *accel_dev);
 void adf_reset_flr(struct adf_accel_dev *accel_dev);
+int adf_dev_autoreset(struct adf_accel_dev *accel_dev);
 void adf_dev_restore(struct adf_accel_dev *accel_dev);
 int adf_init_aer(void);
 void adf_exit_aer(void);
drivers/crypto/intel/qat/qat_common/adf_sysfs.c +37 -0
index d450dad32c9e..4e7f70d4049d 100644
@@ -204,6 +204,42 @@ static ssize_t pm_idle_enabled_store(struct device *dev, struct device_attribute
 }
 static DEVICE_ATTR_RW(pm_idle_enabled);
 
+static ssize_t auto_reset_show(struct device *dev, struct device_attribute *attr,
+			       char *buf)
+{
+	char *auto_reset;
+	struct adf_accel_dev *accel_dev;
+
+	accel_dev = adf_devmgr_pci_to_accel_dev(to_pci_dev(dev));
+	if (!accel_dev)
+		return -EINVAL;
+
+	auto_reset = accel_dev->autoreset_on_error ? "on" : "off";
+
+	return sysfs_emit(buf, "%s\n", auto_reset);
+}
+
+static ssize_t auto_reset_store(struct device *dev, struct device_attribute *attr,
+				const char *buf, size_t count)
+{
+	struct adf_accel_dev *accel_dev;
+	bool enabled = false;
+	int ret;
+
+	ret = kstrtobool(buf, &enabled);
+	if (ret)
+		return ret;
+
+	accel_dev = adf_devmgr_pci_to_accel_dev(to_pci_dev(dev));
+	if (!accel_dev)
+		return -EINVAL;
+
+	accel_dev->autoreset_on_error = enabled;
+
+	return count;
+}
+static DEVICE_ATTR_RW(auto_reset);
+
 static DEVICE_ATTR_RW(state);
 static DEVICE_ATTR_RW(cfg_services);
 
@@ -291,6 +327,7 @@ static struct attribute *qat_attrs[] = {
 	&dev_attr_pm_idle_enabled.attr,
 	&dev_attr_rp2srv.attr,
 	&dev_attr_num_rps.attr,
+	&dev_attr_auto_reset.attr,
 	NULL,
 };
 
Signature · 1 line
2.34.1
#8 lore raw

PATCH v2 8/9 crypto: qat - limit heartbeat notifications

In reply to Mun Chun Yep

From: Furong Zhou <[email protected]>

When the driver detects an heartbeat failure, it starts the recovery flow. Set a limit so that the number of events is limited in case the heartbeat status is read too frequently.

---
 .../crypto/intel/qat/qat_common/adf_heartbeat.c | 17 ++++++++++++++---
 .../crypto/intel/qat/qat_common/adf_heartbeat.h |  3 +++
 2 files changed, 17 insertions(+), 3 deletions(-)
drivers/crypto/intel/qat/qat_common/adf_heartbeat.c +14 -3
index fe8428d4ff39..b19aa1ef8eee 100644
@@ -205,6 +205,19 @@ static int adf_hb_get_status(struct adf_accel_dev *accel_dev)
 	return ret;
 }
 
+static void adf_heartbeat_reset(struct adf_accel_dev *accel_dev)
+{
+	u64 curr_time = adf_clock_get_current_time();
+	u64 time_since_reset = curr_time - accel_dev->heartbeat->last_hb_reset_time;
+
+	if (time_since_reset < ADF_CFG_HB_RESET_MS)
+		return;
+
+	accel_dev->heartbeat->last_hb_reset_time = curr_time;
+	if (adf_notify_fatal_error(accel_dev))
+		dev_err(&GET_DEV(accel_dev), "Failed to notify fatal error\n");
+}
+
 void adf_heartbeat_status(struct adf_accel_dev *accel_dev,
 			  enum adf_device_heartbeat_status *hb_status)
 {
@@ -229,9 +242,7 @@ void adf_heartbeat_status(struct adf_accel_dev *accel_dev,
 			"Heartbeat ERROR: QAT is not responding.\n");
 		*hb_status = HB_DEV_UNRESPONSIVE;
 		hb->hb_failed_counter++;
-		if (adf_notify_fatal_error(accel_dev))
-			dev_err(&GET_DEV(accel_dev),
-				"Failed to notify fatal error\n");
+		adf_heartbeat_reset(accel_dev);
 		return;
 	}
 
drivers/crypto/intel/qat/qat_common/adf_heartbeat.h +3 -0
index 24c3f4f24c86..16fdfb48b196 100644
@@ -13,6 +13,8 @@ struct dentry;
 #define ADF_CFG_HB_TIMER_DEFAULT_MS 500
 #define ADF_CFG_HB_COUNT_THRESHOLD 3
 
+#define ADF_CFG_HB_RESET_MS 5000
+
 enum adf_device_heartbeat_status {
 	HB_DEV_UNRESPONSIVE = 0,
 	HB_DEV_ALIVE,
@@ -30,6 +32,7 @@ struct adf_heartbeat {
 	unsigned int hb_failed_counter;
 	unsigned int hb_timer;
 	u64 last_hb_check_time;
+	u64 last_hb_reset_time;
 	bool ctrs_cnt_checked;
 	struct hb_dma_addr {
 		dma_addr_t phy_addr;
Signature · 1 line
2.34.1
#9 lore raw

PATCH v2 9/9 crypto: qat - improve aer error reset handling

In reply to Mun Chun Yep

Rework the AER reset and recovery flow to take into account root port integrated devices that gets reset between the error detected and the slot reset callbacks.

In adf_error_detected() the devices is gracefully shut down. The worker threads are disabled, the error conditions are notified to listeners and through PFVF comms and finally the device is reset as part of adf_dev_down().

In adf_slot_reset(), the device is brought up again. If SRIOV VFs were enabled before reset, these are re-enabled and VFs are notified of restarting through PFVF comms.

---
 drivers/crypto/intel/qat/qat_common/adf_aer.c | 26 ++++++++++++++++++-
 1 file changed, 25 insertions(+), 1 deletion(-)
drivers/crypto/intel/qat/qat_common/adf_aer.c +25 -1
index b3d4b6b99c65..3597e7605a14 100644
@@ -33,6 +33,19 @@ static pci_ers_result_t adf_error_detected(struct pci_dev *pdev,
 		return PCI_ERS_RESULT_DISCONNECT;
 	}
 
+	set_bit(ADF_STATUS_RESTARTING, &accel_dev->status);
+	if (accel_dev->hw_device->exit_arb) {
+		dev_dbg(&pdev->dev, "Disabling arbitration\n");
+		accel_dev->hw_device->exit_arb(accel_dev);
+	}
+	adf_error_notifier(accel_dev);
+	adf_pf2vf_notify_fatal_error(accel_dev);
+	adf_dev_restarting_notify(accel_dev);
+	adf_pf2vf_notify_restarting(accel_dev);
+	adf_pf2vf_wait_for_restarting_complete(accel_dev);
+	pci_clear_master(pdev);
+	adf_dev_down(accel_dev, false);
+
 	return PCI_ERS_RESULT_NEED_RESET;
 }
 
@@ -180,14 +193,25 @@ static int adf_dev_aer_schedule_reset(struct adf_accel_dev *accel_dev,
 static pci_ers_result_t adf_slot_reset(struct pci_dev *pdev)
 {
 	struct adf_accel_dev *accel_dev = adf_devmgr_pci_to_accel_dev(pdev);
+	int res = 0;
 
 	if (!accel_dev) {
 		pr_err("QAT: Can't find acceleration device\n");
 		return PCI_ERS_RESULT_DISCONNECT;
 	}
-	if (adf_dev_aer_schedule_reset(accel_dev, ADF_DEV_RESET_SYNC))
+
+	if (!pdev->is_busmaster)
+		pci_set_master(pdev);
+	pci_restore_state(pdev);
+	pci_save_state(pdev);
+	res = adf_dev_up(accel_dev, false);
+	if (res && res != -EALREADY)
 		return PCI_ERS_RESULT_DISCONNECT;
 
+	adf_reenable_sriov(accel_dev);
+	adf_pf2vf_notify_restarted(accel_dev);
+	adf_dev_restarted_notify(accel_dev);
+	clear_bit(ADF_STATUS_RESTARTING, &accel_dev->status);
 	return PCI_ERS_RESULT_RECOVERED;
 }
 
Signature · 1 line
2.34.1
maintainer
#10 lore raw
In reply to Mun Chun Yep
Quoted 46 lines · Mun Chun Yep

This set improves the error recovery flows in the QAT drivers and adds a mechanism to test it through an heartbeat simulator.

When a QAT device reports either a fatal error, or an AER fatal error, or fails an heartbeat check, the PF driver sends an error notification to the VFs through PFVF comms and if auto_reset is enabled then the device goes through reset flows for error recovery.
If SRIOV is enabled when an error is encountered, this is re-enabled after the reset cycle is done.

Changed in v2:
- Removed redundant default value in Kconfig
- Removed ccflags define, use the CONFIG option directly in the code
- Reworked the AER reset and recovery flow

Damian Muszynski (2):
crypto: qat - add heartbeat error simulator
crypto: qat - add auto reset on error

Furong Zhou (3):
crypto: qat - add fatal error notify method
crypto: qat - disable arbitration before reset
crypto: qat - limit heartbeat notifications

Mun Chun Yep (4):
crypto: qat - update PFVF protocol for recovery
crypto: qat - re-enable sriov after pf reset
crypto: qat - add fatal error notification
crypto: qat - improve aer error reset handling

 Documentation/ABI/testing/debugfs-driver-qat  |  26 ++++
 Documentation/ABI/testing/sysfs-driver-qat    |  20 +++
 drivers/crypto/intel/qat/Kconfig              |  14 +++
 drivers/crypto/intel/qat/qat_common/Makefile  |   2 +
 .../intel/qat/qat_common/adf_accel_devices.h  |   2 +
 drivers/crypto/intel/qat/qat_common/adf_aer.c | 116 +++++++++++++++++-
 .../intel/qat/qat_common/adf_cfg_strings.h    |   1 +
 .../intel/qat/qat_common/adf_common_drv.h     |  10 ++
 .../intel/qat/qat_common/adf_heartbeat.c      |  20 ++-
 .../intel/qat/qat_common/adf_heartbeat.h      |  21 ++++
 .../qat/qat_common/adf_heartbeat_dbgfs.c      |  52 ++++++++
 .../qat/qat_common/adf_heartbeat_inject.c     |  76 ++++++++++++
 .../intel/qat/qat_common/adf_hw_arbiter.c     |  25 ++++
 .../crypto/intel/qat/qat_common/adf_init.c    |  12 ++
 drivers/crypto/intel/qat/qat_common/adf_isr.c |   7 +-
 .../intel/qat/qat_common/adf_pfvf_msg.h       |   7 +-
Quoted 11 lines
 .../intel/qat/qat_common/adf_pfvf_pf_msg.c    |  64 +++++++++-
 .../intel/qat/qat_common/adf_pfvf_pf_msg.h    |  21 ++++
 .../intel/qat/qat_common/adf_pfvf_pf_proto.c  |   8 ++
 .../intel/qat/qat_common/adf_pfvf_vf_proto.c  |   6 +
 .../crypto/intel/qat/qat_common/adf_sriov.c   |  38 +++++-
 .../crypto/intel/qat/qat_common/adf_sysfs.c   |  37 ++++++
 22 files changed, 571 insertions(+), 14 deletions(-)
 create mode 100644 drivers/crypto/intel/qat/qat_common/adf_heartbeat_inject.c
Signature · 1 line
2.34.1

All applied. Thanks.

Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt