Skip to contentSkip to conversation

Introduce xe_wedge

PATCH Superseded Raag Jadav 1 participant 8 messages Oct 15

Subsystems (4)

v7 of 14 Superseded Latest is v14 →

lore
Share & tools
lore

Raag Jadav

author
#0 lore raw

v7 cover Introduce xe_wedge

In the history of xe technical debt, this series takes one more step and consolidates all wedging implementation into a dedicated xe_wedge component.

While at it, this fixes a few pre-existing issues related to wedge handling. Detailed description in commit message.

v1: https://patchwork.freedesktop.org/series/172727/

v2: Split fixes into separate patches (Rodrigo, Michal)
Also move struct xe_wedge and wedged_mode debt to xe_wedge (Lukasz, Michal) Naming and aesthetic adjustments (Lukasz, Michal)

v3: Respect the component namespace (Rodrigo)
Avoid chaining header includes (Rodrigo) Squash debugfs changes with xe_wedge patch (Lukasz)

v4: Introduce for_each_initialized_gt() (Michal)
Include xe_sriov.h (Rodrigo)

v5: Flush GT wedge worker to avoid races (Sashiko)
Drop IRQ safe wedging for its flawed design

v6: Move wedge worker initialization to xe_gt_init_early() (Rodrigo)
Don't flush GT wedge worker to avoid deadlock (Sashiko) Include IRQ safe wedging with redesign

v7: Guard GT wedge worker with runtime PM (Rodrigo)
Retain xe_pm_runtime_get_noresume() in IRQ context (Rodrigo)

Raag Jadav (7):
drm/xe/gt: Use GT ordered workqueue for wedging drm/xe: Drop xe_device_set_wedged_method() drm/xe: Move xe_device_wedged_fini() registration to
xe_device_probe_early()
drm/xe: Make xe_device_declare_wedged() IRQ safe drm/xe/pci_error: Flush GT wedge worker before bus reset drm/xe: Introduce xe_wedge
drm/xe/ras: Move xe_ras_process_errors() to xe_ras

 Documentation/gpu/xe/xe_device.rst          |   2 +-
 drivers/gpu/drm/xe/Makefile                 |   1 +
 drivers/gpu/drm/xe/tests/xe_kunit_helpers.c |   3 +-
 drivers/gpu/drm/xe/xe_amc.c                 |   4 +-
 drivers/gpu/drm/xe/xe_bo.c                  |   3 +-
 drivers/gpu/drm/xe/xe_debugfs.c             |  66 +----
 drivers/gpu/drm/xe/xe_device.c              | 190 +-------------
 drivers/gpu/drm/xe/xe_device.h              |  14 +-
 drivers/gpu/drm/xe/xe_device_types.h        |  28 +--
 drivers/gpu/drm/xe/xe_gsc.c                 |   4 +-
 drivers/gpu/drm/xe/xe_gt.c                  |  26 +-
 drivers/gpu/drm/xe/xe_gt.h                  |   2 +-
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c         |   3 +-
 drivers/gpu/drm/xe/xe_gt_types.h            |   9 +
 drivers/gpu/drm/xe/xe_guc.c                 |   4 +-
 drivers/gpu/drm/xe/xe_guc.h                 |   2 +-
 drivers/gpu/drm/xe/xe_guc_ct.c              |   7 +-
 drivers/gpu/drm/xe/xe_guc_pc.c              |  11 +-
 drivers/gpu/drm/xe/xe_guc_rc.c              |   5 +-
 drivers/gpu/drm/xe/xe_guc_submit.c          |  14 +-
 drivers/gpu/drm/xe/xe_guc_tlb_inval.c       |   3 +-
 drivers/gpu/drm/xe/xe_mert.c                |   5 +-
 drivers/gpu/drm/xe/xe_pci_error.c           |  13 +-
 drivers/gpu/drm/xe/xe_ras.c                 |  11 +-
 drivers/gpu/drm/xe/xe_sriov_pf.c            |   3 +-
 drivers/gpu/drm/xe/xe_survivability_mode.c  |   4 +-
 drivers/gpu/drm/xe/xe_uc.c                  |   6 +-
 drivers/gpu/drm/xe/xe_uc.h                  |   2 +-
 drivers/gpu/drm/xe/xe_wedge.c               | 262 ++++++++++++++++++++
 drivers/gpu/drm/xe/xe_wedge.h               |  24 ++
 drivers/gpu/drm/xe/xe_wedge_types.h         |  44 ++++
 31 files changed, 444 insertions(+), 331 deletions(-)
 create mode 100644 drivers/gpu/drm/xe/xe_wedge.c
 create mode 100644 drivers/gpu/drm/xe/xe_wedge.h
 create mode 100644 drivers/gpu/drm/xe/xe_wedge_types.h
Signature · 1 line
2.43.0
author
#1 lore raw

PATCH v7 1/7 drm/xe/gt: Use GT ordered workqueue for wedging

Currently, xe_gt_declare_wedged() stops GuC CT and initiates exec queue teardown synchronously. This can result in tdr timeout in cases where the device is declared wedged while jobs are still in-flight.

Introduce a worker for GT specific wedge handling and queue the teardown on GT ordered workqueue, so we don't disrupt the scheduler while jobs are still in-flight.

While at it, introduce for_each_initialized_gt() macro and use it for wedging GT. This helps avoid races in case the device is declared wedged in probe sequence before GT is initialized.

Fixes c9474b726b93 ("drm/xe: Wedge the entire device")

---
v2: Split fixes into separate patches (Rodrigo, Michal)
v4: Introduce for_each_initialized_gt() (Michal)
v5: Flush GT wedge worker to avoid races (Sashiko)
v6: Move wedge worker initialization to xe_gt_init_early() (Rodrigo)
Don't flush GT wedge worker to avoid deadlock (Sashiko)
v7: Guard GT wedge worker with runtime PM (Rodrigo)

---
 drivers/gpu/drm/xe/xe_device.c   |  2 +-
 drivers/gpu/drm/xe/xe_device.h   |  4 ++++
 drivers/gpu/drm/xe/xe_gt.c       | 17 +++++++++++++++--
 drivers/gpu/drm/xe/xe_gt_types.h |  9 +++++++++
 4 files changed, 29 insertions(+), 3 deletions(-)
drivers/gpu/drm/xe/xe_device.c +1 -1
index 6c214799b65b..37d27980dfc8 100644
@@ -1434,7 +1434,7 @@ void xe_device_declare_wedged(struct xe_device *xe)
 			    WEDGED_URL, XE_BUG_URL);
 	}
 
-	for_each_gt(gt, xe, id)
+	for_each_initialized_gt(gt, xe, id)
 		xe_gt_declare_wedged(gt);
 
 	if (xe_device_wedged(xe)) {
drivers/gpu/drm/xe/xe_device.h +4 -0
index 83d6dafab53c..80bda9b548d5 100644
@@ -139,6 +139,10 @@ static inline bool xe_device_uc_enabled(struct xe_device *xe)
 	for_each_gt((gt__), (tile__)->xe, (id__)) \
 		for_each_if((gt__)->tile == (tile__))
 
+#define for_each_initialized_gt(gt__, xe__, id__) \
+	for_each_gt((gt__), (xe__), (id__)) \
+		for_each_if(READ_ONCE((gt__)->initialized))
+
 static inline struct xe_force_wake *gt_to_fw(struct xe_gt *gt)
 {
 	return &gt->pm.fw;
drivers/gpu/drm/xe/xe_gt.c +15 -2
index bc9bf6f0b5a7..d85406ba6f06 100644
@@ -175,6 +175,7 @@ static void xe_gt_enable_comp_1wcoh(struct xe_gt *gt)
 }
 
 static void gt_reset_worker(struct work_struct *w);
+static void gt_wedge_worker(struct work_struct *w);
 
 static int emit_job_sync(struct xe_exec_queue *q, struct xe_bb *bb,
 			 long timeout_jiffies, bool force_reset)
@@ -546,6 +547,7 @@ int xe_gt_init_early(struct xe_gt *gt)
 	xe_gt_mcr_init_early(gt);
 	xe_pat_init(gt);
 
+	INIT_WORK(&gt->wedge.worker, gt_wedge_worker);
 	return 0;
 }
 
@@ -724,6 +726,7 @@ static void xe_gt_fini(void *arg)
 	struct xe_gt *gt = arg;
 	int i;
 
+	disable_work_sync(&gt->wedge.worker);
 	gt_reset_worker_disable(gt);
 
 	for (i = 0; i < XE_ENGINE_CLASS_MAX; ++i)
@@ -784,6 +787,8 @@ int xe_gt_init(struct xe_gt *gt)
 			return err;
 	}
 
+	/* Must be last */
+	WRITE_ONCE(gt->initialized, true);
 	return 0;
 }
 
@@ -1204,6 +1209,15 @@ struct xe_hw_engine *xe_gt_any_hw_engine(struct xe_gt *gt)
 	return NULL;
 }
 
+static void gt_wedge_worker(struct work_struct *w)
+{
+	struct xe_gt *gt = container_of(w, typeof(*gt), wedge.worker);
+
+	guard(xe_pm_runtime)(gt_to_xe(gt));
+	xe_uc_declare_wedged(&gt->uc);
+	xe_tlb_inval_reset(&gt->tlb_inval);
+}
+
 /**
  * xe_gt_declare_wedged() - Declare GT wedged
  * @gt: the GT object
@@ -1215,6 +1229,5 @@ void xe_gt_declare_wedged(struct xe_gt *gt)
 {
 	xe_gt_assert(gt, gt_to_xe(gt)->wedged.mode);
 
-	xe_uc_declare_wedged(&gt->uc);
-	xe_tlb_inval_reset(&gt->tlb_inval);
+	queue_work(gt->ordered_wq, &gt->wedge.worker);
 }
drivers/gpu/drm/xe/xe_gt_types.h +9 -0
index 628911346455..c00a7c171f80 100644
@@ -233,6 +233,12 @@ struct xe_gt {
 		struct work_struct worker;
 	} reset;
 
+	/** @wedge: state for GT wedge */
+	struct {
+		/** @wedge.worker: worker for GT wedge to be done async */
+		struct work_struct worker;
+	} wedge;
+
 	/** @tlb_inval: TLB invalidation state */
 	struct xe_tlb_inval tlb_inval;
 
@@ -419,6 +425,9 @@ struct xe_gt {
 
 	/** @eu_stall: EU stall counters subsystem per gt info */
 	struct xe_eu_stall_gt *eu_stall;
+
+	/** @initialized: GT is initialized */
+	bool initialized;
 };
 
 #endif
Signature · 1 line
2.43.0
author
#2 lore raw

PATCH v7 2/7 drm/xe: Drop xe_device_set_wedged_method()

In reply to Raag Jadav

Commit 60439ac3f235 ("drm/xe: Add a helper function to set recovery method") introduced xe_device_set_wedged_method() helper to allow setting preferred recovery method before wedging the device, but also introduced a flaw where the previously set method can be overwritten by concurrent racing wedge call. This can result in unintended recovery method to be sent to userspace and possibly failed recovery.

Drop xe_device_set_wedged_method() entirely and pass the recovery method as part of xe_device_declare_wedged() argument, which places the preferred recovery method on stack and avoids this issue.

Fixes 60439ac3f235 ("drm/xe: Add a helper function to set recovery method")

---
v2: Split fixes into separate patches (Rodrigo, Michal)

---
 drivers/gpu/drm/xe/xe_amc.c                |  3 ++-
 drivers/gpu/drm/xe/xe_device.c             | 27 ++++------------------
 drivers/gpu/drm/xe/xe_device.h             |  3 +--
 drivers/gpu/drm/xe/xe_gsc.c                |  3 ++-
 drivers/gpu/drm/xe/xe_gt.c                 |  3 ++-
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c        |  2 +-
 drivers/gpu/drm/xe/xe_guc_submit.c         |  5 ++--
 drivers/gpu/drm/xe/xe_mert.c               |  6 +++--
 drivers/gpu/drm/xe/xe_pci_error.c          |  2 +-
 drivers/gpu/drm/xe/xe_ras.c                |  3 +--
 drivers/gpu/drm/xe/xe_survivability_mode.c |  3 +--
 11 files changed, 23 insertions(+), 37 deletions(-)
drivers/gpu/drm/xe/xe_amc.c +2 -1
index 8ecadee6eea3..b7390a4ee3ce 100644
@@ -160,7 +160,8 @@ static void xe_amc_work(struct work_struct *work)
 	case AMC_ALERT_OOB_RESET:
 	case AMC_ALERT_CATERR:
 		dev_warn(amc->i2c->drm_dev, "AMC Alert: %s\n", amc_alert[alert_reason]);
-		xe_device_declare_wedged(i2c_client_to_xe_device(client));
+		xe_device_declare_wedged(i2c_client_to_xe_device(client),
+					 DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
 		break;
 	default:
 		dev_warn(amc->i2c->drm_dev, "unknown AMC alert: %d\n", alert_reason);
drivers/gpu/drm/xe/xe_device.c +5 -22
index 37d27980dfc8..9925a82556b3 100644
@@ -1380,29 +1380,16 @@ u64 xe_device_uncanonicalize_addr(struct xe_device *xe, u64 address)
  *   firmware and restore device to normal operation.
  */
 
-/**
- * xe_device_set_wedged_method - Set wedged recovery method
- * @xe: xe device instance
- * @method: recovery method to set
- *
- * Set wedged recovery method to be sent in drm wedged uevent.
- */
-void xe_device_set_wedged_method(struct xe_device *xe, unsigned long method)
-{
-	xe->wedged.method = method;
-}
-
 #define WEDGED_URL	"https://docs.kernel.org/gpu/drm-uapi.html#device-wedging"
 #define XE_BUG_URL	"https://gitlab.freedesktop.org/drm/xe/kernel/issues/new"
 
 /**
  * xe_device_declare_wedged - Declare device wedged
  * @xe: xe device instance
+ * @method: method(s) to be used for recovery
  *
  * This is a final state that can only be cleared with the recovery method
- * specified in the drm wedged uevent. The method can be set using
- * xe_device_set_wedged_method before declaring the device as wedged. If no method
- * is set, reprobe (unbind/re-bind) will be sent by default.
+ * specified in the drm wedged uevent.
  *
  * In this state every IOCTL will be blocked so the GT cannot be used.
  * In general it will be called upon any critical error such as gt reset
@@ -1413,7 +1400,7 @@ void xe_device_set_wedged_method(struct xe_device *xe, unsigned long method)
  * snapshot capture. In this mode, GT reset won't be attempted so the state of
  * the issue is preserved for further debugging.
  */
-void xe_device_declare_wedged(struct xe_device *xe)
+void xe_device_declare_wedged(struct xe_device *xe, unsigned long method)
 {
 	struct xe_gt *gt;
 	u8 id;
@@ -1444,14 +1431,10 @@ void xe_device_declare_wedged(struct xe_device *xe)
 		 * it available to the user for debugging.
 		 */
 		if (xe->wedged.mode == XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET)
-			xe_device_set_wedged_method(xe, DRM_WEDGE_RECOVERY_NONE);
-		/* If no wedge recovery method is set, use default */
-		else if (!xe->wedged.method)
-			xe_device_set_wedged_method(xe, DRM_WEDGE_RECOVERY_REBIND |
-						    DRM_WEDGE_RECOVERY_BUS_RESET);
+			method = DRM_WEDGE_RECOVERY_NONE;
 
 		/* Notify userspace of wedged device */
-		drm_dev_wedged_event(&xe->drm, xe->wedged.method, NULL);
+		drm_dev_wedged_event(&xe->drm, method, NULL);
 	}
 }
 
drivers/gpu/drm/xe/xe_device.h +1 -2
index 80bda9b548d5..9f35370c0851 100644
@@ -264,8 +264,7 @@ static inline bool xe_debug_page_size_mode_is_mixed(struct xe_device *xe)
 }
 #endif
 
-void xe_device_set_wedged_method(struct xe_device *xe, unsigned long method);
-void xe_device_declare_wedged(struct xe_device *xe);
+void xe_device_declare_wedged(struct xe_device *xe, unsigned long method);
 int xe_device_validate_wedged_mode(struct xe_device *xe, unsigned int mode);
 const char *xe_wedged_mode_to_string(enum xe_wedged_mode mode);
 
drivers/gpu/drm/xe/xe_gsc.c +2 -1
index 524ac56bdcc7..2bbe0f20ff8d 100644
@@ -340,7 +340,8 @@ static int gsc_er_complete(struct xe_gt *gt)
 		 * declare the device as wedged.
 		 */
 		xe_gt_err(gt, "GSC ER timed out!\n");
-		xe_device_declare_wedged(gt_to_xe(gt));
+		xe_device_declare_wedged(gt_to_xe(gt),
+					 DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
 		return -EIO;
 	}
 
drivers/gpu/drm/xe/xe_gt.c +2 -1
index d85406ba6f06..528c53cedcc9 100644
@@ -1006,7 +1006,8 @@ static void gt_reset_worker(struct work_struct *w)
 
 err_fail:
 	xe_log_err_fatal(gt, GT, err, "reset failed\n");
-	xe_device_declare_wedged(gt_to_xe(gt));
+	xe_device_declare_wedged(gt_to_xe(gt),
+				 DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
 err_pm_put:
 	xe_pm_runtime_put(gt_to_xe(gt));
 }
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -1
index 37899fcf5b22..28ec83bb8100 100644
@@ -1462,7 +1462,7 @@ static void vf_post_migration_recovery(struct xe_gt *gt)
 fail:
 	vf_post_migration_abort(gt);
 	xe_gt_sriov_err(gt, "migration recovery failed (%pe)\n", ERR_PTR(err));
-	xe_device_declare_wedged(xe);
+	xe_device_declare_wedged(xe, DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
 	return;
 
 queue:
drivers/gpu/drm/xe/xe_guc_submit.c +3 -2
index e0a3ce6015e0..319956fc4315 100644
@@ -1453,7 +1453,7 @@ static bool guc_submit_hint_wedged(struct xe_guc *guc)
 	if (xe_device_wedged(xe))
 		return true;
 
-	xe_device_declare_wedged(xe);
+	xe_device_declare_wedged(xe, DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
 
 	return true;
 }
@@ -1786,7 +1786,8 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)
 	}
 
 	if (wedge_device)
-		xe_device_declare_wedged(gt_to_xe(q->gt));
+		xe_device_declare_wedged(gt_to_xe(q->gt),
+					 DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
 
 	/*
 	 * We want the job added back to the pending list so it gets freed; this
drivers/gpu/drm/xe/xe_mert.c +4 -2
index 32700c19a1df..934be933d57f 100644
@@ -74,7 +74,8 @@ static void mert_handle_cat_error(struct xe_device *xe)
 		break;
 	case CATERR_UNMAPPED_GGTT:
 		xe_sriov_err(xe, "MERT: CAT_ERR: Access to an unmapped GGTT!\n");
-		xe_device_declare_wedged(xe);
+		xe_device_declare_wedged(xe,
+					 DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
 		break;
 	case CATERR_LMTT_FAULT:
 		xe_sriov_dbg_ratelimited(xe, "MERT: CAT_ERR: VF%u LMTT fault!\n", vfid);
@@ -82,7 +83,8 @@ static void mert_handle_cat_error(struct xe_device *xe)
 		break;
 	default:
 		xe_sriov_err(xe, "MERT: Unexpected CAT_ERR code=%#x!\n", code);
-		xe_device_declare_wedged(xe);
+		xe_device_declare_wedged(xe,
+					 DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
 		break;
 	}
 }
drivers/gpu/drm/xe/xe_pci_error.c +1 -1
index 79ce0c671549..e1b9dc5f0a6d 100644
@@ -86,7 +86,7 @@ static pci_ers_result_t xe_pci_error_mmio_enabled(struct pci_dev *pdev)
 
 	/* User wants to debug the error, prevent reset */
 	if (xe->wedged.mode == XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET) {
-		xe_device_declare_wedged(xe);
+		xe_device_declare_wedged(xe, DRM_WEDGE_RECOVERY_NONE);
 		return PCI_ERS_RESULT_DISCONNECT;
 	}
 
drivers/gpu/drm/xe/xe_ras.c +1 -2
index 7a85735c57d5..fe16a4106060 100644
@@ -321,8 +321,7 @@ static u8 handle_core_compute_errors(struct xe_ras_error_array *arr)
 
 static void punit_error_handler(struct xe_device *xe)
 {
-	xe_device_set_wedged_method(xe, DRM_WEDGE_RECOVERY_COLD_RESET);
-	xe_device_declare_wedged(xe);
+	xe_device_declare_wedged(xe, DRM_WEDGE_RECOVERY_COLD_RESET);
 }
 
 static u8 handle_soc_internal_errors(struct xe_device *xe, struct xe_ras_error_array *arr)
drivers/gpu/drm/xe/xe_survivability_mode.c +1 -2
index 45d44ebe288b..ca031576fecb 100644
@@ -446,8 +446,7 @@ void xe_survivability_mode_runtime_enable(struct xe_device *xe)
 	survivability->type = XE_SURVIVABILITY_TYPE_RUNTIME;
 	xe_log_err(xe, SURVIVABILITY, 0, "Runtime Mode enabled!\n");
 
-	xe_device_set_wedged_method(xe, DRM_WEDGE_RECOVERY_VENDOR);
-	xe_device_declare_wedged(xe);
+	xe_device_declare_wedged(xe, DRM_WEDGE_RECOVERY_VENDOR);
 
 	xe_log_err(xe, SURVIVABILITY, 0, "Firmware flash required!\n");
 	xe_info(xe, "Please refer to the userspace documentation for more details how to flash the firmware on %s!\n",
Signature · 1 line
2.43.0
author
#3 lore raw

PATCH v7 3/7 drm/xe: Move xe_device_wedged_fini() registration to xe_device_probe_early()

In reply to Raag Jadav

The device can be declared wedged while processing boot-time hardware errors, which can result in PM reference leak if driver probe fails before xe_device_wedged_fini() is registered.

Early register the required cleanup action and fix this.

Fixes a7df563b45b0 ("drm/xe/xe_hw_error: Handle CSC Firmware reported Hardware errors")

---
v2: Split fixes into separate patches (Rodrigo, Michal)

---
 drivers/gpu/drm/xe/xe_device.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
drivers/gpu/drm/xe/xe_device.c +6 -4
index 9925a82556b3..8076f29759ed 100644
@@ -697,6 +697,8 @@ static int xe_device_vram_alloc(struct xe_device *xe)
 	return 0;
 }
 
+static void xe_device_wedged_fini(struct drm_device *drm, void *arg);
+
 /**
  * xe_device_probe_early: Device early probe
  * @xe: xe device instance
@@ -765,6 +767,10 @@ int xe_device_probe_early(struct xe_device *xe)
 	if (err)
 		return err;
 
+	err = drmm_add_action_or_reset(&xe->drm, xe_device_wedged_fini, xe);
+	if (err)
+		return err;
+
 	return 0;
 }
 ALLOW_ERROR_INJECTION(xe_device_probe_early, ERRNO); /* See xe_pci_probe() */
@@ -1097,10 +1103,6 @@ int xe_device_probe(struct xe_device *xe)
 
 	detect_preproduction_hw(xe);
 
-	err = drmm_add_action_or_reset(&xe->drm, xe_device_wedged_fini, xe);
-	if (err)
-		goto err_unregister_display;
-
 	/*
 	 * Process and log any errors detected by hardware. Possible results can
 	 * include declaring the device as wedged, which must be done only after
Signature · 1 line
2.43.0
author
#4 lore raw

PATCH v7 4/7 drm/xe: Make xe_device_declare_wedged() IRQ safe

In reply to Raag Jadav

Currently, xe_device_declare_wedged() implementation wedges the GTs and sends wedged uevent to userspace. This is problematic for usecases which require declaring the device as wedged in IRQ context, as it allocates memory for event buffer and acquires mutexes deep into it's call path.

Fix this by deferring wedge handling into its dedicated worker, so that xe_device_declare_wedged() can be called in IRQ context.

First user of this requirement is introduced in commit def675cf3f10 ("drm/xe/mert: Improve handling of MERT CAT errors"), which declares the device as wedged on catastrophic errors received in form of an IRQ.

Fixes def675cf3f10 ("drm/xe/mert: Improve handling of MERT CAT errors")

---
v2: Split fixes into separate patches (Rodrigo, Michal)
v5: Drop IRQ safe wedging for its flawed design
v6: Include IRQ safe wedging with redesign
v7: Retain xe_pm_runtime_get_noresume() in IRQ context (Rodrigo)

---
 drivers/gpu/drm/xe/xe_device.c       | 31 +++++++++++++++++++++-------
 drivers/gpu/drm/xe/xe_device_types.h |  2 ++
 2 files changed, 25 insertions(+), 8 deletions(-)
drivers/gpu/drm/xe/xe_device.c +23 -8
index 8076f29759ed..63e3b22ddf48 100644
@@ -697,8 +697,16 @@ static int xe_device_vram_alloc(struct xe_device *xe)
 	return 0;
 }
 
+static void wedged_work(struct work_struct *work);
 static void xe_device_wedged_fini(struct drm_device *drm, void *arg);
 
+static int xe_device_wedged_init(struct xe_device *xe)
+{
+	INIT_WORK(&xe->wedged.work, wedged_work);
+
+	return drmm_add_action_or_reset(&xe->drm, xe_device_wedged_fini, xe);
+}
+
 /**
  * xe_device_probe_early: Device early probe
  * @xe: xe device instance
@@ -767,7 +775,7 @@ int xe_device_probe_early(struct xe_device *xe)
 	if (err)
 		return err;
 
-	err = drmm_add_action_or_reset(&xe->drm, xe_device_wedged_fini, xe);
+	err = xe_device_wedged_init(xe);
 	if (err)
 		return err;
 
@@ -1382,6 +1390,14 @@ u64 xe_device_uncanonicalize_addr(struct xe_device *xe, u64 address)
  *   firmware and restore device to normal operation.
  */
 
+static void wedged_work(struct work_struct *work)
+{
+	struct xe_device *xe = container_of(work, struct xe_device, wedged.work);
+
+	/* Notify userspace of wedged device */
+	drm_dev_wedged_event(&xe->drm, xe->wedged.method, NULL);
+}
+
 #define WEDGED_URL	"https://docs.kernel.org/gpu/drm-uapi.html#device-wedging"
 #define XE_BUG_URL	"https://gitlab.freedesktop.org/drm/xe/kernel/issues/new"
 
@@ -1421,22 +1437,21 @@ void xe_device_declare_wedged(struct xe_device *xe, unsigned long method)
 			    "For recovery procedure, refer to %s\n"
 			    "Please file a _new_ bug report at %s\n",
 			    WEDGED_URL, XE_BUG_URL);
-	}
 
-	for_each_initialized_gt(gt, xe, id)
-		xe_gt_declare_wedged(gt);
+		for_each_initialized_gt(gt, xe, id)
+			xe_gt_declare_wedged(gt);
 
-	if (xe_device_wedged(xe)) {
 		/*
 		 * XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET is intended for debugging
 		 * hangs, so wedge the device with 'none' recovery method and have
 		 * it available to the user for debugging.
 		 */
 		if (xe->wedged.mode == XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET)
-			method = DRM_WEDGE_RECOVERY_NONE;
+			xe->wedged.method = DRM_WEDGE_RECOVERY_NONE;
+		else
+			xe->wedged.method = method;
 
-		/* Notify userspace of wedged device */
-		drm_dev_wedged_event(&xe->drm, method, NULL);
+		schedule_work(&xe->wedged.work);
 	}
 }
 
drivers/gpu/drm/xe/xe_device_types.h +2 -0
index 4e0e1bc040c1..34f7a2d35db6 100644
@@ -547,6 +547,8 @@ struct xe_device {
 		unsigned long method;
 		/** @wedged.inconsistent_reset: Inconsistent reset policy state between GTs */
 		bool inconsistent_reset;
+		/** @wedged.work: Worker to defer non-atomic calls */
+		struct work_struct work;
 	} wedged;
 
 	/** @devres_group: devres group */
Signature · 1 line
2.43.0
author
#5 lore raw

PATCH v7 5/7 drm/xe/pci_error: Flush GT wedge worker before bus reset

In reply to Raag Jadav

With GT wedging now done async, make sure the wedge worker is flushed to avoid side-effects during PCI bus reset.

Fixes 13b9555ffb03 ("drm/xe/xe_pci_error: Implement PCI error recovery callbacks")
---
 drivers/gpu/drm/xe/xe_pci_error.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_pci_error.c +3 -1
index e1b9dc5f0a6d..a5c8fd67738c 100644
@@ -30,8 +30,10 @@ static void prepare_device_for_reset(struct pci_dev *pdev)
 
 	xe_device_set_in_reset(xe);
 
-	for_each_gt(gt, xe, id)
+	for_each_gt(gt, xe, id) {
 		xe_gt_declare_wedged(gt);
+		flush_work(&gt->wedge.worker);
+	}
 
 	pci_disable_device(pdev);
 }
Signature · 1 line
2.43.0
author
#6 lore raw

PATCH v7 6/7 drm/xe: Introduce xe_wedge

In reply to Raag Jadav

Consolidate wedging implementation into a dedicated xe_wedge component for better maintainability. While at it, rename all wedge helpers to match with component name.

No functional impact.

---
v2: Also move struct xe_wedge and wedged_mode debt to xe_wedge (Lukasz, Michal)
Naming and aesthetic adjustments (Lukasz, Michal)
v3: Respect the component namespace (Rodrigo)
Avoid chaining header includes (Rodrigo)
Squash debugfs changes with xe_wedge patch (Lukasz)
v4: Include xe_sriov.h (Rodrigo)

---
 Documentation/gpu/xe/xe_device.rst          |   2 +-
 drivers/gpu/drm/xe/Makefile                 |   1 +
 drivers/gpu/drm/xe/tests/xe_kunit_helpers.c |   3 +-
 drivers/gpu/drm/xe/xe_amc.c                 |   5 +-
 drivers/gpu/drm/xe/xe_bo.c                  |   3 +-
 drivers/gpu/drm/xe/xe_debugfs.c             |  66 +----
 drivers/gpu/drm/xe/xe_device.c              | 179 +------------
 drivers/gpu/drm/xe/xe_device.h              |   9 -
 drivers/gpu/drm/xe/xe_device_types.h        |  30 +--
 drivers/gpu/drm/xe/xe_gsc.c                 |   5 +-
 drivers/gpu/drm/xe/xe_gt.c                  |  12 +-
 drivers/gpu/drm/xe/xe_gt.h                  |   2 +-
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c         |   3 +-
 drivers/gpu/drm/xe/xe_guc.c                 |   4 +-
 drivers/gpu/drm/xe/xe_guc.h                 |   2 +-
 drivers/gpu/drm/xe/xe_guc_ct.c              |   7 +-
 drivers/gpu/drm/xe/xe_guc_pc.c              |  11 +-
 drivers/gpu/drm/xe/xe_guc_rc.c              |   5 +-
 drivers/gpu/drm/xe/xe_guc_submit.c          |  15 +-
 drivers/gpu/drm/xe/xe_guc_tlb_inval.c       |   3 +-
 drivers/gpu/drm/xe/xe_mert.c                |   7 +-
 drivers/gpu/drm/xe/xe_pci_error.c           |   9 +-
 drivers/gpu/drm/xe/xe_ras.c                 |   3 +-
 drivers/gpu/drm/xe/xe_sriov_pf.c            |   3 +-
 drivers/gpu/drm/xe/xe_survivability_mode.c  |   3 +-
 drivers/gpu/drm/xe/xe_uc.c                  |   6 +-
 drivers/gpu/drm/xe/xe_uc.h                  |   2 +-
 drivers/gpu/drm/xe/xe_wedge.c               | 262 ++++++++++++++++++++
 drivers/gpu/drm/xe/xe_wedge.h               |  24 ++
 drivers/gpu/drm/xe/xe_wedge_types.h         |  44 ++++
 30 files changed, 405 insertions(+), 325 deletions(-)
 create mode 100644 drivers/gpu/drm/xe/xe_wedge.c
 create mode 100644 drivers/gpu/drm/xe/xe_wedge.h
 create mode 100644 drivers/gpu/drm/xe/xe_wedge_types.h
Documentation/gpu/xe/xe_device.rst +1 -1
index d3a022362ade..8baed81580c9 100644
@@ -6,7 +6,7 @@
 Xe Device Wedging
 ==================
 
-.. kernel-doc:: drivers/gpu/drm/xe/xe_device.c
+.. kernel-doc:: drivers/gpu/drm/xe/xe_wedge.c
    :doc: Xe Device Wedging
 
 ====================
drivers/gpu/drm/xe/Makefile +1 -0
index 011f7ef54086..bc203940f114 100644
@@ -154,6 +154,7 @@ xe-y += xe_bb.o \
 	xe_vsec.o \
 	xe_wa.o \
 	xe_wait_user_fence.o \
+	xe_wedge.o \
 	xe_wopcm.o
 
 xe-$(CONFIG_I2C)	+= xe_i2c.o \
drivers/gpu/drm/xe/tests/xe_kunit_helpers.c +2 -1
index 27740b40c8ae..57ea88cce92d 100644
@@ -15,6 +15,7 @@
 #include "xe_device.h"
 #include "xe_device_types.h"
 #include "xe_pm.h"
+#include "xe_wedge.h"
 
 /**
  * xe_kunit_helper_alloc_xe_device - Allocate a &xe_device for a KUnit test.
@@ -123,7 +124,7 @@ int xe_kunit_helper_xe_device_live_test_init(struct kunit *test)
 	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, xe);
 	kunit_info(test, "running on %s device\n", xe->info.platform_name);
 
-	KUNIT_ASSERT_FALSE(test, xe_device_wedged(xe));
+	KUNIT_ASSERT_FALSE(test, xe_wedged(xe));
 	xe_pm_runtime_get(xe);
 	KUNIT_ASSERT_EQ(test, 0, kunit_add_action_or_reset(test, put_xe_pm_runtime, xe));
 
drivers/gpu/drm/xe/xe_amc.c +3 -2
index b7390a4ee3ce..3b91c0d489f5 100644
@@ -18,6 +18,7 @@
 #include "xe_device.h"
 #include "xe_i2c.h"
 #include "xe_mmio.h"
+#include "xe_wedge.h"
 
 /**
  * DOC: Add-In Management Controller (AMC)
@@ -160,8 +161,8 @@ static void xe_amc_work(struct work_struct *work)
 	case AMC_ALERT_OOB_RESET:
 	case AMC_ALERT_CATERR:
 		dev_warn(amc->i2c->drm_dev, "AMC Alert: %s\n", amc_alert[alert_reason]);
-		xe_device_declare_wedged(i2c_client_to_xe_device(client),
-					 DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
+		xe_wedge_declare(i2c_client_to_xe_device(client),
+				 DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
 		break;
 	default:
 		dev_warn(amc->i2c->drm_dev, "unknown AMC alert: %d\n", alert_reason);
drivers/gpu/drm/xe/xe_bo.c +2 -1
index 12e523c61c1f..10a3f156a9d3 100644
@@ -41,6 +41,7 @@
 #include "xe_ttm_stolen_mgr.h"
 #include "xe_vm.h"
 #include "xe_vram_types.h"
+#include "xe_wedge.h"
 
 const char *const xe_mem_type_to_name[TTM_NUM_MEM_TYPES]  = {
 	[XE_PL_SYSTEM] = "system",
@@ -2203,7 +2204,7 @@ static vm_fault_t xe_bo_cpu_fault(struct vm_fault *vmf)
 	int err = 0;
 	int idx;
 
-	if (xe_device_wedged(xe) || !drm_dev_enter(&xe->drm, &idx))
+	if (xe_wedged(xe) || !drm_dev_enter(&xe->drm, &idx))
 		return ttm_bo_vm_dummy_page(vmf, vmf->vma->vm_page_prot);
 
 	ret = xe_bo_cpu_fault_fastpath(vmf, xe, bo, needs_rpm);
drivers/gpu/drm/xe/xe_debugfs.c +3 -63

98 lines not on this page · raw message

drivers/gpu/drm/xe/xe_device.c +5 -174

231 lines not on this page · raw message

drivers/gpu/drm/xe/xe_device.h +0 -9
index 9f35370c0851..6df60edcb8f8 100644
@@ -211,11 +211,6 @@ bool xe_device_is_l2_flush_optimized(struct xe_device *xe);
 void xe_device_td_flush(struct xe_device *xe);
 void xe_device_l2_flush(struct xe_device *xe, bool force);
 
-static inline bool xe_device_wedged(struct xe_device *xe)
-{
-	return atomic_read(&xe->wedged.flag);
-}
-
 #ifdef CONFIG_DRM_XE_DEBUG_PAGE_SIZE
 static inline bool xe_debug_page_size_supported(struct xe_device *xe)
 {
@@ -264,10 +259,6 @@ static inline bool xe_debug_page_size_mode_is_mixed(struct xe_device *xe)
 }
 #endif
 
-void xe_device_declare_wedged(struct xe_device *xe, unsigned long method);
-int xe_device_validate_wedged_mode(struct xe_device *xe, unsigned int mode);
-const char *xe_wedged_mode_to_string(enum xe_wedged_mode mode);
-
 struct xe_file *xe_file_get(struct xe_file *xef);
 void xe_file_put(struct xe_file *xef);
 
drivers/gpu/drm/xe/xe_device_types.h +2 -28
index 34f7a2d35db6..25b38258b629 100644
@@ -30,6 +30,7 @@
 #include "xe_sysctrl_types.h"
 #include "xe_tile_types.h"
 #include "xe_validation.h"
+#include "xe_wedge_types.h"
 
 #if IS_ENABLED(CONFIG_DRM_XE_DEBUG)
 #define TEST_VM_OPS_ERROR
@@ -47,22 +48,6 @@ struct xe_pxp;
 struct xe_ttm_stolen_mgr;
 struct xe_vram_region;
 
-/**
- * enum xe_wedged_mode - possible wedged modes
- * @XE_WEDGED_MODE_NEVER: Device will never be declared wedged.
- * @XE_WEDGED_MODE_UPON_CRITICAL_ERROR: Device will be declared wedged only
- *	when critical error occurs like GT reset failure or firmware failure.
- *	This is the default mode.
- * @XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET: Device will be declared wedged on
- *	any hang. In this mode, engine resets are disabled to avoid automatic
- *	recovery attempts. This mode is primarily intended for debugging hangs.
- */
-enum xe_wedged_mode {
-	XE_WEDGED_MODE_NEVER = 0,
-	XE_WEDGED_MODE_UPON_CRITICAL_ERROR = 1,
-	XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET = 2,
-};
-
 #ifdef CONFIG_DRM_XE_DEBUG_PAGE_SIZE
 /**
  * enum xe_page_size_alloc_ctrl_mode - User BO page-size allocation control modes
@@ -538,18 +523,7 @@ struct xe_device {
 	atomic_t in_reset;
 
 	/** @wedged: Struct to control Wedged States and mode */
-	struct {
-		/** @wedged.flag: Xe device faced a critical error and is now blocked. */
-		atomic_t flag;
-		/** @wedged.mode: Mode controlled by kernel parameter and debugfs */
-		enum xe_wedged_mode mode;
-		/** @wedged.method: Recovery method to be sent in the drm device wedged uevent */
-		unsigned long method;
-		/** @wedged.inconsistent_reset: Inconsistent reset policy state between GTs */
-		bool inconsistent_reset;
-		/** @wedged.work: Worker to defer non-atomic calls */
-		struct work_struct work;
-	} wedged;
+	struct xe_wedge wedged;
 
 	/** @devres_group: devres group */
 	void *devres_group;
drivers/gpu/drm/xe/xe_gsc.c +3 -2
index 2bbe0f20ff8d..db669a953886 100644
@@ -31,6 +31,7 @@
 #include "xe_sched_job.h"
 #include "xe_uc_fw.h"
 #include "xe_wa.h"
+#include "xe_wedge.h"
 #include "instructions/xe_gsc_commands.h"
 #include "regs/xe_gsc_regs.h"
 #include "regs/xe_gt_regs.h"
@@ -340,8 +341,8 @@ static int gsc_er_complete(struct xe_gt *gt)
 		 * declare the device as wedged.
 		 */
 		xe_gt_err(gt, "GSC ER timed out!\n");
-		xe_device_declare_wedged(gt_to_xe(gt),
-					 DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
+		xe_wedge_declare(gt_to_xe(gt),
+				 DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
 		return -EIO;
 	}
 
drivers/gpu/drm/xe/xe_gt.c +6 -6
index 528c53cedcc9..98e24c2196ad 100644
@@ -68,6 +68,7 @@
 #include "xe_uc_fw.h"
 #include "xe_vm.h"
 #include "xe_wa.h"
+#include "xe_wedge.h"
 #include "xe_wopcm.h"
 
 #define GRDOM_RESET_TIMEOUT_MS	5
@@ -947,7 +948,7 @@ static void gt_reset_worker(struct work_struct *w)
 	unsigned int fw_ref;
 	int err;
 
-	if (xe_device_wedged(gt_to_xe(gt)))
+	if (xe_wedged(gt_to_xe(gt)))
 		goto err_pm_put;
 
 	if (xe_device_is_in_reset(gt_to_xe(gt)))
@@ -1006,8 +1007,7 @@ static void gt_reset_worker(struct work_struct *w)
 
 err_fail:
 	xe_log_err_fatal(gt, GT, err, "reset failed\n");
-	xe_device_declare_wedged(gt_to_xe(gt),
-				 DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
+	xe_wedge_declare(gt_to_xe(gt), DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
 err_pm_put:
 	xe_pm_runtime_put(gt_to_xe(gt));
 }
@@ -1215,18 +1215,18 @@ static void gt_wedge_worker(struct work_struct *w)
 	struct xe_gt *gt = container_of(w, typeof(*gt), wedge.worker);
 
 	guard(xe_pm_runtime)(gt_to_xe(gt));
-	xe_uc_declare_wedged(&gt->uc);
+	xe_uc_wedge_declare(&gt->uc);
 	xe_tlb_inval_reset(&gt->tlb_inval);
 }
 
 /**
- * xe_gt_declare_wedged() - Declare GT wedged
+ * xe_gt_wedge_declare() - Declare GT wedged
  * @gt: the GT object
  *
  * Wedge the GT which stops all submission, saves desired debug state, and
  * cleans up anything which could timeout.
  */
-void xe_gt_declare_wedged(struct xe_gt *gt)
+void xe_gt_wedge_declare(struct xe_gt *gt)
 {
 	xe_gt_assert(gt, gt_to_xe(gt)->wedged.mode);
 
drivers/gpu/drm/xe/xe_gt.h +1 -1
index 92d934d50244..7b6952fe804c 100644
@@ -40,7 +40,7 @@ struct xe_gt *xe_gt_alloc(struct xe_tile *tile);
 int xe_gt_init_early(struct xe_gt *gt);
 int xe_gt_init(struct xe_gt *gt);
 void xe_gt_mmio_init(struct xe_gt *gt);
-void xe_gt_declare_wedged(struct xe_gt *gt);
+void xe_gt_wedge_declare(struct xe_gt *gt);
 int xe_gt_record_default_lrcs(struct xe_gt *gt);
 
 /**
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -1
index 28ec83bb8100..2a5f2064ad3e 100644
@@ -37,6 +37,7 @@
 #include "xe_tile_sriov_vf.h"
 #include "xe_tlb_inval.h"
 #include "xe_uc_fw.h"
+#include "xe_wedge.h"
 #include "xe_wopcm.h"
 
 #define make_u64_from_u32(hi, lo) ((u64)((u64)(u32)(hi) << 32 | (u32)(lo)))
@@ -1462,7 +1463,7 @@ static void vf_post_migration_recovery(struct xe_gt *gt)
 fail:
 	vf_post_migration_abort(gt);
 	xe_gt_sriov_err(gt, "migration recovery failed (%pe)\n", ERR_PTR(err));
-	xe_device_declare_wedged(xe, DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
+	xe_wedge_declare(xe, DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
 	return;
 
 queue:
drivers/gpu/drm/xe/xe_guc.c +2 -2
index 3ab4cb9e4964..7d239a0829fb 100644
@@ -1829,13 +1829,13 @@ int xe_guc_print_info(struct xe_guc *guc, struct drm_printer *p)
 }
 
 /**
- * xe_guc_declare_wedged() - Declare GuC wedged
+ * xe_guc_wedge_declare() - Declare GuC wedged
  * @guc: the GuC object
  *
  * Wedge the GuC which stops all submission, saves desired debug state, and
  * cleans up anything which could timeout.
  */
-void xe_guc_declare_wedged(struct xe_guc *guc)
+void xe_guc_wedge_declare(struct xe_guc *guc)
 {
 	xe_gt_assert(guc_to_gt(guc), guc_to_xe(guc)->wedged.mode);
 
drivers/gpu/drm/xe/xe_guc.h +1 -1
index c4eca40d69cf..75c4b9acb065 100644
@@ -62,7 +62,7 @@ void xe_guc_reset_wait(struct xe_guc *guc);
 void xe_guc_stop_prepare(struct xe_guc *guc);
 void xe_guc_stop(struct xe_guc *guc);
 int xe_guc_start(struct xe_guc *guc);
-void xe_guc_declare_wedged(struct xe_guc *guc);
+void xe_guc_wedge_declare(struct xe_guc *guc);
 bool xe_guc_using_main_gamctrl_queues(struct xe_guc *guc);
 bool xe_guc_has_paging_engine(struct xe_guc *guc);
 
drivers/gpu/drm/xe/xe_guc_ct.c +4 -3
index e865d50e8b77..6113beda144b 100644
@@ -36,6 +36,7 @@
 #include "xe_sleep.h"
 #include "xe_sriov_vf.h"
 #include "xe_trace_guc.h"
+#include "xe_wedge.h"
 
 static void receive_g2h(struct xe_guc_ct *ct);
 static void g2h_worker_func(struct work_struct *w);
@@ -1064,7 +1065,7 @@ static int __guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action,
 	xe_gt_assert(gt, g2h_len || !num_g2h);
 	lockdep_assert_held(&ct->lock);
 
-	if (xe_device_wedged(ct_to_xe(ct))) {
+	if (xe_wedged(ct_to_xe(ct))) {
 		ret = -ENOTRECOVERABLE;
 		goto out;
 	}
@@ -1489,7 +1490,7 @@ static int guc_ct_send_recv(struct xe_guc_ct *ct, const u32 *action, u32 len,
 	if (g2h_fence.fail) {
 		if (g2h_fence.cancel) {
 			xe_gt_dbg(gt, "H2G request %#x canceled!\n", action[0]);
-			ret = xe_device_wedged(ct_to_xe(ct)) ? -ENOTRECOVERABLE : -ECANCELED;
+			ret = xe_wedged(ct_to_xe(ct)) ? -ENOTRECOVERABLE : -ECANCELED;
 			goto unlock;
 		}
 		xe_gt_err(gt, "H2G request %#x failed: error %#x hint %#x\n",
@@ -1828,7 +1829,7 @@ static int g2h_read(struct xe_guc_ct *ct, u32 *msg, bool fast_path)
 	xe_gt_assert(gt, xe_guc_ct_initialized(ct));
 	lockdep_assert_held(&ct->fast_lock);
 
-	if (xe_device_wedged(xe))
+	if (xe_wedged(xe))
 		return -ENOTRECOVERABLE;
 
 	if (ct->state == XE_GUC_CT_STATE_DISABLED)
drivers/gpu/drm/xe/xe_guc_pc.c +6 -5
index e0105222a2c7..9558d7196a51 100644
@@ -36,6 +36,7 @@
 #include "xe_pm.h"
 #include "xe_sriov.h"
 #include "xe_wa.h"
+#include "xe_wedge.h"
 
 #define MCHBAR_MIRROR_BASE_SNB	0x140000
 
@@ -188,7 +189,7 @@ static int pc_action_reset(struct xe_guc_pc *pc)
 	int ret;
 
 	ret = xe_guc_ct_send(ct, action, ARRAY_SIZE(action), 0, 0);
-	if (ret && !(xe_device_wedged(pc_to_xe(pc)) && ret == -ECANCELED))
+	if (ret && !(xe_wedged(pc_to_xe(pc)) && ret == -ECANCELED))
 		xe_gt_err(pc_to_gt(pc), "GuC PC reset failed: %pe\n",
 			  ERR_PTR(ret));
 
@@ -212,7 +213,7 @@ static int pc_action_query_task_state(struct xe_guc_pc *pc)
 
 	/* Blocking here to ensure the results are ready before reading them */
 	ret = xe_guc_ct_send_block(ct, action, ARRAY_SIZE(action));
-	if (ret && !(xe_device_wedged(pc_to_xe(pc)) && ret == -ECANCELED))
+	if (ret && !(xe_wedged(pc_to_xe(pc)) && ret == -ECANCELED))
 		xe_gt_err(pc_to_gt(pc), "GuC PC query task state failed: %pe\n",
 			  ERR_PTR(ret));
 
@@ -235,7 +236,7 @@ static int pc_action_set_param(struct xe_guc_pc *pc, u8 id, u32 value)
 		return -EAGAIN;
 
 	ret = xe_guc_ct_send(ct, action, ARRAY_SIZE(action), 0, 0);
-	if (ret && !(xe_device_wedged(pc_to_xe(pc)) && ret == -ECANCELED))
+	if (ret && !(xe_wedged(pc_to_xe(pc)) && ret == -ECANCELED))
 		xe_gt_err(pc_to_gt(pc), "GuC PC set param[%u]=%u failed: %pe\n",
 			  id, value, ERR_PTR(ret));
 
@@ -257,7 +258,7 @@ static int pc_action_unset_param(struct xe_guc_pc *pc, u8 id)
 		return -EAGAIN;
 
 	ret = xe_guc_ct_send(ct, action, ARRAY_SIZE(action), 0, 0);
-	if (ret && !(xe_device_wedged(pc_to_xe(pc)) && ret == -ECANCELED))
+	if (ret && !(xe_wedged(pc_to_xe(pc)) && ret == -ECANCELED))
 		xe_gt_err(pc_to_gt(pc), "GuC PC unset param failed: %pe",
 			  ERR_PTR(ret));
 
@@ -1357,7 +1358,7 @@ static void xe_guc_pc_fini_hw(void *arg)
 	struct xe_guc_pc *pc = arg;
 	struct xe_device *xe = pc_to_xe(pc);
 
-	if (xe_device_wedged(xe))
+	if (xe_wedged(xe))
 		return;
 
 	xe_guc_pc_stop(pc);
drivers/gpu/drm/xe/xe_guc_rc.c +3 -2
index 99fa127b261f..fa3062ab134a 100644
@@ -16,6 +16,7 @@
 #include "xe_guc_pc.h"
 #include "xe_guc_rc.h"
 #include "xe_pm.h"
+#include "xe_wedge.h"
 
 /**
  * DOC: GuC RC (Render C-states)
@@ -40,7 +41,7 @@ static int guc_action_setup_gucrc(struct xe_guc *guc, u32 control)
 	int ret;
 
 	ret = xe_guc_ct_send(&guc->ct, action, ARRAY_SIZE(action), 0, 0);
-	if (ret && !(xe_device_wedged(guc_to_xe(guc)) && ret == -ECANCELED))
+	if (ret && !(xe_wedged(guc_to_xe(guc)) && ret == -ECANCELED))
 		xe_gt_err(guc_to_gt(guc),
 			  "GuC RC setup %s(%u) failed (%pe)\n",
 			   control == GUCRC_HOST_CONTROL ? "HOST_CONTROL" :
@@ -73,7 +74,7 @@ static void xe_guc_rc_fini_hw(void *arg)
 	struct xe_device *xe = guc_to_xe(guc);
 	struct xe_gt *gt = guc_to_gt(guc);
 
-	if (xe_device_wedged(xe))
+	if (xe_wedged(xe))
 		return;
 
 	CLASS(xe_force_wake, fw_ref)(gt_to_fw(gt), XE_FW_GT);
drivers/gpu/drm/xe/xe_guc_submit.c +8 -7
index 319956fc4315..2fcc4e3bfa24 100644
@@ -50,6 +50,7 @@
 #include "xe_trace.h"
 #include "xe_uc_fw.h"
 #include "xe_vm.h"
+#include "xe_wedge.h"
 
 #define XE_GUC_EXEC_QUEUE_CGP_CONTEXT_ERROR_LEN		6
 
@@ -1450,10 +1451,10 @@ static bool guc_submit_hint_wedged(struct xe_guc *guc)
 	if (xe->wedged.mode != XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET)
 		return false;
 
-	if (xe_device_wedged(xe))
+	if (xe_wedged(xe))
 		return true;
 
-	xe_device_declare_wedged(xe, DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
+	xe_wedge_declare(xe, DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
 
 	return true;
 }
@@ -1666,7 +1667,7 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)
 	if (!exec_queue_killed(q))
 		wedged = guc_submit_hint_wedged(exec_queue_to_guc(q));
 	else
-		wedged = xe_device_wedged(xe);
+		wedged = xe_wedged(xe);
 
 	/*
 	 * Only tag as GPU hang if this is the original timeout, not a
@@ -1786,8 +1787,8 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)
 	}
 
 	if (wedge_device)
-		xe_device_declare_wedged(gt_to_xe(q->gt),
-					 DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
+		xe_wedge_declare(gt_to_xe(q->gt),
+				 DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
 
 	/*
 	 * We want the job added back to the pending list so it gets freed; this
@@ -2552,7 +2553,7 @@ static int guc_exec_queue_wait_suspend_done(struct xe_exec_queue *q, bool blocki
 						       WAIT_COND, HZ * 5);
 	}
 
-	if (!blocking && vf_recovery(guc) && !xe_device_wedged(xe))
+	if (!blocking && vf_recovery(guc) && !xe_wedged(xe))
 		return -EAGAIN;
 
 	if (!ret)
@@ -2812,7 +2813,7 @@ int xe_guc_submit_reset_prepare(struct xe_guc *guc)
 
 void xe_guc_submit_reset_wait(struct xe_guc *guc)
 {
-	wait_event(guc->ct.wq, xe_device_wedged(guc_to_xe(guc)) ||
+	wait_event(guc->ct.wq, xe_wedged(guc_to_xe(guc)) ||
 		   !xe_guc_read_stopped(guc));
 }
 
drivers/gpu/drm/xe/xe_guc_tlb_inval.c +2 -1
index 111fef781b2a..dd38d95c6014 100644
@@ -19,6 +19,7 @@
 #include "xe_sa.h"
 #include "xe_tlb_inval.h"
 #include "xe_vm.h"
+#include "xe_wedge.h"
 
 #include "regs/xe_guc_regs.h"
 
@@ -77,7 +78,7 @@ static int send_tlb_inval_ggtt(struct xe_tlb_inval *tlb_inval, u32 seqno)
 		};
 
 		return send_tlb_inval(guc, action, ARRAY_SIZE(action));
-	} else if (xe_device_uc_enabled(xe) && !xe_device_wedged(xe)) {
+	} else if (xe_device_uc_enabled(xe) && !xe_wedged(xe)) {
 		struct xe_mmio *mmio = &gt->mmio;
 
 		if (IS_SRIOV_VF(xe))
drivers/gpu/drm/xe/xe_mert.c +3 -4
index 934be933d57f..a2eae6088c1a 100644
@@ -11,6 +11,7 @@
 #include "xe_mmio.h"
 #include "xe_sriov_printk.h"
 #include "xe_tile.h"
+#include "xe_wedge.h"
 
 /**
  * xe_mert_init_early() - Initialize MERT data
@@ -74,8 +75,7 @@ static void mert_handle_cat_error(struct xe_device *xe)
 		break;
 	case CATERR_UNMAPPED_GGTT:
 		xe_sriov_err(xe, "MERT: CAT_ERR: Access to an unmapped GGTT!\n");
-		xe_device_declare_wedged(xe,
-					 DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
+		xe_wedge_declare(xe, DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
 		break;
 	case CATERR_LMTT_FAULT:
 		xe_sriov_dbg_ratelimited(xe, "MERT: CAT_ERR: VF%u LMTT fault!\n", vfid);
@@ -83,8 +83,7 @@ static void mert_handle_cat_error(struct xe_device *xe)
 		break;
 	default:
 		xe_sriov_err(xe, "MERT: Unexpected CAT_ERR code=%#x!\n", code);
-		xe_device_declare_wedged(xe,
-					 DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
+		xe_wedge_declare(xe, DRM_WEDGE_RECOVERY_REBIND | DRM_WEDGE_RECOVERY_BUS_RESET);
 		break;
 	}
 }
drivers/gpu/drm/xe/xe_pci_error.c +5 -4
index a5c8fd67738c..9bf0da3a3d15 100644
@@ -13,6 +13,7 @@
 #include "xe_printk.h"
 #include "xe_ras.h"
 #include "xe_survivability_mode.h"
+#include "xe_wedge.h"
 
 static void prepare_device_for_reset(struct pci_dev *pdev)
 {
@@ -22,7 +23,7 @@ static void prepare_device_for_reset(struct pci_dev *pdev)
 
 	/*
 	 * Wedge the device to prevent userspace access but do not send the uevent.
-	 * xe_device_wedged_fini() releases runtime pm if wedged flag is set, so acquire a runtime
+	 * xe_wedge_fini() releases runtime pm if wedged flag is set, so acquire a runtime
 	 * pm reference to avoid underflow.
 	 */
 	if (!atomic_xchg(&xe->wedged.flag, 1))
@@ -31,7 +32,7 @@ static void prepare_device_for_reset(struct pci_dev *pdev)
 	xe_device_set_in_reset(xe);
 
 	for_each_gt(gt, xe, id) {
-		xe_gt_declare_wedged(gt);
+		xe_gt_wedge_declare(gt);
 		flush_work(&gt->wedge.worker);
 	}
 
@@ -63,7 +64,7 @@ static pci_ers_result_t xe_pci_error_detected(struct pci_dev *pdev, pci_channel_
 		return PCI_ERS_RESULT_DISCONNECT;
 
 	/* If the device is already wedged or in survivability mode, do not attempt recovery */
-	if (xe_survivability_mode_is_boot_enabled(xe) || xe_device_wedged(xe))
+	if (xe_survivability_mode_is_boot_enabled(xe) || xe_wedged(xe))
 		return PCI_ERS_RESULT_DISCONNECT;
 
 	switch (state) {
@@ -88,7 +89,7 @@ static pci_ers_result_t xe_pci_error_mmio_enabled(struct pci_dev *pdev)
 
 	/* User wants to debug the error, prevent reset */
 	if (xe->wedged.mode == XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET) {
-		xe_device_declare_wedged(xe, DRM_WEDGE_RECOVERY_NONE);
+		xe_wedge_declare(xe, DRM_WEDGE_RECOVERY_NONE);
 		return PCI_ERS_RESULT_DISCONNECT;
 	}
 
drivers/gpu/drm/xe/xe_ras.c +2 -1
index fe16a4106060..922e5e793b13 100644
@@ -16,6 +16,7 @@
 #include "xe_sysctrl_event_types.h"
 #include "xe_sysctrl_mailbox.h"
 #include "xe_sysctrl_mailbox_types.h"
+#include "xe_wedge.h"
 
 #define CORE_COMPUTE_UNCORR_TYPE	GENMASK(26, 25)
 /*
@@ -321,7 +322,7 @@ static u8 handle_core_compute_errors(struct xe_ras_error_array *arr)
 
 static void punit_error_handler(struct xe_device *xe)
 {
-	xe_device_declare_wedged(xe, DRM_WEDGE_RECOVERY_COLD_RESET);
+	xe_wedge_declare(xe, DRM_WEDGE_RECOVERY_COLD_RESET);
 }
 
 static u8 handle_soc_internal_errors(struct xe_device *xe, struct xe_ras_error_array *arr)
drivers/gpu/drm/xe/xe_sriov_pf.c +2 -1
index 33bd754d138f..ce0db26203ec 100644
@@ -19,6 +19,7 @@
 #include "xe_sriov_pf_service.h"
 #include "xe_sriov_pf_sysfs.h"
 #include "xe_sriov_printk.h"
+#include "xe_wedge.h"
 
 static unsigned int wanted_max_vfs(struct xe_device *xe)
 {
@@ -157,7 +158,7 @@ int xe_sriov_pf_wait_ready(struct xe_device *xe)
 	unsigned int id;
 	int err;
 
-	if (xe_device_wedged(xe))
+	if (xe_wedged(xe))
 		return -ECANCELED;
 
 	for_each_gt(gt, xe, id) {
drivers/gpu/drm/xe/xe_survivability_mode.c +2 -1
index ca031576fecb..d73cdad0b3a9 100644
@@ -20,6 +20,7 @@
 #include "xe_pcode_api.h"
 #include "xe_printk.h"
 #include "xe_vsec.h"
+#include "xe_wedge.h"
 
 /**
  * DOC: Survivability Mode
@@ -446,7 +447,7 @@ void xe_survivability_mode_runtime_enable(struct xe_device *xe)
 	survivability->type = XE_SURVIVABILITY_TYPE_RUNTIME;
 	xe_log_err(xe, SURVIVABILITY, 0, "Runtime Mode enabled!\n");
 
-	xe_device_declare_wedged(xe, DRM_WEDGE_RECOVERY_VENDOR);
+	xe_wedge_declare(xe, DRM_WEDGE_RECOVERY_VENDOR);
 
 	xe_log_err(xe, SURVIVABILITY, 0, "Firmware flash required!\n");
 	xe_info(xe, "Please refer to the userspace documentation for more details how to flash the firmware on %s!\n",
drivers/gpu/drm/xe/xe_uc.c +3 -3
index 65f59f06177f..686c28856014 100644
@@ -326,15 +326,15 @@ void xe_uc_runtime_resume(struct xe_uc *uc)
 }
 
 /**
- * xe_uc_declare_wedged() - Declare UC wedged
+ * xe_uc_wedge_declare() - Declare UC wedged
  * @uc: the UC object
  *
  * Wedge the UC which stops all submission, saves desired debug state, and
  * cleans up anything which could timeout.
  */
-void xe_uc_declare_wedged(struct xe_uc *uc)
+void xe_uc_wedge_declare(struct xe_uc *uc)
 {
 	xe_gt_assert(uc_to_gt(uc), uc_to_xe(uc)->wedged.mode);
 
-	xe_guc_declare_wedged(&uc->guc);
+	xe_guc_wedge_declare(&uc->guc);
 }
drivers/gpu/drm/xe/xe_uc.h +1 -1
index 255a54a8f876..23bc0ce1c42f 100644
@@ -21,6 +21,6 @@ int xe_uc_start(struct xe_uc *uc);
 void xe_uc_suspend_prepare(struct xe_uc *uc);
 int xe_uc_suspend(struct xe_uc *uc);
 int xe_uc_sanitize_reset(struct xe_uc *uc);
-void xe_uc_declare_wedged(struct xe_uc *uc);
+void xe_uc_wedge_declare(struct xe_uc *uc);
 
 #endif
drivers/gpu/drm/xe/xe_wedge.c new file +262 -0

262 lines not on this page · raw message

drivers/gpu/drm/xe/xe_wedge.h new file +24 -0
new file mode 100644
index 000000000000..3b5499dfc30d
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2026 Intel Corporation
+ */
+
+#ifndef _XE_WEDGE_H_
+#define _XE_WEDGE_H_
+
+#include <linux/atomic.h>
+#include <linux/types.h>
+
+#include "xe_device_types.h"
+#include "xe_wedge_types.h"
+
+static inline bool xe_wedged(struct xe_device *xe)
+{
+	return atomic_read(&xe->wedged.flag);
+}
+
+void xe_wedge_declare(struct xe_device *xe, unsigned long method);
+int xe_wedge_init(struct xe_device *xe);
+int xe_wedge_set_mode(struct xe_device *xe, enum xe_wedged_mode mode);
+
+#endif
drivers/gpu/drm/xe/xe_wedge_types.h new file +44 -0

44 lines not on this page · raw message

Signature · 1 line
2.43.0
author
#7 lore raw

PATCH v7 7/7 drm/xe/ras: Move xe_ras_process_errors() to xe_ras

In reply to Raag Jadav

With xe_wedge_fini() now moved to xe_wedge and registered as part of xe_device_probe_early(), it is safe to move xe_ras_process_errors() back to xe_ras where it belongs. This essentially reverts commit 20bc4883c7c0 ("drm/xe/ras: Fix boot-time ras error processing"), which was added to prevent PM reference leak in case of early wedging in probe sequence.

---
 drivers/gpu/drm/xe/xe_device.c | 7 -------
 drivers/gpu/drm/xe/xe_ras.c    | 7 +++++++
 2 files changed, 7 insertions(+), 7 deletions(-)
drivers/gpu/drm/xe/xe_device.c +0 -7
index 9c91705dfe50..27a1f657ece0 100644
@@ -1089,13 +1089,6 @@ int xe_device_probe(struct xe_device *xe)
 
 	detect_preproduction_hw(xe);
 
-	/*
-	 * Process and log any errors detected by hardware. Possible results can
-	 * include declaring the device as wedged, which must be done only after
-	 * xe_wedge_fini() is registered.
-	 */
-	xe_ras_process_errors(xe);
-
 	err = devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
 	if (err)
 		goto err_unregister_display;
drivers/gpu/drm/xe/xe_ras.c +7 -0
index 922e5e793b13..8e399ffec985 100644
@@ -932,6 +932,13 @@ void xe_ras_init(struct xe_device *xe)
 	if (IS_ENABLED(CONFIG_PCIEAER))
 		ras_usp_aer_init(xe);
 
+	/*
+	 * During probe, process and log any errors detected by firmware while the driver was not
+	 * loaded. Critical errors such as Punit and CSC are reported through Pcode init failure,
+	 * causing the driver to enter survivability mode.
+	 */
+	xe_ras_process_errors(xe);
+
 	ret = devm_device_add_group(xe->drm.dev, &gpu_health_group);
 	if (ret)
 		xe_err(xe, "Failed to create GPU health sysfs, err=%d\n", ret);
Signature · 1 line
2.43.0