Skip to contentSkip to conversation

drm/xe: Map Wa_22016122933 buffers WC on the CPU side

PATCH In review Navon John Lukose 3 participants 3 messages Oct 4 – Oct 18

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drm/xe: Map Wa_22016122933 buffers WC on the CPU side

xe implements Wa_22016122933 by mapping the media GT's GuC-shared buffers uncached on the CPU side. i915 maps the same buffers WC on the CPU and UC on the GPU.

CPU writes through the uncached mapping are slow, and xe clears and refills the ADS on every GuC load, so probe and every resume pay for it. On ARL-H, xe probe takes 3.1 s with UC and 0.35 s with WC, the same as without the workaround.

Keep XE_BO_FLAG_NEEDS_UC, which also selects XE_CACHE_NONE for the GGTT PTEs, and add XE_BO_FLAG_FORCE_WC, which now makes the CPU mapping write-combined when both flags are set.

Fixes d5b0bf3f37f1 ("drm/xe: Implement Wa_22016122933")

---
Tested on ARL-H 7d51 with this backported to 7.2.8. Probe time is dmesg "GT0: Using GuC firmware" to "Initialized xe": 3.10 and 3.15 s with UC, 0.35-0.37 s with WC (3 boots), 0.35 s on stock Arch 7.2.8, which lacks the workaround.

20 min of VAAPI decode with C-states 2/3 disabled (the reproducer from
https://gitlab.freedesktop.org/drm/xe/kernel/-/work_items/8678): no TLB invalidation timeouts and no late acks, same as with UC.

 drivers/gpu/drm/xe/xe_bo.c  | 7 ++++++-
 drivers/gpu/drm/xe/xe_guc.c | 6 ++++--
 2 files changed, 10 insertions(+), 3 deletions(-)
drivers/gpu/drm/xe/xe_bo.c +6 -1
index 6921b696733..763172b252f 100644
@@ -533,7 +533,12 @@ static struct ttm_tt *xe_ttm_tt_create(struct ttm_buffer_object *ttm_bo,
 		 * which cpu_caching is never initialized.
 		 */
 		xe_assert(xe, bo->cpu_caching == 0);
-		caching = ttm_uncached;
+		/*
+		 * With FORCE_WC also set, map WC on the CPU side. The GGTT PTEs
+		 * stay XE_CACHE_NONE because they key off NEEDS_UC alone.
+		 */
+		caching = bo->flags & XE_BO_FLAG_FORCE_WC ? ttm_write_combined :
+							    ttm_uncached;
 	}
 
 	if (ttm_bo->type != ttm_bo_type_sg)
drivers/gpu/drm/xe/xe_guc.c +4 -2
index 3ab4cb9e496..a39a42bfd0b 100644
@@ -1476,13 +1476,15 @@ int xe_guc_suspend(struct xe_guc *guc)
  * Wa_22016122933: on the standalone media GT, memory shared between the
  * CPU and the GuC must not be mapped cached on the CPU side, otherwise
  * the CPU can read stale data written by the GuC (e.g. G2H CTB writes)
- * for multiple seconds.
+ * for multiple seconds. Map it WC on the CPU side and UC on the GPU side,
+ * as i915 does.
  *
  * Return: additional XE_BO_FLAG_* to use when allocating GuC-shared memory
  */
 u32 xe_guc_bo_wa_flags(struct xe_gt *gt)
 {
-	return XE_GT_WA(gt, 22016122933) ? XE_BO_FLAG_NEEDS_UC : 0;
+	return XE_GT_WA(gt, 22016122933) ?
+	       XE_BO_FLAG_NEEDS_UC | XE_BO_FLAG_FORCE_WC : 0;
 }
 
 void xe_guc_notify(struct xe_guc *guc)
Signature · 1 line
2.56.0
#1 lore raw

✗ LGCI.VerificationFailed: failure for drm/xe: Map Wa_22016122933 buffers WC on the CPU side

== Series Details ==

== Series Details ==

Series: drm/xe: Map Wa_22016122933 buffers WC on the CPU side
URL : https://patchwork.freedesktop.org/series/175580/
State : failure

== Summary ==

Series author address '[email protected]' is not on the allowlist, which prevents CI from being automatically triggered. If you want CI to run for this series, ask Patchwork project owners to click 'retest' on the series in Patchwork. Exception occurred during validation, bailing out!
Build URL: http://intel-gfx-ci-public.igk.intel.com:8080/job/xe_pw_trigger/1303039/ (on master)

Reviewed-by Tested-by on v1 1/1
#2 lore raw
In reply to Navon John Lukose
Quoted 3 lines · Navon John Lukose

xe implements Wa_22016122933 by mapping the media GT's GuC-shared buffers uncached on the CPU side. i915 maps the same buffers WC on the CPU and UC on the GPU.

Thanks for catching this, and sorry for the regression.

I reviewed it against 7.2.11. The GGTT PTEs still key their cache mode off XE_BO_FLAG_NEEDS_UC alone (both encode paths in xe_ggtt.c), so the GPU side stays XE_CACHE_NONE as the workaround needs. Every user of xe_guc_bo_wa_flags() picks up FORCE_WC: ADS, CTB, GuC log, SLPC and engine activity. The only other internal NEEDS_UC user that touches the GuC, the UM queue BO in xe_guc_ads.c, predates the workaround and is left alone. CPU reads through WC still bypass the cache, so the stale G2H reads the workaround is about cannot come back.

Tested on a second ARL-H 7d51 (GuC 70.53.0), on 7.2.11 carrying 63360b32, fc668a6e and d5b0bf3f, with only xe.ko rebuilt with this patch.

Probe, dmesg "GT0: Using GuC firmware" to "Initialized xe":
d5b0bf3f only: 3.56, 3.49, 3.52 s
with this patch: 0.37 s

Resume from s2idle, wake ("ACPI: EC: interrupt unblocked") to "PM: suspend exit":
d5b0bf3f only: 2.43, 2.92, 2.97, 2.58, 2.93 s

                   (xe done after 2.28-2.83 s)

with this patch: 0.39 s (37 s asleep), 0.40 s (386 s asleep)
(xe done after 0.11 s both times)

20 minutes of VA-API decode, two decoders in parallel on 4K 10-bit HEVC clips (528 decodes): no TLB invalidation timeouts, no late acks, no job timeouts, no GT resets. I did not disable C-states for that run.

Over the same session I got one media GT "not started" job timeout in a browser, recovered by a GT reset. It is the same thing I already see without this patch (four devcoredumps since October 7 on the UC version, all with the LRC tail in memory still equal to the head), so it looks unrelated to this patch. I have reported it separately in
https://gitlab.freedesktop.org/drm/xe/kernel/-/work_items/8678.

Thanks,
Tales