drm/xe: Map Wa_22016122933 buffers WC on the CPU side
Subsystems (2)
PATCH 1/1 drm/xe: Map Wa_22016122933 buffers WC on the CPU side
xe implements Wa_22016122933 by mapping the media GT's GuC-shared buffers uncached on the CPU side. i915 maps the same buffers WC on the CPU and UC on the GPU.
CPU writes through the uncached mapping are slow, and xe clears and refills the ADS on every GuC load, so probe and every resume pay for it. On ARL-H, xe probe takes 3.1 s with UC and 0.35 s with WC, the same as without the workaround.
Keep XE_BO_FLAG_NEEDS_UC, which also selects XE_CACHE_NONE for the GGTT PTEs, and add XE_BO_FLAG_FORCE_WC, which now makes the CPU mapping write-combined when both flags are set.
---
Tested on ARL-H 7d51 with this backported to 7.2.8. Probe time is dmesg "GT0: Using GuC firmware" to "Initialized xe": 3.10 and 3.15 s with UC, 0.35-0.37 s with WC (3 boots), 0.35 s on stock Arch 7.2.8, which lacks the workaround.
20 min of VAAPI decode with C-states 2/3 disabled (the reproducer from
https://gitlab.freedesktop.org/drm/xe/kernel/-/work_items/8678): no TLB invalidation timeouts and no late acks, same as with UC.
drivers/gpu/drm/xe/xe_bo.c | 7 ++++++- drivers/gpu/drm/xe/xe_guc.c | 6 ++++-- 2 files changed, 10 insertions(+), 3 deletions(-)
drivers/gpu/drm/xe/xe_bo.c +6 -1
@@ -533,7 +533,12 @@ static struct ttm_tt *xe_ttm_tt_create(struct ttm_buffer_object *ttm_bo,* which cpu_caching is never initialized. */ xe_assert (xe, bo->cpu_caching ==0 ); - caching = ttm_uncached; +/* +* With FORCE_WC also set, map WC on the CPU side. The GGTT PTEs +* stay XE_CACHE_NONE because they key off NEEDS_UC alone. +*/ + caching = bo->flags &XE_BO_FLAG_FORCE_WC ? ttm_write_combined : + ttm_uncached; }if (ttm_bo->type != ttm_bo_type_sg)
drivers/gpu/drm/xe/xe_guc.c +4 -2
@@ -1476,13 +1476,15 @@ int xe_guc_suspend(struct xe_guc *guc)* Wa_22016122933: on the standalone media GT, memory shared between the * CPU and the GuC must not be mapped cached on the CPU side, otherwise * the CPU can read stale data written by the GuC (e.g. G2H CTB writes) -* for multiple seconds. +* for multiple seconds. Map it WC on the CPU side and UC on the GPU side, +* as i915 does. * * Return: additional XE_BO_FLAG_* to use when allocating GuC-shared memory */ u32 xe_guc_bo_wa_flags (struct xe_gt *gt) { -return XE_GT_WA (gt,22016122933 ) ?XE_BO_FLAG_NEEDS_UC :0 ; +return XE_GT_WA (gt,22016122933 ) ? +XE_BO_FLAG_NEEDS_UC |XE_BO_FLAG_FORCE_WC :0 ; }void xe_guc_notify (struct xe_guc *guc)
Signature · 1 line
2.56.0
Patchwork
CI✗ LGCI.VerificationFailed: failure for drm/xe: Map Wa_22016122933 buffers WC on the CPU side
== Series Details ==
== Series Details ==
Series: drm/xe: Map Wa_22016122933 buffers WC on the CPU side
URL : https://patchwork.freedesktop.org/series/175580/
State : failure
== Summary ==
Series author address '[email protected]' is not on the allowlist, which prevents CI from being automatically triggered. If you want CI to run for this series, ask Patchwork project owners to click 'retest' on the series in Patchwork. Exception occurred during validation, bailing out!
Build URL: http://intel-gfx-ci-public.igk.intel.com:8080/job/xe_pw_trigger/1303039/ (on master)
Quoted 3 lines · Navon John Lukose
xe implements
Wa_22016122933by mapping the media GT's GuC-shared buffers uncached on the CPU side. i915 maps the same buffers WC on the CPU and UC on the GPU.
Thanks for catching this, and sorry for the regression.
I reviewed it against 7.2.11. The GGTT PTEs still key their cache mode off XE_BO_FLAG_NEEDS_UC alone (both encode paths in xe_ggtt.c), so the GPU side stays XE_CACHE_NONE as the workaround needs. Every user of xe_guc_bo_wa_flags() picks up FORCE_WC: ADS, CTB, GuC log, SLPC and engine activity. The only other internal NEEDS_UC user that touches the GuC, the UM queue BO in xe_guc_ads.c, predates the workaround and is left alone. CPU reads through WC still bypass the cache, so the stale G2H reads the workaround is about cannot come back.
Tested on a second ARL-H 7d51 (GuC 70.53.0), on 7.2.11 carrying 63360b32, fc668a6e and d5b0bf3f, with only xe.ko rebuilt with this patch.
Probe, dmesg "GT0: Using GuC firmware" to "Initialized xe":
d5b0bf3f only: 3.56, 3.49, 3.52 s
with this patch: 0.37 s
Resume from s2idle, wake ("ACPI: EC: interrupt unblocked") to "PM: suspend exit":
d5b0bf3f only: 2.43, 2.92, 2.97, 2.58, 2.93 s
(xe done after 2.28-2.83 s)
with this patch: 0.39 s (37 s asleep), 0.40 s (386 s asleep)
(xe done after 0.11 s both times)
20 minutes of VA-API decode, two decoders in parallel on 4K 10-bit HEVC clips (528 decodes): no TLB invalidation timeouts, no late acks, no job timeouts, no GT resets. I did not disable C-states for that run.
Over the same session I got one media GT "not started" job timeout in a browser, recovered by a GT reset. It is the same thing I already see without this patch (four devcoredumps since October 7 on the UC version, all with the LRC tail in memory still equal to the head), so it looks unrelated to this patch. I have reported it separately in
https://gitlab.freedesktop.org/drm/xe/kernel/-/work_items/8678.
Thanks,
Tales