drm/i915: Move to CPU domain in pwrite/pread
Subsystems (2)
A message that is not in the archive (sent off-list or not yet received): [email protected]
PATCH v3 1/1 drm/i915: Move to CPU domain in pwrite/pread
From: Ville Syrjälä <[email protected]>
Currently it's possible to get visible cache dirt on scanout on LLC machines when using pwrite on the future scanout bo if its cache_level is already NONE.
pwrite's "does this need clflush?" checks would decide that no clflush is necessary since the bo isn't currently pinned to the display and LLC makes everything else coherent. The subsequent set_cache_level(NONE) would also do nothing since cache_level is already correct. And hence no clflush will be performed and we flip to a bo which can still have dirty data in the caches.
To correctly track the cache dirtyness move the object to CPU write domain in pwrite. This cures the cache dirt since we explicitly flush the CPU write domain in the pin_to_display path.
Give pread the same treatment simply in the name of symmetry.
v2: Use trace_i915_gem_object_change_domain() and provide some kind
of commit message
v3: Don't mark things as clean if we're not sure everything got
flushed (Chris)
--- drivers/gpu/drm/i915/i915_gem.c | 39 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 38 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_gem.c +38 -1
@@ -456,6 +456,7 @@ __copy_from_user_swizzled(char *gpu_vaddr, int gpu_offset,int i915_gem_obj_prepare_shmem_read (struct drm_i915_gem_object *obj,int *needs_clflush) { +u32 old_read_domains;int ret; *needs_clflush =0 ;
@@ -475,6 +476,7 @@ int i915_gem_obj_prepare_shmem_read(struct drm_i915_gem_object *obj,return ret;i915_gem_object_retire (obj); +i915_gem_object_flush_gtt_write_domain (obj); } ret =i915_gem_object_get_pages (obj);
@@ -483,6 +485,13 @@ int i915_gem_obj_prepare_shmem_read(struct drm_i915_gem_object *obj,i915_gem_object_pin_pages (obj); + old_read_domains = obj->base.read_domains; + obj->base.read_domains |=I915_GEM_DOMAIN_CPU ; + +trace_i915_gem_object_change_domain (obj, + old_read_domains, + obj->base.write_domain); +return ret; }
@@ -875,6 +884,7 @@ i915_gem_shmem_pwrite(struct drm_device *dev,int needs_clflush_after =0 ;int needs_clflush_before =0 ;struct sg_page_iter sg_iter; +u32 old_write_domain, old_read_domains; user_data =to_user_ptr (args->data_ptr); remain = args->size;
@@ -892,6 +902,7 @@ i915_gem_shmem_pwrite(struct drm_device *dev,return ret;i915_gem_object_retire (obj); +i915_gem_object_flush_gtt_write_domain (obj); }/* Same trick applies to invalidate partially written cachelines read * before writing. */
@@ -908,6 +919,15 @@ i915_gem_shmem_pwrite(struct drm_device *dev, offset = args->offset; obj->dirty =1 ; + old_read_domains = obj->base.read_domains; + old_write_domain = obj->base.write_domain; + obj->base.read_domains =I915_GEM_DOMAIN_CPU ; + obj->base.write_domain =I915_GEM_DOMAIN_CPU ; + +trace_i915_gem_object_change_domain (obj, + old_read_domains, + old_write_domain); +for_each_sg_page (obj->pages->sgl, &sg_iter, obj->pages->nents, offset >>PAGE_SHIFT ) {struct page *page =sg_page_iter_page (&sg_iter);
@@ -978,9 +998,26 @@ out: } } -if (needs_clflush_after) +if (needs_clflush_after) {i915_gem_chipset_flush (dev); +/* +* If the caches were clean in the start they should +* still be clean due to the clflushes we did. Otherwise +* we may not have flushed all the dirty cachelines and +* thus must leave write_domain alone. +*/ +if (old_write_domain !=I915_GEM_DOMAIN_CPU && + obj->base.write_domain ==I915_GEM_DOMAIN_CPU ) { + old_write_domain = obj->base.write_domain; + obj->base.write_domain =0 ; + +trace_i915_gem_object_change_domain (obj, + obj->base.read_domains, + old_write_domain); + } + } +return ret; }
Signature · 6 lines
2.0.4 _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx
-------------------------------------Summary-------------------------------------
Platform: baseline_drm_intel_nightly_pass_rate->patch_applied_pass_rate
BYT: pass/total=291/291->284/291
PNV: pass/total=356/356->350/356
ILK: pass/total=372/372->365/372
IVB: pass/total=544/546->545/546
SNB: pass/total=378/380->379/380
HSW: pass/total=579/579->577/579
BDW: pass/total=433/435->434/435
-------------------------------------Detailed-------------------------------------test_platform: test_suite, test_case, result_with_drm_intel_nightly(count, machine_id...)...->result_with_patch_applied(count, machine_id)...
BYT: Intel_gpu_tools, igt_gem_pread_after_blit_interruptible, FAIL(3, M36)PASS(1, M38) -> FAIL(1, M36)PASS(3, M36)
BYT: Intel_gpu_tools, igt_gem_pread_after_blit_interruptible-display, FAIL(3, M36)PASS(1, M38) -> FAIL(1, M36)PASS(3, M36)
BYT: Intel_gpu_tools, igt_gem_pread_after_blit_interruptible-uncached, FAIL(2, M36)CRASH(1, M36)PASS(1, M38) -> FAIL(1, M36)PASS(3, M36)
BYT: Intel_gpu_tools, igt_gem_pread_after_blit_normal, FAIL(3, M36)PASS(1, M38) -> FAIL(1, M36)PASS(3, M36)
BYT: Intel_gpu_tools, igt_gem_pread_after_blit_normal-display, FAIL(3, M36)PASS(1, M38) -> FAIL(1, M36)PASS(3, M36)
BYT: Intel_gpu_tools, igt_gem_pread_after_blit_normal-uncached, FAIL(3, M36)PASS(1, M38) -> FAIL(1, M36)PASS(3, M36)
BYT: Intel_gpu_tools, igt_kms_setmode_invalid-clone-single-crtc, TIMEOUT(3, M36)PASS(1, M38) -> TIMEOUT(4, M36)
PNV: Intel_gpu_tools, igt_gem_pread_after_blit_interruptible, FAIL(3, M24)PASS(1, M23) -> FAIL(1, M24)PASS(3, M24)
PNV: Intel_gpu_tools, igt_gem_pread_after_blit_interruptible-display, FAIL(3, M24)PASS(1, M23) -> FAIL(1, M24)PASS(3, M24)
PNV: Intel_gpu_tools, igt_gem_pread_after_blit_interruptible-uncached, FAIL(3, M24)PASS(1, M23) -> FAIL(1, M24)PASS(3, M24)
PNV: Intel_gpu_tools, igt_gem_pread_after_blit_normal, FAIL(3, M24)PASS(1, M23) -> FAIL(1, M24)DMESG_WARN(1, M24)PASS(2, M24)
PNV: Intel_gpu_tools, igt_gem_pread_after_blit_normal-display, FAIL(3, M24)PASS(1, M23) -> FAIL(1, M24)PASS(3, M24)
PNV: Intel_gpu_tools, igt_gem_pread_after_blit_normal-uncached, FAIL(3, M24)PASS(1, M23) -> FAIL(1, M24)PASS(3, M24)
ILK: Intel_gpu_tools, igt_gem_pread_after_blit_interruptible, FAIL(3, M37)PASS(1, M6) -> FAIL(1, M37)PASS(3, M37)
ILK: Intel_gpu_tools, igt_gem_pread_after_blit_interruptible-display, FAIL(3, M37)PASS(1, M6) -> FAIL(1, M37)PASS(3, M37)
ILK: Intel_gpu_tools, igt_gem_pread_after_blit_interruptible-uncached, FAIL(3, M37)PASS(1, M6) -> FAIL(1, M37)PASS(3, M37)
ILK: Intel_gpu_tools, igt_gem_pread_after_blit_normal, FAIL(3, M37)PASS(1, M6) -> FAIL(1, M37)PASS(3, M37)
ILK: Intel_gpu_tools, igt_gem_pread_after_blit_normal-display, FAIL(3, M37)PASS(1, M6) -> FAIL(1, M37)PASS(3, M37)
ILK: Intel_gpu_tools, igt_gem_pread_after_blit_normal-uncached, FAIL(3, M37)PASS(1, M6) -> FAIL(1, M37)PASS(3, M37)
ILK: Intel_gpu_tools, igt_kms_setmode_invalid-clone-single-crtc, FAIL(3, M26)DMESG_FAIL(1, M26)TIMEOUT(17, M37M6M26)PASS(1, M26) -> TIMEOUT(4, M37)
IVB: Intel_gpu_tools, igt_gem_bad_reloc_negative-reloc, NSPT(2, M4M21)PASS(2, M21) -> NSPT(1, M21)PASS(3, M21)
SNB: Intel_gpu_tools, igt_gem_concurrent_blit_gpuX-bcs-overwrite-source-forked, FAIL(1, M22)PASS(3, M35) -> PASS(4, M35)
HSW: Intel_gpu_tools, igt_kms_flip_tiling_flip-changes-tiling, PASS(4, M39) -> DMESG_WARN(1, M39)PASS(3, M39)
HSW: Intel_gpu_tools, igt_pm_rpm_fences-dpms, PASS(4, M39) -> DMESG_WARN(1, M39)PASS(3, M39)
BDW: Intel_gpu_tools, igt_gem_reset_stats_ban-bsd, DMESG_WARN(1, M30)PASS(3, M30) -> PASS(4, M30)
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH 1/1 tests/kms_pwrite_crc: Add pwrite vs display coherency test
In reply to [email protected]From: Ville Syrjälä <[email protected]>
Add a test to verify that pwriting to a future scanout buffer works correctly. The specific problem occurs when the buffer is already UC/WT before the pwrite, not the current scanout buffer, and not currently in the CPU write domain. With the buggy kernel no clflush will be performed after the pwrite, and hence we end up with cache dirt on the display.
The problem only affects LLC platforms (non-LLC would clflush anywway after pwrite), but we can let th test run on all platforms.
--- tests/Makefile.sources | 3 +- tests/kms_pwrite_crc.c | 217 +++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 219 insertions(+), 1 deletion(-) create mode 100644 tests/kms_pwrite_crc.c
tests/Makefile.sources +2 -1
@@ -145,9 +145,10 @@ TESTS_progs = \ gen3_render_tiledy_blits \ gen7_forcewake_mt \ kms_3d \ + kms_fence_pin_leak \ kms_force_connector \ + kms_pwrite_crc \ kms_sink_crc_basic \ - kms_fence_pin_leak \ prime_udl \ $(NULL)
tests/kms_pwrite_crc.c new file +217 -0
217 lines not on this page · raw message
Signature · 6 lines
2.0.4 _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 17 lines · On Wed, Nov 12, 2014 at 11:47:14PM +0200, [email protected] wrote:
From: Ville Syrjälä <[email protected]>
Currently it's possible to get visible cache dirt on scanout on LLC machines when using pwrite on the future scanout bo if its
cache_levelis already NONE.pwrite's "does this need clflush?" checks would decide that no clflush is necessary since the bo isn't currently pinned to the display and LLC makes everything else coherent. The subsequent
set_cache_level(NONE)would also do nothing sincecache_levelis already correct. And hence no clflush will be performed and we flip to a bo which can still have dirty data in the caches.To correctly track the cache dirtyness move the object to CPU write domain in pwrite. This cures the cache dirt since we explicitly flush the CPU write domain in the
pin_to_displaypath.
Quoted 6 lines
Give pread the same treatment simply in the name of symmetry.
v2: Use
trace_i915_gem_object_change_domain()and provide some kind
of commit message
v3: Don't mark things as clean if we're not sure everything got
flushed (Chris)
I think we just want to be more conservative during clflushes after pwrite:
drivers/gpu/drm/i915/i915_gem.c +1 -1
@@ -75,7 +75,7 @@ static bool cpu_cache_is_coherent(struct drm_device *dev,static bool cpu_write_needs_clflush (struct drm_i915_gem_object *obj) { -if (!cpu_cache_is_coherent (obj->base.dev, obj->cache_level)) +if (level !=I915_CACHE_NONE )return true;return obj->pin_display;
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 33 lines · Chris Wilson
Quoted 17 lines · On Wed, Nov 12, 2014 at 11:47:14PM +0200, [email protected] wrote:
From: Ville Syrjälä <[email protected]>
Currently it's possible to get visible cache dirt on scanout on LLC machines when using pwrite on the future scanout bo if its
cache_levelis already NONE.pwrite's "does this need clflush?" checks would decide that no clflush is necessary since the bo isn't currently pinned to the display and LLC makes everything else coherent. The subsequent
set_cache_level(NONE)would also do nothing sincecache_levelis already correct. And hence no clflush will be performed and we flip to a bo which can still have dirty data in the caches.To correctly track the cache dirtyness move the object to CPU write domain in pwrite. This cures the cache dirt since we explicitly flush the CPU write domain in the
pin_to_displaypath.Quoted 6 lines
Give pread the same treatment simply in the name of symmetry.
v2: Use
trace_i915_gem_object_change_domain()and provide some kind
of commit message
v3: Don't mark things as clean if we're not sure everything got
flushed (Chris)I think we just want to be more conservative during clflushes after pwrite:
drivers/gpu/drm/i915/i915_gem.c +0
-0
drivers/gpu/drm/i915/i915_gem.c static bool cpu_cache_is_coherent(struct drm_device *dev, · line 78
Quoted 5 lines
drivers/gpu/drm/i915/i915_gem.c +1
-1@@ -75,7 +75,7 @@ static bool cpu_cache_is_coherent(struct drm_device *dev,static bool cpu_write_needs_clflush (struct drm_i915_gem_object *obj) { -if (!cpu_cache_is_coherent (obj->base.dev, obj->cache_level)) +if (level !=I915_CACHE_NONE )
You mean == ?
And I guess you'd then have to consider WT as well.
It would mean we'd end up clflushing even when not strictly needed. But maybe that's acceptable.
Quoted 6 lines
drivers/gpu/drm/i915/i915_gem.c +0
-0return true;return obj->pin_display;Signature · 2 lines
-- Chris Wilson, Intel Open Source Technology Centre
Signature · 6 lines
Ville Syrjälä Intel OTC _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 35 lines · Ville Syrjälä
Quoted 34 lines · Chris Wilson
Quoted 17 lines · On Wed, Nov 12, 2014 at 11:47:14PM +0200, [email protected] wrote:
From: Ville Syrjälä <[email protected]>
Currently it's possible to get visible cache dirt on scanout on LLC machines when using pwrite on the future scanout bo if its
cache_levelis already NONE.pwrite's "does this need clflush?" checks would decide that no clflush is necessary since the bo isn't currently pinned to the display and LLC makes everything else coherent. The subsequent
set_cache_level(NONE)would also do nothing sincecache_levelis already correct. And hence no clflush will be performed and we flip to a bo which can still have dirty data in the caches.To correctly track the cache dirtyness move the object to CPU write domain in pwrite. This cures the cache dirt since we explicitly flush the CPU write domain in the
pin_to_displaypath.Quoted 6 lines
Give pread the same treatment simply in the name of symmetry.
v2: Use
trace_i915_gem_object_change_domain()and provide some kind
of commit message
v3: Don't mark things as clean if we're not sure everything got
flushed (Chris)I think we just want to be more conservative during clflushes after pwrite:
drivers/gpu/drm/i915/i915_gem.c +0
-0@@ -75,7 +75,7 @@ static bool cpu_cache_is_coherent(struct drm_device *dev,
Quoted 6 lines
drivers/gpu/drm/i915/i915_gem.c static bool cpu_cache_is_coherent(struct drm_device *dev, · line 78
Quoted 4 lines
drivers/gpu/drm/i915/i915_gem.c +1
-1static bool cpu_write_needs_clflush (struct drm_i915_gem_object *obj) { -if (!cpu_cache_is_coherent (obj->base.dev, obj->cache_level)) +if (level !=I915_CACHE_NONE )You mean == ?
Indeed.
Quoted 4 lines
And I guess you'd then have to consider WT as well.
It would mean we'd end up clflushing even when not strictly needed. But maybe that's acceptable.
Yes. I think it is better to err on clflushing too often and worry later if someone batters us over the head with a profile. The only mistake I know about, in this regard, is my pwrite of the sprite planes and the mmap(wc) fixes that, or I can flag those planes as fenced which may be preferrable by hw (I didn't think so at the time)? The much more frequent use case (and the only other one that immediately springs to mind) will not trigger the clflushes.
-Chris
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx