Skip to contentSkip to conversation

drm/i915: Move to CPU domain in pwrite/pread

PATCH In review [email protected] 3 participants 6 messages Nov 12, 2014 – Nov 15, 2014 1 message not in the archive

Subsystems (2)
lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw

PATCH v3 1/1 drm/i915: Move to CPU domain in pwrite/pread

From: Ville Syrjälä <[email protected]>

Currently it's possible to get visible cache dirt on scanout on LLC machines when using pwrite on the future scanout bo if its cache_level is already NONE.

pwrite's "does this need clflush?" checks would decide that no clflush is necessary since the bo isn't currently pinned to the display and LLC makes everything else coherent. The subsequent set_cache_level(NONE) would also do nothing since cache_level is already correct. And hence no clflush will be performed and we flip to a bo which can still have dirty data in the caches.

To correctly track the cache dirtyness move the object to CPU write domain in pwrite. This cures the cache dirt since we explicitly flush the CPU write domain in the pin_to_display path.

Give pread the same treatment simply in the name of symmetry.

v2: Use trace_i915_gem_object_change_domain() and provide some kind
of commit message
v3: Don't mark things as clean if we're not sure everything got
flushed (Chris)

---
 drivers/gpu/drm/i915/i915_gem.c | 39 ++++++++++++++++++++++++++++++++++++++-
 1 file changed, 38 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_gem.c +38 -1
index 3e0cabe..117b064 100644
@@ -456,6 +456,7 @@ __copy_from_user_swizzled(char *gpu_vaddr, int gpu_offset,
 int i915_gem_obj_prepare_shmem_read(struct drm_i915_gem_object *obj,
 				    int *needs_clflush)
 {
+	u32 old_read_domains;
 	int ret;
 
 	*needs_clflush = 0;
@@ -475,6 +476,7 @@ int i915_gem_obj_prepare_shmem_read(struct drm_i915_gem_object *obj,
 			return ret;
 
 		i915_gem_object_retire(obj);
+		i915_gem_object_flush_gtt_write_domain(obj);
 	}
 
 	ret = i915_gem_object_get_pages(obj);
@@ -483,6 +485,13 @@ int i915_gem_obj_prepare_shmem_read(struct drm_i915_gem_object *obj,
 
 	i915_gem_object_pin_pages(obj);
 
+	old_read_domains = obj->base.read_domains;
+	obj->base.read_domains |= I915_GEM_DOMAIN_CPU;
+
+	trace_i915_gem_object_change_domain(obj,
+					    old_read_domains,
+					    obj->base.write_domain);
+
 	return ret;
 }
 
@@ -875,6 +884,7 @@ i915_gem_shmem_pwrite(struct drm_device *dev,
 	int needs_clflush_after = 0;
 	int needs_clflush_before = 0;
 	struct sg_page_iter sg_iter;
+	u32 old_write_domain, old_read_domains;
 
 	user_data = to_user_ptr(args->data_ptr);
 	remain = args->size;
@@ -892,6 +902,7 @@ i915_gem_shmem_pwrite(struct drm_device *dev,
 			return ret;
 
 		i915_gem_object_retire(obj);
+		i915_gem_object_flush_gtt_write_domain(obj);
 	}
 	/* Same trick applies to invalidate partially written cachelines read
 	 * before writing. */
@@ -908,6 +919,15 @@ i915_gem_shmem_pwrite(struct drm_device *dev,
 	offset = args->offset;
 	obj->dirty = 1;
 
+	old_read_domains = obj->base.read_domains;
+	old_write_domain = obj->base.write_domain;
+	obj->base.read_domains = I915_GEM_DOMAIN_CPU;
+	obj->base.write_domain = I915_GEM_DOMAIN_CPU;
+
+	trace_i915_gem_object_change_domain(obj,
+					    old_read_domains,
+					    old_write_domain);
+
 	for_each_sg_page(obj->pages->sgl, &sg_iter, obj->pages->nents,
 			 offset >> PAGE_SHIFT) {
 		struct page *page = sg_page_iter_page(&sg_iter);
@@ -978,9 +998,26 @@ out:
 		}
 	}
 
-	if (needs_clflush_after)
+	if (needs_clflush_after) {
 		i915_gem_chipset_flush(dev);
 
+		/*
+		 * If the caches were clean in the start they should
+		 * still be clean due to the clflushes we did. Otherwise
+		 * we may not have flushed all the dirty cachelines and
+		 * thus must leave write_domain alone.
+		 */
+		if (old_write_domain != I915_GEM_DOMAIN_CPU &&
+		    obj->base.write_domain == I915_GEM_DOMAIN_CPU) {
+			old_write_domain = obj->base.write_domain;
+			obj->base.write_domain = 0;
+
+			trace_i915_gem_object_change_domain(obj,
+							    obj->base.read_domains,
+							    old_write_domain);
+		}
+	}
+
 	return ret;
 }
 
Signature · 6 lines
2.0.4

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Tested-by on v3 1/1
#2 lore raw

-------------------------------------Summary-------------------------------------
Platform: baseline_drm_intel_nightly_pass_rate->patch_applied_pass_rate
BYT: pass/total=291/291->284/291
PNV: pass/total=356/356->350/356
ILK: pass/total=372/372->365/372
IVB: pass/total=544/546->545/546
SNB: pass/total=378/380->379/380
HSW: pass/total=579/579->577/579
BDW: pass/total=433/435->434/435
-------------------------------------Detailed-------------------------------------
test_platform: test_suite, test_case, result_with_drm_intel_nightly(count, machine_id...)...->result_with_patch_applied(count, machine_id)...
BYT: Intel_gpu_tools, igt_gem_pread_after_blit_interruptible, FAIL(3, M36)PASS(1, M38) -> FAIL(1, M36)PASS(3, M36)
BYT: Intel_gpu_tools, igt_gem_pread_after_blit_interruptible-display, FAIL(3, M36)PASS(1, M38) -> FAIL(1, M36)PASS(3, M36)
BYT: Intel_gpu_tools, igt_gem_pread_after_blit_interruptible-uncached, FAIL(2, M36)CRASH(1, M36)PASS(1, M38) -> FAIL(1, M36)PASS(3, M36)
BYT: Intel_gpu_tools, igt_gem_pread_after_blit_normal, FAIL(3, M36)PASS(1, M38) -> FAIL(1, M36)PASS(3, M36)
BYT: Intel_gpu_tools, igt_gem_pread_after_blit_normal-display, FAIL(3, M36)PASS(1, M38) -> FAIL(1, M36)PASS(3, M36)
BYT: Intel_gpu_tools, igt_gem_pread_after_blit_normal-uncached, FAIL(3, M36)PASS(1, M38) -> FAIL(1, M36)PASS(3, M36)
BYT: Intel_gpu_tools, igt_kms_setmode_invalid-clone-single-crtc, TIMEOUT(3, M36)PASS(1, M38) -> TIMEOUT(4, M36)
PNV: Intel_gpu_tools, igt_gem_pread_after_blit_interruptible, FAIL(3, M24)PASS(1, M23) -> FAIL(1, M24)PASS(3, M24)
PNV: Intel_gpu_tools, igt_gem_pread_after_blit_interruptible-display, FAIL(3, M24)PASS(1, M23) -> FAIL(1, M24)PASS(3, M24)
PNV: Intel_gpu_tools, igt_gem_pread_after_blit_interruptible-uncached, FAIL(3, M24)PASS(1, M23) -> FAIL(1, M24)PASS(3, M24)
PNV: Intel_gpu_tools, igt_gem_pread_after_blit_normal, FAIL(3, M24)PASS(1, M23) -> FAIL(1, M24)DMESG_WARN(1, M24)PASS(2, M24)
PNV: Intel_gpu_tools, igt_gem_pread_after_blit_normal-display, FAIL(3, M24)PASS(1, M23) -> FAIL(1, M24)PASS(3, M24)
PNV: Intel_gpu_tools, igt_gem_pread_after_blit_normal-uncached, FAIL(3, M24)PASS(1, M23) -> FAIL(1, M24)PASS(3, M24)
ILK: Intel_gpu_tools, igt_gem_pread_after_blit_interruptible, FAIL(3, M37)PASS(1, M6) -> FAIL(1, M37)PASS(3, M37)
ILK: Intel_gpu_tools, igt_gem_pread_after_blit_interruptible-display, FAIL(3, M37)PASS(1, M6) -> FAIL(1, M37)PASS(3, M37)
ILK: Intel_gpu_tools, igt_gem_pread_after_blit_interruptible-uncached, FAIL(3, M37)PASS(1, M6) -> FAIL(1, M37)PASS(3, M37)
ILK: Intel_gpu_tools, igt_gem_pread_after_blit_normal, FAIL(3, M37)PASS(1, M6) -> FAIL(1, M37)PASS(3, M37)
ILK: Intel_gpu_tools, igt_gem_pread_after_blit_normal-display, FAIL(3, M37)PASS(1, M6) -> FAIL(1, M37)PASS(3, M37)
ILK: Intel_gpu_tools, igt_gem_pread_after_blit_normal-uncached, FAIL(3, M37)PASS(1, M6) -> FAIL(1, M37)PASS(3, M37)
ILK: Intel_gpu_tools, igt_kms_setmode_invalid-clone-single-crtc, FAIL(3, M26)DMESG_FAIL(1, M26)TIMEOUT(17, M37M6M26)PASS(1, M26) -> TIMEOUT(4, M37)
IVB: Intel_gpu_tools, igt_gem_bad_reloc_negative-reloc, NSPT(2, M4M21)PASS(2, M21) -> NSPT(1, M21)PASS(3, M21)
SNB: Intel_gpu_tools, igt_gem_concurrent_blit_gpuX-bcs-overwrite-source-forked, FAIL(1, M22)PASS(3, M35) -> PASS(4, M35)
HSW: Intel_gpu_tools, igt_kms_flip_tiling_flip-changes-tiling, PASS(4, M39) -> DMESG_WARN(1, M39)PASS(3, M39)
HSW: Intel_gpu_tools, igt_pm_rpm_fences-dpms, PASS(4, M39) -> DMESG_WARN(1, M39)PASS(3, M39)
BDW: Intel_gpu_tools, igt_gem_reset_stats_ban-bsd, DMESG_WARN(1, M30)PASS(3, M30) -> PASS(4, M30)
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#3 lore raw

PATCH 1/1 tests/kms_pwrite_crc: Add pwrite vs display coherency test

In reply to [email protected]

From: Ville Syrjälä <[email protected]>

Add a test to verify that pwriting to a future scanout buffer works correctly. The specific problem occurs when the buffer is already UC/WT before the pwrite, not the current scanout buffer, and not currently in the CPU write domain. With the buggy kernel no clflush will be performed after the pwrite, and hence we end up with cache dirt on the display.

The problem only affects LLC platforms (non-LLC would clflush anywway after pwrite), but we can let th test run on all platforms.

---
 tests/Makefile.sources |   3 +-
 tests/kms_pwrite_crc.c | 217 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 219 insertions(+), 1 deletion(-)
 create mode 100644 tests/kms_pwrite_crc.c
tests/Makefile.sources +2 -1
index afbe597..2677bf7 100644
@@ -145,9 +145,10 @@ TESTS_progs = \
 	gen3_render_tiledy_blits \
 	gen7_forcewake_mt \
 	kms_3d \
+	kms_fence_pin_leak \
 	kms_force_connector \
+	kms_pwrite_crc \
 	kms_sink_crc_basic \
-	kms_fence_pin_leak \
 	prime_udl \
 	$(NULL)
 
tests/kms_pwrite_crc.c new file +217 -0

217 lines not on this page · raw message

Signature · 6 lines
2.0.4

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#4 lore raw
In reply to [email protected]
Quoted 17 lines · On Wed, Nov 12, 2014 at 11:47:14PM +0200, [email protected] wrote:

From: Ville Syrjälä <[email protected]>

Currently it's possible to get visible cache dirt on scanout on LLC machines when using pwrite on the future scanout bo if its cache_level is already NONE.

pwrite's "does this need clflush?" checks would decide that no clflush is necessary since the bo isn't currently pinned to the display and LLC makes everything else coherent. The subsequent set_cache_level(NONE) would also do nothing since cache_level is already correct. And hence no clflush will be performed and we flip to a bo which can still have dirty data in the caches.

To correctly track the cache dirtyness move the object to CPU write domain in pwrite. This cures the cache dirt since we explicitly flush the CPU write domain in the pin_to_display path.

Quoted 6 lines

Give pread the same treatment simply in the name of symmetry.

v2: Use trace_i915_gem_object_change_domain() and provide some kind
of commit message
v3: Don't mark things as clean if we're not sure everything got
flushed (Chris)

I think we just want to be more conservative during clflushes after pwrite:

drivers/gpu/drm/i915/i915_gem.c +1 -1
index 557746b2b72b..e9f98531b9d2 100644
@@ -75,7 +75,7 @@ static bool cpu_cache_is_coherent(struct drm_device *dev,
 
 static bool cpu_write_needs_clflush(struct drm_i915_gem_object *obj)
 {
-       if (!cpu_cache_is_coherent(obj->base.dev, obj->cache_level))
+       if (level != I915_CACHE_NONE)
                return true;
 
        return obj->pin_display;
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#5 lore raw
Quoted 33 lines · Chris Wilson
Quoted 17 lines · On Wed, Nov 12, 2014 at 11:47:14PM +0200, [email protected] wrote:

From: Ville Syrjälä <[email protected]>

Currently it's possible to get visible cache dirt on scanout on LLC machines when using pwrite on the future scanout bo if its cache_level is already NONE.

pwrite's "does this need clflush?" checks would decide that no clflush is necessary since the bo isn't currently pinned to the display and LLC makes everything else coherent. The subsequent set_cache_level(NONE) would also do nothing since cache_level is already correct. And hence no clflush will be performed and we flip to a bo which can still have dirty data in the caches.

To correctly track the cache dirtyness move the object to CPU write domain in pwrite. This cures the cache dirt since we explicitly flush the CPU write domain in the pin_to_display path.

Quoted 6 lines

Give pread the same treatment simply in the name of symmetry.

v2: Use trace_i915_gem_object_change_domain() and provide some kind
of commit message
v3: Don't mark things as clean if we're not sure everything got
flushed (Chris)

I think we just want to be more conservative during clflushes after pwrite:

drivers/gpu/drm/i915/i915_gem.c +0 -0
index 557746b2b72b..e9f98531b9d2 100644

drivers/gpu/drm/i915/i915_gem.c static bool cpu_cache_is_coherent(struct drm_device *dev, · line 78

Quoted 5 lines
drivers/gpu/drm/i915/i915_gem.c +1 -1
@@ -75,7 +75,7 @@ static bool cpu_cache_is_coherent(struct drm_device *dev,
 
 static bool cpu_write_needs_clflush(struct drm_i915_gem_object *obj)
 {
-       if (!cpu_cache_is_coherent(obj->base.dev, obj->cache_level))
+       if (level != I915_CACHE_NONE)

You mean == ?

And I guess you'd then have to consider WT as well.

It would mean we'd end up clflushing even when not strictly needed. But maybe that's acceptable.

Quoted 6 lines
drivers/gpu/drm/i915/i915_gem.c +0 -0
                return true;
 
        return obj->pin_display;
Signature · 2 lines
-- 
Chris Wilson, Intel Open Source Technology Centre
Signature · 6 lines
Ville Syrjälä
Intel OTC
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#6 lore raw
Quoted 35 lines · Ville Syrjälä
Quoted 34 lines · Chris Wilson
Quoted 17 lines · On Wed, Nov 12, 2014 at 11:47:14PM +0200, [email protected] wrote:

From: Ville Syrjälä <[email protected]>

Currently it's possible to get visible cache dirt on scanout on LLC machines when using pwrite on the future scanout bo if its cache_level is already NONE.

pwrite's "does this need clflush?" checks would decide that no clflush is necessary since the bo isn't currently pinned to the display and LLC makes everything else coherent. The subsequent set_cache_level(NONE) would also do nothing since cache_level is already correct. And hence no clflush will be performed and we flip to a bo which can still have dirty data in the caches.

To correctly track the cache dirtyness move the object to CPU write domain in pwrite. This cures the cache dirt since we explicitly flush the CPU write domain in the pin_to_display path.

Quoted 6 lines

Give pread the same treatment simply in the name of symmetry.

v2: Use trace_i915_gem_object_change_domain() and provide some kind
of commit message
v3: Don't mark things as clean if we're not sure everything got
flushed (Chris)

I think we just want to be more conservative during clflushes after pwrite:

drivers/gpu/drm/i915/i915_gem.c +0 -0
index 557746b2b72b..e9f98531b9d2 100644
@@ -75,7 +75,7 @@ static bool cpu_cache_is_coherent(struct drm_device *dev,
 
Quoted 6 lines

drivers/gpu/drm/i915/i915_gem.c static bool cpu_cache_is_coherent(struct drm_device *dev, · line 78

Quoted 4 lines
drivers/gpu/drm/i915/i915_gem.c +1 -1
 static bool cpu_write_needs_clflush(struct drm_i915_gem_object *obj)
 {
-       if (!cpu_cache_is_coherent(obj->base.dev, obj->cache_level))
+       if (level != I915_CACHE_NONE)

You mean == ?

Indeed.

Quoted 4 lines

And I guess you'd then have to consider WT as well.

It would mean we'd end up clflushing even when not strictly needed. But maybe that's acceptable.

Yes. I think it is better to err on clflushing too often and worry later if someone batters us over the head with a profile. The only mistake I know about, in this regard, is my pwrite of the sprite planes and the mmap(wc) fixes that, or I can flag those planes as fenced which may be preferrable by hw (I didn't think so at the time)? The much more frequent use case (and the only other one that immediately springs to mind) will not trigger the clflushes.
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx