Skip to contentSkip to conversation

LLC and per-BO cache handling

RFC PATCH In review Eugeni Dodonov 3 participants 7 messages Dec 14, 2011

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 LLC and per-BO cache handling

With base on comments and feedback on LLC handling from Daniel Vetter, Chris Wilson and Ben Widawsky, I reworked my patch into a more useful approach for detecting presence of different cache levels from userspace.

So, in this context, PATCH 1 adds the very same .has_llc flag and debugfs entry for it. PATCH 2 does the initial job of preparing to export cache levels to userspace, which can be queried via the DRM interface by means of PATCH 3. And finally, PATCH 4 allows to get and set a specific cache level for each bo.

The i915_gem_object_set_cache_level() routine does a good job of checking whether the objects should be put into specific cache levels already. When thinking about possible optimizations and tweaks for different cache levels, this is what we thought:
- The most usual case is for I915_MADV_WILLNEED object, where we can change its cache level as we want.
- If we are trying to change cache level of I915_MADV_DONTNEED object, on first glance, it is pointless to do so. However, it could make sense for some sort of prefetching or quick change of mind (we certainly can mark an object as DONTNEED and get to reallocate and WILLNEED it again). So why not?
- If we are trying to change cache level of a PURGED object, nothing will happen, as it has no GTT space anyway.

One possible optimization would be to remove an object from LLC on gem_madvise_ioctl(..I915_MADV_DONTNEED). However, I don't think it is worth the extra cycles - at some point, it will be removed eventually anyway, even if we leave it at LLC.

So those are all the possible cases I could think of. Of course, I could have missed something, so just yell if so :).

#1 lore raw

PATCH 1/4 drm/i915: add a LLC feature flag in device description

From: Eugeni Dodonov <[email protected]>

LLC is not SNB-specific, so we should check for it in a more generic way.

v2: export LLC support status via debugfs.

---
 drivers/gpu/drm/i915/i915_debugfs.c |    1 +
 drivers/gpu/drm/i915/i915_drv.c     |    4 ++++
 drivers/gpu/drm/i915/i915_drv.h     |    2 ++
 drivers/gpu/drm/i915/i915_gem.c     |    4 ++--
 4 files changed, 9 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_debugfs.c +1 -0
index d09a6e0..cb8a153 100644
@@ -82,6 +82,7 @@ static int i915_capabilities(struct seq_file *m, void *data)
 	B(supports_tv);
 	B(has_bsd_ring);
 	B(has_blt_ring);
+	B(has_llc);
 #undef B
 
 	return 0;
drivers/gpu/drm/i915/i915_drv.c +4 -0
index 15bfa91..19fb7a4 100644
@@ -214,6 +214,7 @@ static const struct intel_device_info intel_sandybridge_d_info = {
 	.need_gfx_hws = 1, .has_hotplug = 1,
 	.has_bsd_ring = 1,
 	.has_blt_ring = 1,
+	.has_llc = 1,
 };
 
 static const struct intel_device_info intel_sandybridge_m_info = {
@@ -222,6 +223,7 @@ static const struct intel_device_info intel_sandybridge_m_info = {
 	.has_fbc = 1,
 	.has_bsd_ring = 1,
 	.has_blt_ring = 1,
+	.has_llc = 1,
 };
 
 static const struct intel_device_info intel_ivybridge_d_info = {
@@ -229,6 +231,7 @@ static const struct intel_device_info intel_ivybridge_d_info = {
 	.need_gfx_hws = 1, .has_hotplug = 1,
 	.has_bsd_ring = 1,
 	.has_blt_ring = 1,
+	.has_llc = 1,
 };
 
 static const struct intel_device_info intel_ivybridge_m_info = {
@@ -237,6 +240,7 @@ static const struct intel_device_info intel_ivybridge_m_info = {
 	.has_fbc = 0,	/* FBC is not enabled on Ivybridge mobile yet */
 	.has_bsd_ring = 1,
 	.has_blt_ring = 1,
+	.has_llc = 1,
 };
 
 static const struct pci_device_id pciidlist[] = {		/* aka */
drivers/gpu/drm/i915/i915_drv.h +2 -0
index 4a9c1b9..abbbf32 100644
@@ -250,6 +250,7 @@ struct intel_device_info {
 	u8 supports_tv:1;
 	u8 has_bsd_ring:1;
 	u8 has_blt_ring:1;
+	u8 has_llc:1;
 };
 
 enum no_fbc_reason {
@@ -961,6 +962,7 @@ struct drm_i915_file_private {
 
 #define HAS_BSD(dev)            (INTEL_INFO(dev)->has_bsd_ring)
 #define HAS_BLT(dev)            (INTEL_INFO(dev)->has_blt_ring)
+#define HAS_LLC(dev)            (INTEL_INFO(dev)->has_llc)
 #define I915_NEED_GFX_HWS(dev)	(INTEL_INFO(dev)->need_gfx_hws)
 
 #define HAS_OVERLAY(dev)		(INTEL_INFO(dev)->has_overlay)
drivers/gpu/drm/i915/i915_gem.c +2 -2
index 60ff1b6..fb69337 100644
@@ -3620,8 +3620,8 @@ struct drm_i915_gem_object *i915_gem_alloc_object(struct drm_device *dev,
 	obj->base.write_domain = I915_GEM_DOMAIN_CPU;
 	obj->base.read_domains = I915_GEM_DOMAIN_CPU;
 
-	if (IS_GEN6(dev) || IS_GEN7(dev)) {
-		/* On Gen6, we can have the GPU use the LLC (the CPU
+	if (HAS_LLC(dev)) {
+		/* On some devices, we can have the GPU use the LLC (the CPU
 		 * cache) for about a 10% performance improvement
 		 * compared to uncached.  Graphics requests other than
 		 * display scanout are coherent with the CPU in
Signature · 1 line
1.7.7.4
Reviewed-by on v1 1/4
#2 lore raw

Re: [PATCH 1/4] drm/i915: add a LLC feature flag in device description

Quoted 7 lines · On Wed, 14 Dec 2011 00:33:01 -0200 Eugeni Dodonov <[email protected]> wrote:

From: Eugeni Dodonov <[email protected]>

LLC is not SNB-specific, so we should check for it in a more generic way.

v2: export LLC support status via debugfs.

I like it.

Signature · 1 line
Jesse Barnes, Intel Open Source Technology Center

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#3 lore raw

PATCH 2/4 drm/i915: simplify cache parameters for userspace

In reply to Eugeni Dodonov

From: Eugeni Dodonov <[email protected]>

Simplify the code and cache-related parameters handling, and prepare to export them to userspace in subsequent patches.

---
 drivers/gpu/drm/i915/i915_drv.h     |   10 ++--------
 drivers/gpu/drm/i915/i915_gem.c     |    2 +-
 drivers/gpu/drm/i915/i915_gem_gtt.c |    4 ++--
 include/drm/i915_drm.h              |    8 ++++++++
 4 files changed, 13 insertions(+), 11 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +2 -8
index abbbf32..45b3f5b 100644
@@ -737,12 +737,6 @@ typedef struct drm_i915_private {
 	atomic_t forcewake_count;
 } drm_i915_private_t;
 
-enum i915_cache_level {
-	I915_CACHE_NONE,
-	I915_CACHE_LLC,
-	I915_CACHE_LLC_MLC, /* gen6+ */
-};
-
 struct drm_i915_gem_object {
 	struct drm_gem_object base;
 
@@ -1213,13 +1207,13 @@ i915_gem_get_unfenced_gtt_alignment(struct drm_device *dev,
 				    int tiling_mode);
 
 int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
-				    enum i915_cache_level cache_level);
+				    uint32_t cache_level);
 
 /* i915_gem_gtt.c */
 void i915_gem_restore_gtt_mappings(struct drm_device *dev);
 int __must_check i915_gem_gtt_bind_object(struct drm_i915_gem_object *obj);
 void i915_gem_gtt_rebind_object(struct drm_i915_gem_object *obj,
-				enum i915_cache_level cache_level);
+				uint32_t cache_level);
 void i915_gem_gtt_unbind_object(struct drm_i915_gem_object *obj);
 
 /* i915_gem_evict.c */
drivers/gpu/drm/i915/i915_gem.c +1 -1
index fb69337..dccec77 100644
@@ -2949,7 +2949,7 @@ i915_gem_object_set_to_gtt_domain(struct drm_i915_gem_object *obj, bool write)
 }
 
 int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
-				    enum i915_cache_level cache_level)
+				    uint32_t cache_level)
 {
 	int ret;
 
drivers/gpu/drm/i915/i915_gem_gtt.c +2 -2
index 6042c5e..d556dc8 100644
@@ -31,7 +31,7 @@
 
 /* XXX kill agp_type! */
 static unsigned int cache_level_to_agp_type(struct drm_device *dev,
-					    enum i915_cache_level cache_level)
+					    uint32_t cache_level)
 {
 	switch (cache_level) {
 	case I915_CACHE_LLC_MLC:
@@ -117,7 +117,7 @@ int i915_gem_gtt_bind_object(struct drm_i915_gem_object *obj)
 }
 
 void i915_gem_gtt_rebind_object(struct drm_i915_gem_object *obj,
-				enum i915_cache_level cache_level)
+				uint32_t cache_level)
 {
 	struct drm_device *dev = obj->base.dev;
 	struct drm_i915_private *dev_priv = dev->dev_private;
include/drm/i915_drm.h +8 -0
index 28c0d11..e9f1cf4 100644
@@ -844,4 +844,12 @@ struct drm_intel_overlay_attrs {
 	__u32 gamma5;
 };
 
+
+/**
+ * Cache level definitions
+ */
+#define I915_CACHE_NONE				0
+#define I915_CACHE_LLC				1
+#define I915_CACHE_LLC_MLC			2
+
 #endif				/* _I915_DRM_H_ */
Signature · 1 line
1.7.7.4
#4 lore raw

PATCH 3/4 drm/i915: add drm PARAM to query available cache levels

In reply to Eugeni Dodonov

From: Eugeni Dodonov <[email protected]>

This allows to query available cache levels from libdrm and check for presence of LLC from userspace.

---
 drivers/gpu/drm/i915/i915_dma.c |    6 ++++++
 include/drm/i915_drm.h          |    1 +
 2 files changed, 7 insertions(+), 0 deletions(-)
drivers/gpu/drm/i915/i915_dma.c +6 -0
index a9533c5..5ffbd95 100644
@@ -781,6 +781,12 @@ static int i915_getparam(struct drm_device *dev, void *data,
 	case I915_PARAM_HAS_RELAXED_DELTA:
 		value = 1;
 		break;
+	case I915_PARAM_CACHE_LEVELS:
+		/* Everyone has CACHE_NONE but not everyone has LLC */
+		value = 1 << I915_CACHE_NONE;
+		if (HAS_LLC(dev))
+			value |= 1 << I915_CACHE_LLC;
+		break;
 	default:
 		DRM_DEBUG_DRIVER("Unknown parameter %d\n",
 				 param->param);
include/drm/i915_drm.h +1 -0
index e9f1cf4..7f778f5 100644
@@ -291,6 +291,7 @@ typedef struct drm_i915_irq_wait {
 #define I915_PARAM_HAS_COHERENT_RINGS	 13
 #define I915_PARAM_HAS_EXEC_CONSTANTS	 14
 #define I915_PARAM_HAS_RELAXED_DELTA	 15
+#define I915_PARAM_CACHE_LEVELS	 16
 
 typedef struct drm_i915_getparam {
 	int param;
Signature · 1 line
1.7.7.4
#5 lore raw

PATCH 4/4 drm/i915: add support to get/set cache type for bo

In reply to Eugeni Dodonov

From: Eugeni Dodonov <[email protected]>

Allow userspace to discover current cache level for a bo and set a specific cache level if necessary.

The patch is mostly based on the original patch from Ben Widawsky.

---
 drivers/gpu/drm/i915/i915_dma.c  |    2 +
 drivers/gpu/drm/i915/i915_gem.c  |   74 ++++++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/intel_drv.h |    5 +++
 include/drm/i915_drm.h           |   14 +++++++
 4 files changed, 95 insertions(+), 0 deletions(-)
drivers/gpu/drm/i915/i915_dma.c +2 -0
index 5ffbd95..adb8bca 100644
@@ -2301,6 +2301,8 @@ struct drm_ioctl_desc i915_ioctls[] = {
 	DRM_IOCTL_DEF_DRV(I915_GEM_MADVISE, i915_gem_madvise_ioctl, DRM_UNLOCKED),
 	DRM_IOCTL_DEF_DRV(I915_OVERLAY_PUT_IMAGE, intel_overlay_put_image, DRM_MASTER|DRM_CONTROL_ALLOW|DRM_UNLOCKED),
 	DRM_IOCTL_DEF_DRV(I915_OVERLAY_ATTRS, intel_overlay_attrs, DRM_MASTER|DRM_CONTROL_ALLOW|DRM_UNLOCKED),
+	DRM_IOCTL_DEF_DRV(I915_GET_CACHE_TYPE, intel_get_cache_type_ioctl, DRM_UNLOCKED),
+	DRM_IOCTL_DEF_DRV(I915_SET_CACHE_TYPE, intel_set_cache_type_ioctl, DRM_UNLOCKED),
 };
 
 int i915_max_ioctl = DRM_ARRAY_SIZE(i915_ioctls);
drivers/gpu/drm/i915/i915_gem.c +74 -0

77 lines not on this page · raw message

drivers/gpu/drm/i915/intel_drv.h +5 -0
index bd9a604..2e0841a 100644
@@ -376,6 +376,11 @@ extern int intel_overlay_put_image(struct drm_device *dev, void *data,
 extern int intel_overlay_attrs(struct drm_device *dev, void *data,
 			       struct drm_file *file_priv);
 
+extern int intel_get_cache_type_ioctl(struct drm_device *dev, void *data,
+			       struct drm_file *file_priv);
+extern int intel_set_cache_type_ioctl(struct drm_device *dev, void *data,
+			       struct drm_file *file_priv);
+
 extern void intel_fb_output_poll_changed(struct drm_device *dev);
 extern void intel_fb_restore_mode(struct drm_device *dev);
 
include/drm/i915_drm.h +14 -0
index 7f778f5..de6bb61 100644
@@ -198,6 +198,8 @@ typedef struct _drm_i915_sarea {
 #define DRM_I915_OVERLAY_PUT_IMAGE	0x27
 #define DRM_I915_OVERLAY_ATTRS	0x28
 #define DRM_I915_GEM_EXECBUFFER2	0x29
+#define DRM_I915_GET_CACHE_TYPE		0x2a
+#define DRM_I915_SET_CACHE_TYPE		0x2b
 
 #define DRM_IOCTL_I915_INIT		DRM_IOW( DRM_COMMAND_BASE + DRM_I915_INIT, drm_i915_init_t)
 #define DRM_IOCTL_I915_FLUSH		DRM_IO ( DRM_COMMAND_BASE + DRM_I915_FLUSH)
@@ -239,6 +241,8 @@ typedef struct _drm_i915_sarea {
 #define DRM_IOCTL_I915_GEM_MADVISE	DRM_IOWR(DRM_COMMAND_BASE + DRM_I915_GEM_MADVISE, struct drm_i915_gem_madvise)
 #define DRM_IOCTL_I915_OVERLAY_PUT_IMAGE	DRM_IOW(DRM_COMMAND_BASE + DRM_I915_OVERLAY_PUT_IMAGE, struct drm_intel_overlay_put_image)
 #define DRM_IOCTL_I915_OVERLAY_ATTRS	DRM_IOWR(DRM_COMMAND_BASE + DRM_I915_OVERLAY_ATTRS, struct drm_intel_overlay_attrs)
+#define DRM_IOCTL_I915_GET_CACHE_TYPE	DRM_IOWR(DRM_COMMAND_BASE + DRM_I915_GET_CACHE_TYPE, struct drm_i915_gem_get_cache_type)
+#define DRM_IOCTL_I915_SET_CACHE_TYPE	DRM_IOWR(DRM_COMMAND_BASE + DRM_I915_SET_CACHE_TYPE, struct drm_i915_gem_set_cache_type)
 
 /* Allow drivers to submit batchbuffers directly to hardware, relying
  * on the security mechanisms provided by hardware.
@@ -853,4 +857,14 @@ struct drm_intel_overlay_attrs {
 #define I915_CACHE_LLC				1
 #define I915_CACHE_LLC_MLC			2
 
+struct drm_i915_gem_get_cache_type {
+	__u32 handle;
+	__u32 cache_level;
+};
+
+struct drm_i915_gem_set_cache_type {
+	__u32 handle;
+	__u32 cache_level;
+};
+
 #endif				/* _I915_DRM_H_ */
Signature · 1 line
1.7.7.4
#6 lore raw
In reply to Eugeni Dodonov
Quoted 9 lines · Eugeni Dodonov

With base on comments and feedback on LLC handling from Daniel Vetter, Chris Wilson and Ben Widawsky, I reworked my patch into a more useful approach for detecting presence of different cache levels from userspace.

So, in this context, PATCH 1 adds the very same .has_llc flag and debugfs entry for it. PATCH 2 does the initial job of preparing to export cache levels to userspace, which can be queried via the DRM interface by means of PATCH 3. And finally, PATCH 4 allows to get and set a specific cache level for each bo.

So, I haven't come up with a usecase, and I don't think I've heard of
one: why do we want the cache level setting ioctl?

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx