Skip to contentSkip to conversation

drm/i915: Context aware user agnostic EU/Slice/Sub-slice control within kernel

PATCH Superseded Ankit Navik 3 participants 6 messages Mar 16, 2020

Subsystems (2)

v7 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

v7 cover drm/i915: Context aware user agnostic EU/Slice/Sub-slice control within kernel

This patch sets improves GPU power consumption on Linux kernel based OS such as Chromium OS, Ubuntu, etc. Following are the power savings.

Power savings on GLK-GT1 Bobba platform running on Chrome OS.

-----------------------------------------------|
App /KPI                | % Power Benefit (mW) |
------------------------|----------------------|
Hangout Call- 20 minute |	1.8%           |
Youtube 4K VPB          |       14.13%         |
WebGL Aquarium          |       13.76%         |
Unity3D                 |       6.78%          |
			|		       |
------------------------|----------------------|
Chrome PLT              | BatteryLife Improves |
			| by ~45 minute        |
-----------------------------------------------|

Power savings on KBL-GT3 running on Android and Ubuntu (Linux). -----------------------------------------------|
App /KPI | % Power Benefit (mW) |

                        |----------------------|
			|  Android |  Ubuntu   |
------------------------|----------|-----------|
3D Mark (Ice storm)     | 2.30%    | N.A.      |
TRex On screen          | 2.49%    | 2.97%     |
Manhattan On screen     | 3.11%    | 4.90%     |
Carchase On Screen	| N.A.     | 5.06%     |
AnTuTu 6.1.4            | 3.42%    | N.A.      |
SynMark2		| N.A.     | 1.7%      |
-----------------------------------------------|

We have also observed GPU core residencies improves by 1.035%.

Technical Insights of the patch:
Current GPU configuration code for i915 does not allow us to change EU/Slice/Sub-slice configuration dynamically. Its done only once while context is created.

While particular graphics application is running, if we examine the command requests from user space, we observe that command density is not consistent. It means there is scope to change the graphics configuration dynamically even while context is running actively. This patch series proposes the solution to find the active pending load for all active context at given time and based on that, dynamically perform graphics configuration for each context.

The feature can be enabled using sysfs. we examine pending commands for a context in the queue, essentially, we intercept them before they are executed by GPU and we update context with required number of EUs.

For the prior one, empirical data to achieve best performance in least power was considered. For the later one, we roughly categorized number of EUs logically based on platform. Now we compare number of pending commands with a particular threshold and then set number of EUs accordingly with update context. That threshold is also based on experiments & findings. If GPU is able to catch up with CPU, typically there are no pending commands, the EU config would remain unchanged there. In case there are more pending commands we reprogram context with higher number of EUs.

Ankit Navik (3):
drm/i915: Get active pending request for given context drm/i915: set optimum eu/slice/sub-slice configuration based on load
type
drm/i915: Predictive governor to control slice/subslice/eu

 drivers/gpu/drm/i915/gem/i915_gem_context.c       |  4 ++
 drivers/gpu/drm/i915/gem/i915_gem_context_types.h | 37 +++++++++++
 drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c    |  2 +
 drivers/gpu/drm/i915/gt/intel_context_sseu.c      |  2 +
 drivers/gpu/drm/i915/gt/intel_context_types.h     |  2 +
 drivers/gpu/drm/i915/gt/intel_lrc.c               | 79 ++++++++++++++++++++++-
 drivers/gpu/drm/i915/i915_drv.h                   |  5 ++
 drivers/gpu/drm/i915/i915_sysfs.c                 | 32 +++++++++
 drivers/gpu/drm/i915/intel_device_info.c          | 55 +++++++++++++++-
 9 files changed, 214 insertions(+), 4 deletions(-)
Signature · 6 lines
2.7.4

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#1 lore raw

PATCH v7 1/3 drm/i915: Get active pending request for given context

This patch gives us the active pending request count which is yet to be submitted to the GPU.

V2:
* Change 64-bit to atomic for request count. (Tvrtko Ursulin)

V3:
* Remove mutex for request count.
* Rebase.
* Fixes hitting underflow for predictive request. (Tvrtko Ursulin)

V4:
* Rebase.

V5:
* Rebase.

V6:
* Rebase.

V7:
* Rebase.
* Add GEM_BUG_ON for req_cnt.

---
 drivers/gpu/drm/i915/gem/i915_gem_context.c       | 1 +
 drivers/gpu/drm/i915/gem/i915_gem_context_types.h | 5 +++++
 drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c    | 2 ++
 drivers/gpu/drm/i915/gt/intel_lrc.c               | 9 +++++++++
 4 files changed, 17 insertions(+)
drivers/gpu/drm/i915/gem/i915_gem_context.c +1 -0
index 026999b34abd..d0ff999429ff 100644
@@ -879,6 +879,7 @@ i915_gem_create_context(struct drm_i915_private *i915, unsigned int flags)
 	}
 
 	trace_i915_context_create(ctx);
+	atomic_set(&ctx->req_cnt, 0);
 
 	return ctx;
 }
drivers/gpu/drm/i915/gem/i915_gem_context_types.h +5 -0
index 28760bd03265..a9ba13f8865e 100644
@@ -171,6 +171,11 @@ struct i915_gem_context {
 	 */
 	struct radix_tree_root handles_vma;
 
+	/** req_cnt: tracks the pending commands, based on which we decide to
+	 * go for low/medium/high load configuration of the GPU.
+	 */
+	atomic_t req_cnt;
+
 	/**
 	 * @name: arbitrary name, used for user debug
 	 *
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +2 -0
index d3f4f28e9468..f90c968f95cd 100644
@@ -2565,6 +2565,8 @@ i915_gem_do_execbuffer(struct drm_device *dev,
 	if (batch->private)
 		intel_engine_pool_mark_active(batch->private, eb.request);
 
+	atomic_inc(&eb.gem_context->req_cnt);
+
 	trace_i915_request_queue(eb.request, eb.batch_flags);
 	err = eb_submit(&eb, batch);
 err_request:
drivers/gpu/drm/i915/gt/intel_lrc.c +9 -0
index 112531b29f59..ccfebebb0071 100644
@@ -2143,6 +2143,8 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
 			}
 
 			if (__i915_request_submit(rq)) {
+				struct i915_gem_context *ctx;
+
 				if (!merge) {
 					*port = execlists_schedule_in(last, port - execlists->pending);
 					port++;
@@ -2158,6 +2160,13 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
 
 				submit = true;
 				last = rq;
+
+				ctx = rcu_dereference_protected(
+					rq->context->gem_context, true);
+
+				GEM_BUG_ON(atomic_read(&ctx->req_cnt));
+				if (atomic_read(&ctx->req_cnt) > 0)
+					atomic_dec(&ctx->req_cnt);
 			}
 		}
 
Signature · 6 lines
2.7.4

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#2 lore raw

Re: [Intel-gfx] [PATCH v7 1/3] drm/i915: Get active pending request for given context

Quoting Ankit Navik (2020-03-16 13:29:49)

Quoted 88 lines

This patch gives us the active pending request count which is yet to be submitted to the GPU.

V2:
* Change 64-bit to atomic for request count. (Tvrtko Ursulin)

V3:
* Remove mutex for request count.
* Rebase.
* Fixes hitting underflow for predictive request. (Tvrtko Ursulin)

V4:
* Rebase.

V5:
* Rebase.

V6:
* Rebase.

V7:
* Rebase.
* Add GEM_BUG_ON for req_cnt.

---
 drivers/gpu/drm/i915/gem/i915_gem_context.c       | 1 +
 drivers/gpu/drm/i915/gem/i915_gem_context_types.h | 5 +++++
 drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c    | 2 ++
 drivers/gpu/drm/i915/gt/intel_lrc.c               | 9 +++++++++
 4 files changed, 17 insertions(+)
drivers/gpu/drm/i915/gem/i915_gem_context.c +1 -0
index 026999b34abd..d0ff999429ff 100644
@@ -879,6 +879,7 @@ i915_gem_create_context(struct drm_i915_private *i915, unsigned int flags)
        }
 
        trace_i915_context_create(ctx);
+       atomic_set(&ctx->req_cnt, 0);
 
        return ctx;
 }
drivers/gpu/drm/i915/gem/i915_gem_context_types.h +5 -0
index 28760bd03265..a9ba13f8865e 100644
@@ -171,6 +171,11 @@ struct i915_gem_context {
         */
        struct radix_tree_root handles_vma;
 
+       /** req_cnt: tracks the pending commands, based on which we decide to
+        * go for low/medium/high load configuration of the GPU.
+        */
+       atomic_t req_cnt;
+
        /**
         * @name: arbitrary name, used for user debug
         *
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +2 -0
index d3f4f28e9468..f90c968f95cd 100644
@@ -2565,6 +2565,8 @@ i915_gem_do_execbuffer(struct drm_device *dev,
        if (batch->private)
                intel_engine_pool_mark_active(batch->private, eb.request);
 
+       atomic_inc(&eb.gem_context->req_cnt);
+
        trace_i915_request_queue(eb.request, eb.batch_flags);
        err = eb_submit(&eb, batch);
 err_request:
drivers/gpu/drm/i915/gt/intel_lrc.c +2 -0
index 112531b29f59..ccfebebb0071 100644
@@ -2143,6 +2143,8 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
                        }
 
                        if (__i915_request_submit(rq)) {
+                               struct i915_gem_context *ctx;
+
                                if (!merge) {
                                        *port = execlists_schedule_in(last, port - execlists->pending);
                                        port++;

drivers/gpu/drm/i915/gt/intel_lrc.c static void execlists_dequeue(struct intel_engine_cs *engine) · line 2169

Quoted 10 lines
drivers/gpu/drm/i915/gt/intel_lrc.c +7 -0
@@ -2158,6 +2160,13 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
 
                                submit = true;
                                last = rq;
+
+                               ctx = rcu_dereference_protected(
+                                       rq->context->gem_context, true);
+
+                               GEM_BUG_ON(atomic_read(&ctx->req_cnt));
+                               if (atomic_read(&ctx->req_cnt) > 0)
+                                       atomic_dec(&ctx->req_cnt);

This is wrong on so many levels. The GEM context is an opaque pointer here, and often not available. The rcu_dereference_protected is woeful. There is not even a 1:1 relationship between execbuf and requests -- you should have recognised that the moment you "handled" the bug.

Please do look at the other metrics we have time and time again pointed you towards.
-Chris
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#3 lore raw

PATCH v7 2/3 drm/i915: set optimum eu/slice/sub-slice configuration based on load type

In reply to Ankit Navik

This patch will select optimum eu/slice/sub-slice configuration based on type of load (low, medium, high) as input.
Based on our readings and experiments we have predefined set of optimum configuration for each platform(CHT, KBL). i915_gem_context_set_load_type will select optimum configuration from pre-defined optimum configuration table(opt_config).

It also introduce flag update_render_config which can set by any governor.

v2:
* Move static optimum_config to device init time.
* Rename function to appropriate name, fix data types and patch ordering.
* Rename prev_load_type to pending_load_type. (Tvrtko Ursulin)

v3:
* Add safe guard check in i915_gem_context_set_load_type.
* Rename struct from optimum_config to i915_sseu_optimum_config to avoid namespace clashes.
* Reduces memcpy for space efficient.
* Rebase.
* Improved commit message. (Tvrtko Ursulin)

v4:
* Move optimum config table to file scope. (Tvrtko Ursulin)

v5:
* Adds optimal table of slice/sub-slice/EU for Gen 9 GT1.
* Rebase.

v6:
* Rebase.
* Fix warnings.

v7:
* Fix return conditions.
* Remove i915_gem_context_set_load_type and move logic to __execlists_update_reg_state. (Tvrtko Ursulin)

---
 drivers/gpu/drm/i915/gem/i915_gem_context.c       |  3 +
 drivers/gpu/drm/i915/gem/i915_gem_context_types.h | 32 +++++++++++
 drivers/gpu/drm/i915/gt/intel_context_sseu.c      |  2 +
 drivers/gpu/drm/i915/gt/intel_context_types.h     |  2 +
 drivers/gpu/drm/i915/gt/intel_lrc.c               | 70 ++++++++++++++++++++++-
 drivers/gpu/drm/i915/i915_drv.h                   |  5 ++
 drivers/gpu/drm/i915/intel_device_info.c          | 55 +++++++++++++++++-
 7 files changed, 165 insertions(+), 4 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_context.c +3 -0
index d0ff999429ff..3aad45b0ba5a 100644
@@ -880,6 +880,9 @@ i915_gem_create_context(struct drm_i915_private *i915, unsigned int flags)
 
 	trace_i915_context_create(ctx);
 	atomic_set(&ctx->req_cnt, 0);
+	ctx->slice_cnt = hweight8(RUNTIME_INFO(i915)->sseu.slice_mask);
+	ctx->subslice_cnt = hweight8(RUNTIME_INFO(i915)->sseu.subslice_mask[0]);
+	ctx->eu_cnt = RUNTIME_INFO(i915)->sseu.eu_per_subslice;
 
 	return ctx;
 }
drivers/gpu/drm/i915/gem/i915_gem_context_types.h +32 -0
index a9ba13f8865e..1af1acd73794 100644
@@ -46,6 +46,19 @@ struct i915_gem_engines_iter {
 	const struct i915_gem_engines *engines;
 };
 
+enum gem_load_type {
+	LOAD_TYPE_LOW,
+	LOAD_TYPE_MEDIUM,
+	LOAD_TYPE_HIGH,
+	LOAD_TYPE_LAST
+};
+
+struct i915_sseu_optimum_config {
+	u8 slice;
+	u8 subslice;
+	u8 eu;
+};
+
 /**
  * struct i915_gem_context - client state
  *
@@ -155,6 +168,25 @@ struct i915_gem_context {
 	 */
 	atomic_t active_count;
 
+	/** slice_cnt: used to set the # of slices to be enabled. */
+	u8 slice_cnt;
+
+	/** subslice_cnt: used to set the # of subslices to be enabled. */
+	u8 subslice_cnt;
+
+	/** eu_cnt: used to set the # of eu to be enabled. */
+	u8 eu_cnt;
+
+	/** load_type: The designated load_type (high/medium/low) for a given
+	 * number of pending commands in the command queue.
+	 */
+	enum gem_load_type load_type;
+
+	/** pending_load_type: The earlier load type that the GPU was configured
+	 * for (high/medium/low).
+	 */
+	enum gem_load_type pending_load_type;
+
 	/**
 	 * @hang_timestamp: The last time(s) this context caused a GPU hang
 	 */
drivers/gpu/drm/i915/gt/intel_context_sseu.c +2 -0
index 57a30956c922..4f51bfb9690c 100644
@@ -84,6 +84,8 @@ intel_context_reconfigure_sseu(struct intel_context *ce,
 	if (ret)
 		return ret;
 
+	ce->user_sseu = true;
+
 	/* Nothing to do if unmodified. */
 	if (!memcmp(&ce->sseu, &sseu, sizeof(sseu)))
 		goto unlock;
drivers/gpu/drm/i915/gt/intel_context_types.h +2 -0
index 0f3b68b95c56..fd5811110026 100644
@@ -93,6 +93,8 @@ struct intel_context {
 
 	const struct intel_context_ops *ops;
 
+	bool user_sseu;
+
 	/** sseu: Control eu/slice partitioning */
 	struct intel_sseu sseu;
 };
drivers/gpu/drm/i915/gt/intel_lrc.c +68 -2

102 lines not on this page · raw message

drivers/gpu/drm/i915/i915_drv.h +5 -0
index 1f5b9a584f71..304d95aa4974 100644
@@ -926,6 +926,11 @@ struct drm_i915_private {
 	/* protects panel power sequencer state */
 	struct mutex pps_mutex;
 
+	/* optimal slice/subslice/EU configration state */
+	struct i915_sseu_optimum_config *opt_config;
+
+	bool predictive_load_enable;
+
 	unsigned int fsb_freq, mem_freq, is_ddr3;
 	unsigned int skl_preferred_vco_freq;
 	unsigned int max_cdclk_freq;
drivers/gpu/drm/i915/intel_device_info.c +53 -2

71 lines not on this page · raw message

Signature · 6 lines
2.7.4

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#4 lore raw

PATCH v7 3/3 drm/i915: Predictive governor to control slice/subslice/eu

In reply to Ankit Navik

Load classification is used for predictive governor to control eu/slice/subslice based on workloads.

sysfs is provided to enable/disable the feature

V2:
* Fix code style.
* Move predictive_load_timer into a drm_i915_private structure.
* Make generic function to set optimum config. (Tvrtko Ursulin)

V3:
* Rebase.
* Fix race condition for predictive load set.
* Add slack to start hrtimer for more power efficient. (Tvrtko Ursulin)

V4:
* Fix data type and initialization of mutex to protect predictive load state.
* Move predictive timer init to i915_gem_init_early. (Tvrtko Ursulin)
* Move debugfs to kernel parameter.

V5:
* Rebase.
* Remove mutex for pred_timer

V6:
* Rebase.
* Fix warnings.

V7:
* Drop timer and move logic to __execlists_update_reg_state. (Tvrtko Ursulin)
* Remove kernel boot param and make it to sysfs entry. (Jani Nikula)

---
 drivers/gpu/drm/i915/i915_sysfs.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
drivers/gpu/drm/i915/i915_sysfs.c +32 -0
index 45d32ef42787..5d76e4992c8d 100644
@@ -433,12 +433,43 @@ static ssize_t gt_min_freq_mhz_store(struct device *kdev,
 	return ret ?: count;
 }
 
+static ssize_t deu_enable_show(struct device *kdev, struct device_attribute *attr, char *buf)
+{
+	struct drm_i915_private *i915 = kdev_minor_to_i915(kdev);
+
+	return snprintf(buf, PAGE_SIZE, "%u\n", i915->predictive_load_enable);
+}
+
+static ssize_t deu_enable_store(struct device *kdev,
+				struct device_attribute *attr,
+				const char *buf,
+				size_t count)
+{
+	struct drm_i915_private *i915 = kdev_minor_to_i915(kdev);
+	ssize_t ret;
+	u32 val;
+
+	ret = kstrtou32(buf, 0, &val);
+	if (ret)
+		return ret;
+
+	/* Check invalid values */
+	if (val != 0 && val != 1)
+		ret = -EINVAL;
+
+	i915->predictive_load_enable = val;
+
+	return count;
+}
+
 static DEVICE_ATTR_RO(gt_act_freq_mhz);
 static DEVICE_ATTR_RO(gt_cur_freq_mhz);
 static DEVICE_ATTR_RW(gt_boost_freq_mhz);
 static DEVICE_ATTR_RW(gt_max_freq_mhz);
 static DEVICE_ATTR_RW(gt_min_freq_mhz);
 
+static DEVICE_ATTR_RW(deu_enable);
+
 static DEVICE_ATTR_RO(vlv_rpe_freq_mhz);
 
 static ssize_t gt_rp_mhz_show(struct device *kdev, struct device_attribute *attr, char *buf);
@@ -474,6 +505,7 @@ static const struct attribute * const gen6_attrs[] = {
 	&dev_attr_gt_RP0_freq_mhz.attr,
 	&dev_attr_gt_RP1_freq_mhz.attr,
 	&dev_attr_gt_RPn_freq_mhz.attr,
+	&dev_attr_deu_enable.attr,
 	NULL,
 };
 
Signature · 6 lines
2.7.4

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#5 lore raw

[Intel-gfx] ✗ Fi.CI.BUILD: failure for drm/i915: Context aware user agnostic EU/Slice/Sub-slice control within kernel (rev3)

In reply to Ankit Navik
== Series Details ==

== Series Details ==

Series: drm/i915: Context aware user agnostic EU/Slice/Sub-slice control within kernel (rev3)
URL : https://patchwork.freedesktop.org/series/57989/
State : failure

== Summary ==

CALL scripts/checksyscalls.sh
CALL scripts/atomic/check-atomics.sh
DESCEND objtool
CHK include/generated/compile.h
CC [M] drivers/gpu/drm/i915/intel_device_info.o
drivers/gpu/drm/i915/intel_device_info.c: In function ‘intel_device_info_runtime_init’:
drivers/gpu/drm/i915/intel_device_info.c:1061:24: error: assignment discards ‘const’ qualifier from pointer target type [-Werror=discarded-qualifiers]

   dev_priv->opt_config = chv_config;
                        ^
drivers/gpu/drm/i915/intel_device_info.c:1073:25: error: assignment discards ‘const’ qualifier from pointer target type [-Werror=discarded-qualifiers]
    dev_priv->opt_config = glk_gt1_config;
                         ^
drivers/gpu/drm/i915/intel_device_info.c:1078:25: error: assignment discards ‘const’ qualifier from pointer target type [-Werror=discarded-qualifiers]
    dev_priv->opt_config = kbl_gt2_config;
                         ^
drivers/gpu/drm/i915/intel_device_info.c:1083:25: error: assignment discards ‘const’ qualifier from pointer target type [-Werror=discarded-qualifiers]
    dev_priv->opt_config = kbl_gt3_config;
                         ^

cc1: all warnings being treated as errors
scripts/Makefile.build:267: recipe for target 'drivers/gpu/drm/i915/intel_device_info.o' failed make[4]: *** [drivers/gpu/drm/i915/intel_device_info.o] Error 1 scripts/Makefile.build:505: recipe for target 'drivers/gpu/drm/i915' failed make[3]: *** [drivers/gpu/drm/i915] Error 2
scripts/Makefile.build:505: recipe for target 'drivers/gpu/drm' failed
make[2]: *** [drivers/gpu/drm] Error 2
scripts/Makefile.build:505: recipe for target 'drivers/gpu' failed
make[1]: *** [drivers/gpu] Error 2
Makefile:1683: recipe for target 'drivers' failed
make: *** [drivers] Error 2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx