Inherit GPU scheduling priority from process nice
Subsystems (2)
cover Inherit GPU scheduling priority from process nice
From: Tvrtko Ursulin <[email protected]>
Current processing landscape seems to be more and more composed of pipelines where computations are done on multiple hardware devices. Furthermore some of the non-CPU devices, like in this case many GPUs supported by the i915 driver, actually support priority based scheduling which is currently rather inaccessible to the user (in terms of being able to control it from the outside).
Quoted 1 line
From these two statements a question arises on how to allow for a simple,
effective and consolidated user experience. In other words why user would not be able to do something like:
$ nice ffmmpeg ...transcode my videos... $ my-favourite-game
And have the nice hint apply to GPU parts of the transcode pipeline as well?
This would in fact follow the approach taken by kernel's block scheduler where ionice is by default inherited from process nice.
This series implements the same idea by inheriting context creator and batch buffer submitted nice value as context nice. To avoid influencing GPU scheduling aware clients this is done only one for contexts where userspace hasn't explicitly specified a non-default scheduling priority
The approach is completely compatible with GuC and drm/scheduler since all support at least low/normal/high priority levels with just the granularity of available control differing. In other words with GuC scheduling there is no difference between nice 5 and 10, both would map to low priority, but the default case of positive or negative nice, versus nice 0, is still correctly propagated to the firmware scheduler.
With the series applied I simulated the scenario of a background GPU task running simultaneously with an interactive client, varying the former's nice value.
Simulating a non-interactive GPU background task was:
vblank_mode=0 nice -n <N> glxgears -geometry 1600x800
Interactive client was simulated with:
gem_wsim -w ~/test.wsim -r 300 -v # (This one is self-capped at ~60fps.)
These were the results on DG1, first with execlists (default):
Background nice | Interactive FPS
-------------------+--------------------
<not running> | 59
0 | 35
10 | 42As we can see running the background load with nice 10 can somewhat help the performance of the interactive/foreground task. (Although to be noted is that without the fair scheduler completed there are possible starvation issues depending on the workload which cannot be fixed by this patch.)
Now results with GuC (although it is not default on DG1):
Background nice | Interactive FPS
-------------------+--------------------
<not running> | 58
0 | 26
10 | 25Unfortunately GuC is not showing any change (25 vs 26 is rounding/run error). But reverse mesurement with background client at nice 0 and foreground at nice -10 does give 40 FPS proving the priority adjustment does work. (Same reverse test gives 46 FPS with execlists). What is happening with GuC here is something to be looked at since it seems normal-vs-low GuC priority time slices differently than normal-vs-high. Normal does not seem to be preferred over low, in this test at least.
v2:
* Moved notifier outside task_rq_lock.
* Some improvements and restructuring on the i915 side of the series.
v3:
* Dropped task nice notifier - inheriting nice on request submit time is good enough.
v4:
* Realisation came that this can be heavily simplified and only one simple patch is enough to achieve the desired behaviour.
* Fixed the priority adjustment location to actually worked after rebase!
* Re-done the benchmarking.
v5:
* I am sending wrong files out yet again (v4), apologies for the spam..
Tvrtko Ursulin (1):
drm/i915: Inherit submitter nice when scheduling requests
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c | 8 ++++++++ 1 file changed, 8 insertions(+)
Signature · 1 line
2.32.0
PATCH 1/1 drm/i915: Inherit submitter nice when scheduling requests
From: Tvrtko Ursulin <[email protected]>
Inherit submitter nice at point of request submission to account for long running processes getting either externally or self re-niced.
This accounts for the current processing landscape where computational pipelines are composed of CPU and GPU parts working in tandem.
Nice value will only apply to requests which originate from user contexts and have default context priority. This is to avoid disturbing any application made choices of low and high (batch processing and latency sensitive compositing). In this case nice value adjusts the effective priority in the narrow band of -19 to +20 around I915_CONTEXT_DEFAULT_PRIORITY.
This means that userspace using the context priority uapi directly has a wider range of possible adjustments (in practice that only applies to execlists platforms - with GuC there are only three priority buckets), but in all cases nice adjustment has the expected effect: positive nice lowering the scheduling priority and negative nice raising it.
--- drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c | 8 ++++++++ 1 file changed, 8 insertions(+)
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +8 -0
@@ -3043,6 +3043,14 @@ static int eb_request_add(struct i915_execbuffer *eb, struct i915_request *rq,/* Check that the context wasn't destroyed before submission */ if (likely (!intel_context_is_closed (eb->context))) { attr = eb->gem_context->sched; +/* +* Inherit process nice when scheduling user contexts but only +* if context has the default priority to avoid touching +* contexts where GEM uapi has been used to explicitly lower +* or elevate it. +*/ +if (attr.priority ==I915_CONTEXT_DEFAULT_PRIORITY ) + attr.priority = -task_nice (current); }else {/* Serialise with context_close via the add_to_timeline */ i915_request_set_error_once (rq, -ENOENT );
Signature · 1 line
2.32.0
Patchwork
CI[Intel-gfx] ✗ Fi.CI.BAT: failure for Inherit GPU scheduling priority from process nice
In reply to Tvrtko Ursulin== Series Details ==
== Series Details ==
Series: Inherit GPU scheduling priority from process nice
URL : https://patchwork.freedesktop.org/series/102348/
State : failure
== Summary ==
CI Bug Log - changes from CI_DRM_11472 -> Patchwork_22817
====================================================
Summary
-------
**FAILURE**
Serious unknown changes coming with Patchwork_22817 absolutely need to be verified manually.
If you think the reported changes have nothing to do with the changes introduced in Patchwork_22817, please notify your bug team to allow them to document this new failure mode, which will reduce false positives in CI.
External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/index.html
Participating hosts (48 -> 36)
------------------------------
Additional (2): fi-bwr-2160 fi-pnv-d510
Missing (14): fi-bdw-samus shard-tglu bat-adls-5 bat-dg1-6 bat-dg1-5 bat-dg2-8 shard-rkl bat-dg2-9 fi-bsw-cyan bat-adlp-6 bat-rpls-1 fi-blb-e6850 bat-jsl-2 bat-jsl-1
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in Patchwork_22817:
### CI changes ###
#### Possible regressions ####
* boot:
- fi-pnv-d510: NOTRUN -> [FAIL][1]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-pnv-d510/boot.html
- fi-bwr-2160: NOTRUN -> [FAIL][2]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-bwr-2160/boot.html
### IGT changes ###
#### Possible regressions ####
* igt@gem_lmem_swapping@basic:
- fi-cfl-8109u: NOTRUN -> [FAIL][3]
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-cfl-8109u/igt@[email protected]
#### Warnings ####
* igt@runner@aborted:
- fi-rkl-11600: [FAIL][4] ([i915#4312]) -> [FAIL][5]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11472/fi-rkl-11600/igt@[email protected]
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-rkl-11600/igt@[email protected]
- fi-rkl-guc: [FAIL][6] ([i915#4312]) -> [FAIL][7]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11472/fi-rkl-guc/igt@[email protected]
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-rkl-guc/igt@[email protected]
- fi-kbl-7567u: [FAIL][8] ([i915#4312]) -> [FAIL][9]
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11472/fi-kbl-7567u/igt@[email protected]
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-kbl-7567u/igt@[email protected]
#### Suppressed ####
The following results come from untrusted machines, tests, or statuses. They do not affect the overall result.
* igt@runner@aborted:
- {fi-jsl-1}: [FAIL][10] ([i915#4312]) -> [FAIL][11]
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11472/fi-jsl-1/igt@[email protected]
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-jsl-1/igt@[email protected]
Known issues
------------
Here are the changes found in Patchwork_22817 that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@core_auth@basic-auth:
- fi-kbl-8809g: NOTRUN -> [SKIP][12] ([fdo#109271]) +1 similar issue
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-kbl-8809g/igt@[email protected]
* igt@fbdev@eof:
- fi-kbl-8809g: NOTRUN -> [INCOMPLETE][13] ([i915#5557])
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-kbl-8809g/igt@[email protected]
* igt@gem_close_race@basic-process:
- fi-ivb-3770: NOTRUN -> [SKIP][14] ([fdo#109271]) +146 similar issues
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-ivb-3770/igt@[email protected]
* igt@kms_flip@basic-flip-vs-dpms:
- fi-kbl-soraka: NOTRUN -> [SKIP][15] ([fdo#109271]) +146 similar issues
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-kbl-soraka/igt@[email protected]
* igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-b:
- fi-cfl-8109u: NOTRUN -> [SKIP][16] ([fdo#109271]) +145 similar issues
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-cfl-8109u/igt@[email protected]
* igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-c:
- fi-cfl-8109u: NOTRUN -> [SKIP][17] ([fdo#109271] / [i915#5341])
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-cfl-8109u/igt@[email protected]
- fi-kbl-soraka: NOTRUN -> [SKIP][18] ([fdo#109271] / [i915#5341])
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-kbl-soraka/igt@[email protected]
- fi-ivb-3770: NOTRUN -> [SKIP][19] ([fdo#109271] / [i915#5341])
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-ivb-3770/igt@[email protected]
#### Warnings ####
* igt@gem_lmem_swapping@basic:
- fi-glk-j4005: [FAIL][20] -> [SKIP][21] ([fdo#109271])
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11472/fi-glk-j4005/igt@[email protected]
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-glk-j4005/igt@[email protected]
* igt@runner@aborted:
- fi-bsw-kefka: [FAIL][22] ([i915#4312]) -> [FAIL][23] ([i915#3690] / [i915#4312])
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11472/fi-bsw-kefka/igt@[email protected]
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-bsw-kefka/igt@[email protected]
- fi-cfl-8700k: [FAIL][24] ([i915#4312] / [i915#5257]) -> [FAIL][25] ([i915#4312])
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11472/fi-cfl-8700k/igt@[email protected]
[25]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-cfl-8700k/igt@[email protected]
- fi-cfl-8109u: [FAIL][26] -> [FAIL][27] ([i915#4312])
[26]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11472/fi-cfl-8109u/igt@[email protected]
[27]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-cfl-8109u/igt@[email protected]
- fi-kbl-8809g: [FAIL][28] -> [FAIL][29] ([i915#2722])
[28]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11472/fi-kbl-8809g/igt@[email protected]
[29]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-kbl-8809g/igt@[email protected]
- fi-kbl-soraka: [FAIL][30] -> [FAIL][31] ([i915#4312])
[30]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11472/fi-kbl-soraka/igt@[email protected]
[31]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-kbl-soraka/igt@[email protected]
- fi-hsw-4770: [FAIL][32] ([i915#4312]) -> [FAIL][33] ([i915#4312] / [i915#5594])
[32]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11472/fi-hsw-4770/igt@[email protected]
[33]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-hsw-4770/igt@[email protected]
- fi-kbl-guc: [FAIL][34] ([i915#4312] / [i915#5257]) -> [FAIL][35] ([i915#4312])
[34]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11472/fi-kbl-guc/igt@[email protected]
[35]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-kbl-guc/igt@[email protected]
- fi-ivb-3770: [FAIL][36] -> [FAIL][37] ([i915#4312])
[36]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11472/fi-ivb-3770/igt@[email protected]
[37]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/fi-ivb-3770/igt@[email protected]
{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).
[fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
[i915#2722]: https://gitlab.freedesktop.org/drm/intel/issues/2722
[i915#3690]: https://gitlab.freedesktop.org/drm/intel/issues/3690
[i915#4312]: https://gitlab.freedesktop.org/drm/intel/issues/4312
[i915#5257]: https://gitlab.freedesktop.org/drm/intel/issues/5257
[i915#5341]: https://gitlab.freedesktop.org/drm/intel/issues/5341
[i915#5557]: https://gitlab.freedesktop.org/drm/intel/issues/5557
[i915#5594]: https://gitlab.freedesktop.org/drm/intel/issues/5594
Build changes
-------------
* Linux: CI_DRM_11472 -> Patchwork_22817
CI-20190529: 20190529
CI_DRM_11472: 85882df13168c5f46b41401b96975de857e3ccac @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_6415: c3b690bd5f7fb1fb7ed786ab0f3b815930a6a55f @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
Patchwork_22817: d2efe297cebfd1122bfb29200ce55b808abf5c43 @ git://anongit.freedesktop.org/gfx-ci/linux
== Linux commits ==
d2efe297cebf drm/i915: Inherit submitter nice when scheduling requests
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_22817/index.html