Skip to contentSkip to conversation

sched: restrict iowait boost for boosted task only

RFC PATCH In review Wei Wang 5 participants 10 messages Jan 24, 2020 – Jan 26, 2020

Subsystems (2)
lore
Share & tools
lore

Wei Wang

author
#0 lore raw

RFC PATCH 1/1 sched: restrict iowait boost for boosted task only

Currently iowait doesn't distinguish background/foreground tasks. This may not be a problem in servers but could impact devices that are more sensitive on energy consumption. [1]
In Android world, we have seen many cases where device run to high and inefficient frequency unnecessarily when running some background task which are interacting I/O.

The ultimate goal is to only apply iowait boost on UX related tasks and leave the rest for EAS scheduler for better efficiency.

This patch limits iowait boost for tasks which are associated with boost signal from user land. On Android specifically, those are foreground or top app tasks.

[1] ANDROID discussion:
https://android-review.googlesource.com/c/kernel/common/+/1215695

---
 kernel/sched/fair.c  |  4 +++-
 kernel/sched/sched.h | 12 ++++++++++++
 2 files changed, 15 insertions(+), 1 deletion(-)
kernel/sched/fair.c +3 -1
index ba749f579714..221c0fbcea0e 100644
@@ -5220,8 +5220,10 @@ enqueue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 	 * If in_iowait is set, the code below may not trigger any cpufreq
 	 * utilization updates, so do it here explicitly with the IOWAIT flag
 	 * passed.
+	 * If CONFIG_ENERGY_MODEL and CONFIG_UCLAMP_TASK are both configured,
+	 * only boost for tasks set with non-null min-clamp.
 	 */
-	if (p->in_iowait)
+	if (iowait_boosted(p))
 		cpufreq_update_util(rq, SCHED_CPUFREQ_IOWAIT);
 
 	for_each_sched_entity(se) {
kernel/sched/sched.h +12 -0
index 280a3c735935..a13d49d835ed 100644
@@ -2341,6 +2341,18 @@ static inline unsigned int uclamp_util(struct rq *rq, unsigned int util)
 }
 #endif /* CONFIG_UCLAMP_TASK */
 
+#if defined(CONFIG_ENERGY_MODEL) && defined(CONFIG_UCLAMP_TASK)
+static inline bool iowait_boosted(struct task_struct *p)
+{
+	return p->in_iowait && uclamp_eff_value(p, UCLAMP_MIN) > 0;
+}
+#else /* defined(CONFIG_ENERGY_MODEL) && defined(CONFIG_UCLAMP_TASK) */
+static inline bool iowait_boosted(struct task_struct *p)
+{
+	return p->in_iowait;
+}
+#endif /* defined(CONFIG_ENERGY_MODEL) && defined(CONFIG_UCLAMP_TASK) */
+
 #ifdef arch_scale_freq_capacity
 # ifndef arch_scale_freq_invariant
 #  define arch_scale_freq_invariant()	true
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#1 lore raw
Quoted 44 lines · Wei Wang

Currently iowait doesn't distinguish background/foreground tasks. This may not be a problem in servers but could impact devices that are more sensitive on energy consumption. [1]
In Android world, we have seen many cases where device run to high and inefficient frequency unnecessarily when running some background task which are interacting I/O.

The ultimate goal is to only apply iowait boost on UX related tasks and leave the rest for EAS scheduler for better efficiency.

This patch limits iowait boost for tasks which are associated with boost signal from user land. On Android specifically, those are foreground or top app tasks.

[1] ANDROID discussion:
https://android-review.googlesource.com/c/kernel/common/+/1215695

---
 kernel/sched/fair.c  |  4 +++-
 kernel/sched/sched.h | 12 ++++++++++++
 2 files changed, 15 insertions(+), 1 deletion(-)
kernel/sched/fair.c +3 -1
index ba749f579714..221c0fbcea0e 100644
@@ -5220,8 +5220,10 @@ enqueue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 	 * If in_iowait is set, the code below may not trigger any cpufreq
 	 * utilization updates, so do it here explicitly with the IOWAIT flag
 	 * passed.
+	 * If CONFIG_ENERGY_MODEL and CONFIG_UCLAMP_TASK are both configured,
+	 * only boost for tasks set with non-null min-clamp.
 	 */
-	if (p->in_iowait)
+	if (iowait_boosted(p))
 		cpufreq_update_util(rq, SCHED_CPUFREQ_IOWAIT);
 
 	for_each_sched_entity(se) {
kernel/sched/sched.h +0 -0
index 280a3c735935..a13d49d835ed 100644

kernel/sched/sched.h static inline unsigned int uclamp_util(struct rq *rq, unsigned int util) · line 2344

Quoted 4 lines
kernel/sched/sched.h +1 -0
@@ -2341,6 +2341,18 @@ static inline unsigned int uclamp_util(struct rq *rq, unsigned int util)
 }
 #endif /* CONFIG_UCLAMP_TASK */
 
+#if defined(CONFIG_ENERGY_MODEL) && defined(CONFIG_UCLAMP_TASK)

Why depend on CONFIG_ENERGY_MODEL? Laptops are battery powered and might not have this option enabled. Do we really need energy model here?

kernel/sched/sched.h static inline unsigned int uclamp_util(struct rq *rq, unsigned int util) · line 2347

Quoted 3 lines
kernel/sched/sched.h +3 -0
+static inline bool iowait_boosted(struct task_struct *p)
+{
+	return p->in_iowait && uclamp_eff_value(p, UCLAMP_MIN) > 0;

I think this is overloading the usage of util clamp. You're basically using cpu.uclamp.min to temporarily switch iowait boost on/off.

Isn't it better to add a new cgroup attribute to toggle this feature?

The problem does seem generic enough and could benefit other battery-powered devices outside of the Android world. I don't think the dependency on uclamp && energy model are necessary to solve this.

Thanks

--
Qais Yousef

Quoted 14 lines
kernel/sched/sched.h +8 -0
+}
+#else /* defined(CONFIG_ENERGY_MODEL) && defined(CONFIG_UCLAMP_TASK) */
+static inline bool iowait_boosted(struct task_struct *p)
+{
+	return p->in_iowait;
+}
+#endif /* defined(CONFIG_ENERGY_MODEL) && defined(CONFIG_UCLAMP_TASK) */
+
 #ifdef arch_scale_freq_capacity
 # ifndef arch_scale_freq_invariant
 #  define arch_scale_freq_invariant()	true
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#2 lore raw
Quoted 24 lines · Qais Yousef
Quoted 24 lines
kernel/sched/fair.c +3 -1
index ba749f579714..221c0fbcea0e 100644
@@ -5220,8 +5220,10 @@ enqueue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 	 * If in_iowait is set, the code below may not trigger any cpufreq
 	 * utilization updates, so do it here explicitly with the IOWAIT flag
 	 * passed.
+	 * If CONFIG_ENERGY_MODEL and CONFIG_UCLAMP_TASK are both configured,
+	 * only boost for tasks set with non-null min-clamp.
 	 */
-	if (p->in_iowait)
+	if (iowait_boosted(p))
 		cpufreq_update_util(rq, SCHED_CPUFREQ_IOWAIT);
 
 	for_each_sched_entity(se) {
kernel/sched/sched.h +0 -0
index 280a3c735935..a13d49d835ed 100644
@@ -2341,6 +2341,18 @@ static inline unsigned int uclamp_util(struct rq *rq, unsigned int util)
 }
 #endif /* CONFIG_UCLAMP_TASK */
 
Quoted 4 lines

kernel/sched/sched.h static inline unsigned int uclamp_util(struct rq *rq, unsigned int util) · line 2344

Quoted 1 line
kernel/sched/sched.h +1 -0
+#if defined(CONFIG_ENERGY_MODEL) && defined(CONFIG_UCLAMP_TASK)

Why depend on CONFIG_ENERGY_MODEL? Laptops are battery powered and might not have this option enabled. Do we really need energy model here?

+1

Quoted 7 lines

kernel/sched/sched.h static inline unsigned int uclamp_util(struct rq *rq, unsigned int util) · line 2347

Quoted 3 lines
kernel/sched/sched.h +3 -0
+static inline bool iowait_boosted(struct task_struct *p)
+{
+	return p->in_iowait && uclamp_eff_value(p, UCLAMP_MIN) > 0;

I think this is overloading the usage of util clamp. You're basically using cpu.uclamp.min to temporarily switch iowait boost on/off.

Quoted 5 lines

Isn't it better to add a new cgroup attribute to toggle this feature?

The problem does seem generic enough and could benefit other battery-powered devices outside of the Android world. I don't think the dependency on uclamp && energy model are necessary to solve this.

I think using uclamp is not a bad idea here, but perhaps we could do things differently. As of today the iowait boost escapes the clamping mechanism, so one option would be to change that. That would let us set a low max clamp in the 'background' cgroup, which in turns would limit the frequency request for those tasks even if they're IO-intensive.

That'll have to be done at the RQ level, but figuring out what's the current max clamp on the rq should be doable from sugov I think.

Wei: would that work for your use case ?

Also, the iowait boost really should be per-task and not per-cpu, so it can be taken into account during wake-up balance on big.LITTLE. That might also help on SMP if a task doing a lot of IO migrates, as the boost would migrate with it. But that's perhaps for later ...

Thanks,
Quentin

#3 lore raw
Quoted 13 lines · Quentin Perret
Quoted 7 lines

kernel/sched/sched.h static inline unsigned int uclamp_util(struct rq *rq, unsigned int util) · line 2347

Quoted 3 lines
kernel/sched/sched.h +3 -0
+static inline bool iowait_boosted(struct task_struct *p)
+{
+	return p->in_iowait && uclamp_eff_value(p, UCLAMP_MIN) > 0;

I think this is overloading the usage of util clamp. You're basically using cpu.uclamp.min to temporarily switch iowait boost on/off.

Quoted 5 lines

Isn't it better to add a new cgroup attribute to toggle this feature?

The problem does seem generic enough and could benefit other battery-powered devices outside of the Android world. I don't think the dependency on uclamp && energy model are necessary to solve this.

Quoted 6 lines

I think using uclamp is not a bad idea here, but perhaps we could do things differently. As of today the iowait boost escapes the clamping mechanism, so one option would be to change that. That would let us set a low max clamp in the 'background' cgroup, which in turns would limit the frequency request for those tasks even if they're IO-intensive.

So I'm pretty sure we *do* want tasks with the default clamps to get iowait boost'd. What we don't want are background tasks driving up the frequency, and that should be via uclamp.max (as Quentin is suggesting) rather than uclamp.min (as is suggested in the patch).

Now, whether that is overloading the usage of uclamp... I'm not sure. One of the argument for uclamp was actually frequency selection, so if we just make iowait boost respect that, IOW not boost further than uclamp.max (which is a bit better than a simple on/off switch), that wouldn't be too crazy I think.

Quoted 13 lines

That'll have to be done at the RQ level, but figuring out what's the current max clamp on the rq should be doable from sugov I think.

Wei: would that work for your use case ?

Also, the iowait boost really should be per-task and not per-cpu, so it can be taken into account during wake-up balance on big.LITTLE. That might also help on SMP if a task doing a lot of IO migrates, as the boost would migrate with it. But that's perhaps for later ...

Thanks,
Quentin

#4 lore raw
Quoted 26 lines · Valentin Schneider
Quoted 13 lines · Quentin Perret
Quoted 7 lines

kernel/sched/sched.h static inline unsigned int uclamp_util(struct rq *rq, unsigned int util) · line 2347

Quoted 3 lines
kernel/sched/sched.h +3 -0
+static inline bool iowait_boosted(struct task_struct *p)
+{
+	return p->in_iowait && uclamp_eff_value(p, UCLAMP_MIN) > 0;

I think this is overloading the usage of util clamp. You're basically using cpu.uclamp.min to temporarily switch iowait boost on/off.

Quoted 5 lines

Isn't it better to add a new cgroup attribute to toggle this feature?

The problem does seem generic enough and could benefit other battery-powered devices outside of the Android world. I don't think the dependency on uclamp && energy model are necessary to solve this.

Quoted 6 lines

I think using uclamp is not a bad idea here, but perhaps we could do things differently. As of today the iowait boost escapes the clamping mechanism, so one option would be to change that. That would let us set a low max clamp in the 'background' cgroup, which in turns would limit the frequency request for those tasks even if they're IO-intensive.

So I'm pretty sure we *do* want tasks with the default clamps to get iowait boost'd. What we don't want are background tasks driving up the frequency, and that should be via uclamp.max (as Quentin is suggesting) rather than uclamp.min (as is suggested in the patch).

Quoted 5 lines

Now, whether that is overloading the usage of uclamp... I'm not sure. One of the argument for uclamp was actually frequency selection, so if we just make iowait boost respect that, IOW not boost further than uclamp.max (which is a bit better than a simple on/off switch), that wouldn't be too crazy I think.

Capping iowait boost value in schedutil based on uclamp makes sense indeed.

What didn't make sense to me is the use of uclamp as a switch to toggle iowait boost on/off.

Cheers

--
Qais Yousef

author
#5 lore raw
Quoted 36 lines · Qais Yousef
Quoted 26 lines · Valentin Schneider
Quoted 13 lines · Quentin Perret
Quoted 7 lines

kernel/sched/sched.h static inline unsigned int uclamp_util(struct rq *rq, unsigned int util) · line 2347

Quoted 3 lines
kernel/sched/sched.h +3 -0
+static inline bool iowait_boosted(struct task_struct *p)
+{
+ return p->in_iowait && uclamp_eff_value(p, UCLAMP_MIN) > 0;

I think this is overloading the usage of util clamp. You're basically using cpu.uclamp.min to temporarily switch iowait boost on/off.

Quoted 5 lines

Isn't it better to add a new cgroup attribute to toggle this feature?

The problem does seem generic enough and could benefit other battery-powered devices outside of the Android world. I don't think the dependency on uclamp && energy model are necessary to solve this.

Quoted 6 lines

I think using uclamp is not a bad idea here, but perhaps we could do things differently. As of today the iowait boost escapes the clamping mechanism, so one option would be to change that. That would let us set a low max clamp in the 'background' cgroup, which in turns would limit the frequency request for those tasks even if they're IO-intensive.

So I'm pretty sure we *do* want tasks with the default clamps to get iowait boost'd. What we don't want are background tasks driving up the frequency, and that should be via uclamp.max (as Quentin is suggesting) rather than uclamp.min (as is suggested in the patch).

Quoted 5 lines

Now, whether that is overloading the usage of uclamp... I'm not sure. One of the argument for uclamp was actually frequency selection, so if we just make iowait boost respect that, IOW not boost further than uclamp.max (which is a bit better than a simple on/off switch), that wouldn't be too crazy I think.

Capping iowait boost value in schedutil based on uclamp makes sense indeed.

Quoted 7 lines

What didn't make sense to me is the use of uclamp as a switch to toggle iowait boost on/off.

Cheers

--
Qais Yousef

Sounds like we all agree on adding a new toggle, so will move forward with that then.
For capping iowait boost, it should be a seperate patch. I am not sure if we want to apply what's the current max clamp on the rq but I do see the per-task iowait boost makes sense.

#6 lore raw
Quoted 11 lines · Wei Wang
Quoted 11 lines
Quoted 10 lines

So I'm pretty sure we *do* want tasks with the default clamps to get iowait boost'd. What we don't want are background tasks driving up the frequency, and that should be via uclamp.max (as Quentin is suggesting) rather than uclamp.min (as is suggested in the patch).

Now, whether that is overloading the usage of uclamp... I'm not sure. One of the argument for uclamp was actually frequency selection, so if we just make iowait boost respect that, IOW not boost further than uclamp.max (which is a bit better than a simple on/off switch), that wouldn't be too crazy I think.

Quoted 7 lines
Quoted 4 lines

Capping iowait boost value in schedutil based on uclamp makes sense indeed.

What didn't make sense to me is the use of uclamp as a switch to toggle iowait boost on/off.

Sounds like we all agree on adding a new toggle, so will move forward with that then.

Looking more closely at iowait boost, it's not actually a generic cpufreq attribute. Only schedutil and intel_pstate have it. Other governors might implement something similar but under a different name.

So I'm not sure how easy it'd be to implement a generic toggle for something that probably should be considered an implementation detail of a governor and userspace shouldn't care much about.

Of course, the maintainers might have a different opinion. So don't let mine discourage you from pursuing this further! :-)

Quoted 3 lines

For capping iowait boost, it should be a seperate patch. I am not sure if we want to apply what's the current max clamp on the rq but I do see the per-task iowait boost makes sense.

It is true the 2 patches are orthogonal, but if you already cap the max frequencies the background task can use, by ensuring the iowait_boost in schedutil respects the uclamp restrictions then this should solve your problem too, no?

The patch below only compile tested.

background/cpu.uclamp.max = 200 # Cap background tasks max frequencies

---

kernel/sched/cpufreq_schedutil.c +2 -1
index 9b8916fd00a2..a76c02eecdaf 100644
@@ -421,7 +421,8 @@ static unsigned long sugov_iowait_apply(struct sugov_cpu *sg_cpu, u64 time,
         * into the same scale so we can compare.
         */
        boost = (sg_cpu->iowait_boost * max) >> SCHED_CAPACITY_SHIFT;
-       return max(boost, util);
+       boost = max(boost, util);
+       return uclamp_util_with(cpu_rq(sg_cpu->cpu), boost, NULL);
 }
 
 #ifdef CONFIG_NO_HZ_COMMON

--
Qais Yousef

#7 lore raw
Quoted 5 lines · Qais Yousef
kernel/sched/cpufreq_schedutil.c +0 -0
index 9b8916fd00a2..a76c02eecdaf 100644

kernel/sched/cpufreq_schedutil.c static unsigned long sugov_iowait_apply(struct sugov_cpu *sg_cpu, u64 time, · line 428

Quoted 9 lines
kernel/sched/cpufreq_schedutil.c +2 -1
@@ -421,7 +421,8 @@ static unsigned long sugov_iowait_apply(struct sugov_cpu *sg_cpu, u64 time,
         * into the same scale so we can compare.
         */
        boost = (sg_cpu->iowait_boost * max) >> SCHED_CAPACITY_SHIFT;
-       return max(boost, util);
+       boost = max(boost, util);
+       return uclamp_util_with(cpu_rq(sg_cpu->cpu), boost, NULL);
 }
 
 #ifdef CONFIG_NO_HZ_COMMON

Forgot to mention this will work if the background task is the only task running on this CPU. Like Quentin already pointed out, the iowait_boost might need more massaging to allow finer per task control.

--
Qais Yousef

#8 lore raw
In reply to Qais Yousef
Quoted 25 lines · Qais Yousef
Quoted 11 lines · Wei Wang
Quoted 11 lines
Quoted 10 lines

So I'm pretty sure we *do* want tasks with the default clamps to get iowait boost'd. What we don't want are background tasks driving up the frequency, and that should be via uclamp.max (as Quentin is suggesting) rather than uclamp.min (as is suggested in the patch).

Now, whether that is overloading the usage of uclamp... I'm not sure. One of the argument for uclamp was actually frequency selection, so if we just make iowait boost respect that, IOW not boost further than uclamp.max (which is a bit better than a simple on/off switch), that wouldn't be too crazy I think.

Quoted 7 lines
Quoted 4 lines

Capping iowait boost value in schedutil based on uclamp makes sense indeed.

What didn't make sense to me is the use of uclamp as a switch to toggle iowait boost on/off.

Sounds like we all agree on adding a new toggle, so will move forward with that then.

Looking more closely at iowait boost, it's not actually a generic cpufreq attribute. Only schedutil and intel_pstate have it. Other governors might implement something similar but under a different name.

Quoted 7 lines

So I'm not sure how easy it'd be to implement a generic toggle for something that probably should be considered an implementation detail of a governor and userspace shouldn't care much about.

Of course, the maintainers might have a different opinion. So don't let mine discourage you from pursuing this further! :-)

Indeed, that's why I was hesitate to add the toggle and wanted to bring this up for Android common kernel first. :-)

Quoted 9 lines
Quoted 3 lines

For capping iowait boost, it should be a separate patch. I am not sure if we want to apply what's the current max clamp on the rq but I do see the per-task iowait boost makes sense.

It is true the 2 patches are orthogonal, but if you already cap the max frequencies the background task can use, by ensuring the iowait_boost in schedutil respects the uclamp restrictions then this should solve your problem too, no?

We haven't tried on 5.4, and there is no uclamp policy placed for background yet. Also I am not sure it is beneficial to do uclamp restriction on those kernel threads (e.g. is f2fs's gc), but that is an interesting experiment on power balance. Also for applying at rq lever, what if there are foreground tasks (and it could be the case sometimes).

Quoted 24 lines

The patch below only compile tested.

background/cpu.uclamp.max = 200 # Cap background tasks max frequencies

---

kernel/sched/cpufreq_schedutil.c +2 -2
index 9b8916fd00a2..a76c02eecdaf 100644
@@ -421,7 +421,8 @@ static unsigned long sugov_iowait_apply(struct sugov_cpu *sg_cpu, u64 time,
         * into the same scale so we can compare.
         */
        boost = (sg_cpu->iowait_boost * max) >> SCHED_CAPACITY_SHIFT;
-       return max(boost, util);
+       boost = max(boost, util);
+       return uclamp_util_with(cpu_rq(sg_cpu->cpu), boost, NULL);
 }
 
 #ifdef CONFIG_NO_HZ_COMMON
 
--

Qais Yousef

author
#9 lore raw
In reply to Valentin Schneider
Quoted 15 lines · Valentin Schneider
Quoted 13 lines · Quentin Perret
Quoted 12 lines

kernel/sched/sched.h static inline unsigned int uclamp_util(struct rq *rq, unsigned int util) · line 2347

Quoted 3 lines
kernel/sched/sched.h +3 -0
+static inline bool iowait_boosted(struct task_struct *p)
+{
+   return p->in_iowait && uclamp_eff_value(p, UCLAMP_MIN) > 0;

I think this is overloading the usage of util clamp. You're basically using cpu.uclamp.min to temporarily switch iowait boost on/off.

Isn't it better to add a new cgroup attribute to toggle this feature?

The problem does seem generic enough and could benefit other battery-powered devices outside of the Android world. I don't think the dependency on uclamp && energy model are necessary to solve this.

Quoted 6 lines
Quoted 6 lines

I think using uclamp is not a bad idea here, but perhaps we could do things differently. As of today the iowait boost escapes the clamping mechanism, so one option would be to change that. That would let us set a low max clamp in the 'background' cgroup, which in turns would limit the frequency request for those tasks even if they're IO-intensive.

Something we see e.g. is f2fs's gc thread, and my thought on this is instead of chasing everything down, it is a lot easier for us to only boost what we know in foreground (and on Android we sort of have that information on hand from framework).
I was hesitant to introduce a new switch ( e.g. Android's older EAS kernel prefer_idle toggle ) but would be happy to do that if someone supports that idea.

Quoted 28 lines

So I'm pretty sure we *do* want tasks with the default clamps to get iowait boost'd. What we don't want are background tasks driving up the frequency, and that should be via uclamp.max (as Quentin is suggesting) rather than uclamp.min (as is suggested in the patch).

Now, whether that is overloading the usage of uclamp... I'm not sure. One of the argument for uclamp was actually frequency selection, so if we just make iowait boost respect that, IOW not boost further than uclamp.max (which is a bit better than a simple on/off switch), that wouldn't be too crazy I think.

Quoted 13 lines

That'll have to be done at the RQ level, but figuring out what's the current max clamp on the rq should be doable from sugov I think.

Wei: would that work for your use case ?

Also, the iowait boost really should be per-task and not per-cpu, so it can be taken into account during wake-up balance on big.LITTLE. That might also help on SMP if a task doing a lot of IO migrates, as the boost would migrate with it. But that's perhaps for later ...

Thanks,
Quentin