Skip to contentSkip to conversation

timers/migration: Temporarily disable per capacity hierarchies

PATCH Applied Frederic Weisbecker 3 participants 5 messages Jun 9 – Jun 16 High-resolution timers, timer wheel, clockevents

lore
Share & tools
lore

Frederic Weisbecker

author maintainer
#0 lore raw

PATCH 1/1 timers/migration: Temporarily disable per capacity hierarchies

Some workloads with different CPU capacities consume more power with timer migration than before. The recently introduced per capacity hierarchies were supposed to alleviate this problem. However it appears to also regress other types of workloads, especially when plenty of capacities live together in the same machine.

Disable the feature until a reasonable solution is found.

Fixes 098cbaad8e57 ("timers/migration: Split per-capacity hierarchies")
---
 kernel/time/timer_migration.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/time/timer_migration.c +1 -1
index 548d84955f4c..e9d96d96e251 100644
@@ -1473,7 +1473,7 @@ static unsigned int tmigr_get_capacity(int cpu)
 	 * timekeeper must then belong to the same hierarchy as all the nohz_full
 	 * CPUs. Simply turn off capacity awareness when nohz_full is running.
 	 */
-	if (tick_nohz_full_enabled())
+	if (tick_nohz_full_enabled() || !IS_ENABLED(CONFIG_BROKEN))
 		return SCHED_CAPACITY_SCALE;
 	else
 		return arch_scale_cpu_capacity(cpu);
Signature · 1 line
2.53.0
#1 lore raw
The following commit has been merged into the timers/core branch of tip:

The following commit has been merged into the timers/core branch of tip:

Commit-ID:     bb0c250e8e1132723795c1046442ceb01a5ed1b1
Gitweb:        https://git.kernel.org/tip/bb0c250e8e1132723795c1046442ceb01a5ed1b1
Author:        Frederic Weisbecker <[email protected]>
AuthorDate:    Tue, 09 Jun 2026 14:33:56 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Tue, 09 Jun 2026 17:25:54 +02:00

timers/migration: Temporarily disable per capacity hierarchies

Some workloads with different CPU capacities consume more power with timer migration than before. The recently introduced per capacity hierarchies were supposed to alleviate this problem. However it appears to also regress other types of workloads, especially when plenty of capacities live together in the same machine.

Disable the feature until a reasonable solution is found.

---
 kernel/time/timer_migration.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/time/timer_migration.c +1 -1
index 548d849..e9d96d9 100644
@@ -1473,7 +1473,7 @@ static unsigned int tmigr_get_capacity(int cpu)
 	 * timekeeper must then belong to the same hierarchy as all the nohz_full
 	 * CPUs. Simply turn off capacity awareness when nohz_full is running.
 	 */
-	if (tick_nohz_full_enabled())
+	if (tick_nohz_full_enabled() || !IS_ENABLED(CONFIG_BROKEN))
 		return SCHED_CAPACITY_SCALE;
 	else
 		return arch_scale_cpu_capacity(cpu);
Reviewed-by on v1 1/1
#2 lore raw
In reply to Frederic Weisbecker
Quoted 20 lines · Frederic Weisbecker

Some workloads with different CPU capacities consume more power with timer migration than before. The recently introduced per capacity hierarchies were supposed to alleviate this problem. However it appears to also regress other types of workloads, especially when plenty of capacities live together in the same machine.

Disable the feature until a reasonable solution is found.

Fixes 098cbaad8e57 ("timers/migration: Split per-capacity hierarchies")
---
 kernel/time/timer_migration.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/time/timer_migration.c +0 -0
index 548d84955f4c..e9d96d96e251 100644

kernel/time/timer_migration.c static unsigned int tmigr_get_capacity(int cpu) · line 1479

Quoted 8 lines
kernel/time/timer_migration.c +1 -1
@@ -1473,7 +1473,7 @@ static unsigned int tmigr_get_capacity(int cpu)
 	 * timekeeper must then belong to the same hierarchy as all the nohz_full
 	 * CPUs. Simply turn off capacity awareness when nohz_full is running.
 	 */
-	if (tick_nohz_full_enabled())
+	if (tick_nohz_full_enabled() || !IS_ENABLED(CONFIG_BROKEN))
 		return SCHED_CAPACITY_SCALE;
 	else
 		return arch_scale_cpu_capacity(cpu);

FWIW

Thanks for looking into this after the late response!

I have something based on avg_idle which doesn't look unreasonable on first glance, I'll do some more testing and hopefully post it with some number soon!

author maintainer
#3 lore raw
Quoted 31 lines · Christian Loehle
Quoted 20 lines · Frederic Weisbecker

Some workloads with different CPU capacities consume more power with timer migration than before. The recently introduced per capacity hierarchies were supposed to alleviate this problem. However it appears to also regress other types of workloads, especially when plenty of capacities live together in the same machine.

Disable the feature until a reasonable solution is found.

Fixes 098cbaad8e57 ("timers/migration: Split per-capacity hierarchies")
---
 kernel/time/timer_migration.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/time/timer_migration.c +0 -0
index 548d84955f4c..e9d96d96e251 100644

kernel/time/timer_migration.c static unsigned int tmigr_get_capacity(int cpu) · line 1479

Quoted 8 lines
kernel/time/timer_migration.c +1 -1
@@ -1473,7 +1473,7 @@ static unsigned int tmigr_get_capacity(int cpu)
 	 * timekeeper must then belong to the same hierarchy as all the nohz_full
 	 * CPUs. Simply turn off capacity awareness when nohz_full is running.
 	 */
-	if (tick_nohz_full_enabled())
+	if (tick_nohz_full_enabled() || !IS_ENABLED(CONFIG_BROKEN))
 		return SCHED_CAPACITY_SCALE;
 	else
 		return arch_scale_cpu_capacity(cpu);

FWIW

Quoted 5 lines

Thanks for looking into this after the late response!

I have something based on avg_idle which doesn't look unreasonable on first glance, I'll do some more testing and hopefully post it with some number soon!

Here is another thing we can try.

We can build the hierarchy just like we do with NUMA but instead of NUMA nodes, use the capacity (initial idea of Thomas). But connect them through a common root.

This is roughly equivalent to per capacity hierarchies, but all those hierarchies are connected eventually to the root.

And then upon CPU wakup, actively "steal" the migrator duty from higher capacity CPU.

This way lower capacity CPUs have more chances to process global timers and they also have more chances to stay active due to that added work and also ground work tasks woken up by those timers (hopefully locally).

I can try something like that, I'll just need to use atomic64_t for struct tmigr_group::migr_state to store the capacity of the migrator.

Thanks.

Signature · 2 lines
Frederic Weisbecker
SUSE Labs
#4 lore raw
Quoted 50 lines · Frederic Weisbecker
Quoted 31 lines · Christian Loehle
Quoted 20 lines · Frederic Weisbecker

Some workloads with different CPU capacities consume more power with timer migration than before. The recently introduced per capacity hierarchies were supposed to alleviate this problem. However it appears to also regress other types of workloads, especially when plenty of capacities live together in the same machine.

Disable the feature until a reasonable solution is found.

Fixes 098cbaad8e57 ("timers/migration: Split per-capacity hierarchies")
---
 kernel/time/timer_migration.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/time/timer_migration.c +0 -0
index 548d84955f4c..e9d96d96e251 100644

kernel/time/timer_migration.c static unsigned int tmigr_get_capacity(int cpu) · line 1479

Quoted 8 lines
kernel/time/timer_migration.c +1 -1
@@ -1473,7 +1473,7 @@ static unsigned int tmigr_get_capacity(int cpu)
 	 * timekeeper must then belong to the same hierarchy as all the nohz_full
 	 * CPUs. Simply turn off capacity awareness when nohz_full is running.
 	 */
-	if (tick_nohz_full_enabled())
+	if (tick_nohz_full_enabled() || !IS_ENABLED(CONFIG_BROKEN))
 		return SCHED_CAPACITY_SCALE;
 	else
 		return arch_scale_cpu_capacity(cpu);

FWIW

Quoted 5 lines

Thanks for looking into this after the late response!

I have something based on avg_idle which doesn't look unreasonable on first glance, I'll do some more testing and hopefully post it with some number soon!

Here is another thing we can try.

We can build the hierarchy just like we do with NUMA but instead of NUMA nodes, use the capacity (initial idea of Thomas). But connect them through a common root.

This is roughly equivalent to per capacity hierarchies, but all those hierarchies are connected eventually to the root.

And then upon CPU wakup, actively "steal" the migrator duty from higher capacity CPU.

Quoted 9 lines

This way lower capacity CPUs have more chances to process global timers and they also have more chances to stay active due to that added work and also ground work tasks woken up by those timers (hopefully locally).

I can try something like that, I'll just need to use atomic64_t for struct tmigr_group::migr_state to store the capacity of the migrator.

Thanks.

Sounds reasonable to me, happy to give that a spin!