timers/migration: Temporarily disable per capacity hierarchies
Frederic Weisbecker
maintainerPATCH 1/1 timers/migration: Temporarily disable per capacity hierarchies
Some workloads with different CPU capacities consume more power with timer migration than before. The recently introduced per capacity hierarchies were supposed to alleviate this problem. However it appears to also regress other types of workloads, especially when plenty of capacities live together in the same machine.
Disable the feature until a reasonable solution is found.
--- kernel/time/timer_migration.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/time/timer_migration.c +1 -1
@@ -1473,7 +1473,7 @@ static unsigned int tmigr_get_capacity(int cpu)* timekeeper must then belong to the same hierarchy as all the nohz_full * CPUs. Simply turn off capacity awareness when nohz_full is running. */ -if (tick_nohz_full_enabled ()) +if (tick_nohz_full_enabled () || !IS_ENABLED (CONFIG_BROKEN ))return SCHED_CAPACITY_SCALE ;else return arch_scale_cpu_capacity (cpu);
Signature · 1 line
2.53.0
The following commit has been merged into the timers/core branch of tip:
The following commit has been merged into the timers/core branch of tip:
Commit-ID: bb0c250e8e1132723795c1046442ceb01a5ed1b1 Gitweb: https://git.kernel.org/tip/bb0c250e8e1132723795c1046442ceb01a5ed1b1 Author: Frederic Weisbecker <[email protected]> AuthorDate: Tue, 09 Jun 2026 14:33:56 +02:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Tue, 09 Jun 2026 17:25:54 +02:00
timers/migration: Temporarily disable per capacity hierarchies
Some workloads with different CPU capacities consume more power with timer migration than before. The recently introduced per capacity hierarchies were supposed to alleviate this problem. However it appears to also regress other types of workloads, especially when plenty of capacities live together in the same machine.
Disable the feature until a reasonable solution is found.
--- kernel/time/timer_migration.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/time/timer_migration.c +1 -1
@@ -1473,7 +1473,7 @@ static unsigned int tmigr_get_capacity(int cpu)* timekeeper must then belong to the same hierarchy as all the nohz_full * CPUs. Simply turn off capacity awareness when nohz_full is running. */ -if (tick_nohz_full_enabled ()) +if (tick_nohz_full_enabled () || !IS_ENABLED (CONFIG_BROKEN ))return SCHED_CAPACITY_SCALE ;else return arch_scale_cpu_capacity (cpu);
Quoted 20 lines · Frederic Weisbecker
Some workloads with different CPU capacities consume more power with timer migration than before. The recently introduced per capacity hierarchies were supposed to alleviate this problem. However it appears to also regress other types of workloads, especially when plenty of capacities live together in the same machine.
Disable the feature until a reasonable solution is found.
Fixes 098cbaad8e57 ("timers/migration: Split per-capacity hierarchies")Reported-by Christian Loehle <[email protected]>Signed-off-by Frederic Weisbecker <[email protected]>--- kernel/time/timer_migration.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)kernel/time/timer_migration.c +0
-0
kernel/time/timer_migration.c static unsigned int tmigr_get_capacity(int cpu) · line 1479
Quoted 8 lines
kernel/time/timer_migration.c +1
-1@@ -1473,7 +1473,7 @@ static unsigned int tmigr_get_capacity(int cpu)* timekeeper must then belong to the same hierarchy as all the nohz_full * CPUs. Simply turn off capacity awareness when nohz_full is running. */ -if (tick_nohz_full_enabled ()) +if (tick_nohz_full_enabled () || !IS_ENABLED (CONFIG_BROKEN ))return SCHED_CAPACITY_SCALE ;else return arch_scale_cpu_capacity (cpu);
FWIW
Thanks for looking into this after the late response!
I have something based on avg_idle which doesn't look unreasonable on first glance, I'll do some more testing and hopefully post it with some number soon!
Frederic Weisbecker
maintainerQuoted 31 lines · Christian Loehle
Quoted 20 lines · Frederic Weisbecker
Some workloads with different CPU capacities consume more power with timer migration than before. The recently introduced per capacity hierarchies were supposed to alleviate this problem. However it appears to also regress other types of workloads, especially when plenty of capacities live together in the same machine.
Disable the feature until a reasonable solution is found.
Fixes 098cbaad8e57 ("timers/migration: Split per-capacity hierarchies")Reported-by Christian Loehle <[email protected]>Signed-off-by Frederic Weisbecker <[email protected]>--- kernel/time/timer_migration.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)kernel/time/timer_migration.c +0
-0kernel/time/timer_migration.c static unsigned int tmigr_get_capacity(int cpu) · line 1479
Quoted 8 lines
kernel/time/timer_migration.c +1
-1@@ -1473,7 +1473,7 @@ static unsigned int tmigr_get_capacity(int cpu)* timekeeper must then belong to the same hierarchy as all the nohz_full * CPUs. Simply turn off capacity awareness when nohz_full is running. */ -if (tick_nohz_full_enabled ()) +if (tick_nohz_full_enabled () || !IS_ENABLED (CONFIG_BROKEN ))return SCHED_CAPACITY_SCALE ;else return arch_scale_cpu_capacity (cpu);FWIW
Quoted 5 lines
Reviewed-by Christian Loehle <[email protected]>Thanks for looking into this after the late response!
I have something based on
avg_idlewhich doesn't look unreasonable on first glance, I'll do some more testing and hopefully post it with some number soon!
Here is another thing we can try.
We can build the hierarchy just like we do with NUMA but instead of NUMA nodes, use the capacity (initial idea of Thomas). But connect them through a common root.
This is roughly equivalent to per capacity hierarchies, but all those hierarchies are connected eventually to the root.
And then upon CPU wakup, actively "steal" the migrator duty from higher capacity CPU.
This way lower capacity CPUs have more chances to process global timers and they also have more chances to stay active due to that added work and also ground work tasks woken up by those timers (hopefully locally).
I can try something like that, I'll just need to use atomic64_t for struct tmigr_group::migr_state to store the capacity of the migrator.
Thanks.
Signature · 2 lines
Frederic Weisbecker SUSE Labs
Quoted 50 lines · Frederic Weisbecker
Quoted 31 lines · Christian Loehle
Quoted 20 lines · Frederic Weisbecker
Some workloads with different CPU capacities consume more power with timer migration than before. The recently introduced per capacity hierarchies were supposed to alleviate this problem. However it appears to also regress other types of workloads, especially when plenty of capacities live together in the same machine.
Disable the feature until a reasonable solution is found.
Fixes 098cbaad8e57 ("timers/migration: Split per-capacity hierarchies")Reported-by Christian Loehle <[email protected]>Signed-off-by Frederic Weisbecker <[email protected]>--- kernel/time/timer_migration.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)kernel/time/timer_migration.c +0
-0kernel/time/timer_migration.c static unsigned int tmigr_get_capacity(int cpu) · line 1479
Quoted 8 lines
kernel/time/timer_migration.c +1
-1@@ -1473,7 +1473,7 @@ static unsigned int tmigr_get_capacity(int cpu)* timekeeper must then belong to the same hierarchy as all the nohz_full * CPUs. Simply turn off capacity awareness when nohz_full is running. */ -if (tick_nohz_full_enabled ()) +if (tick_nohz_full_enabled () || !IS_ENABLED (CONFIG_BROKEN ))return SCHED_CAPACITY_SCALE ;else return arch_scale_cpu_capacity (cpu);FWIW
Quoted 5 lines
Reviewed-by Christian Loehle <[email protected]>Thanks for looking into this after the late response!
I have something based on
avg_idlewhich doesn't look unreasonable on first glance, I'll do some more testing and hopefully post it with some number soon!Here is another thing we can try.
We can build the hierarchy just like we do with NUMA but instead of NUMA nodes, use the capacity (initial idea of Thomas). But connect them through a common root.
This is roughly equivalent to per capacity hierarchies, but all those hierarchies are connected eventually to the root.
And then upon CPU wakup, actively "steal" the migrator duty from higher capacity CPU.
Quoted 9 lines
This way lower capacity CPUs have more chances to process global timers and they also have more chances to stay active due to that added work and also ground work tasks woken up by those timers (hopefully locally).
I can try something like that, I'll just need to use
atomic64_tfor structtmigr_group::migr_stateto store the capacity of the migrator.Thanks.
Sounds reasonable to me, happy to give that a spin!