sched/topology: Disable sched_asym_cpucapacity on domain destruction
Subsystems (2)
PATCH 1/1 sched/topology: Disable sched_asym_cpucapacity on domain destruction
While the static key is correctly initialized as being disabled, it will remain forever enabled once turned on. This means that if we start with an asymmetric system and hotplug out enough CPUs to end up with an SMP system, the static key will remain set - which is obviously wrong. We should detect this and turn off things like misfit migration and EAS wakeups.
Having that key enabled should also mandate
per_cpu(sd_asym_cpucapacity, cpu) != NULL
for all CPUs, but this is obviously not true with the above.
On top of that, sched domain rebuilds first lead to attaching the NULL domain to the affected CPUs, which means there will be a window where the static key is set but the sd_asym_cpucapacity shortcut points to NULL even if asymmetry hasn't been hotplugged out.
Disable the static key when destroying domains, and let build_sched_domains() (re) enable it as needed.
--- kernel/sched/topology.c | 2 ++ 1 file changed, 2 insertions(+)
kernel/sched/topology.c +2 -1
@@ -2123,7 +2123,8 @@ static void detach_destroy_domains(const struct cpumask *cpu_map) {int i; +static_branch_disable_cpuslocked (&sched_asym_cpucapacity); +rcu_read_lock ();for_each_cpu (i, cpu_map)cpu_attach_domain (NULL , &def_root_domain, i); --
2.22.0
Quoted 18 lines · Valentin Schneider
While the static key is correctly initialized as being disabled, it will remain forever enabled once turned on. This means that if we start with an asymmetric system and hotplug out enough CPUs to end up with an SMP system, the static key will remain set - which is obviously wrong. We should detect this and turn off things like misfit migration and EAS wakeups.
Having that key enabled should also mandate
per_cpu(sd_asym_cpucapacity, cpu)!= NULLfor all CPUs, but this is obviously not true with the above.
On top of that, sched domain rebuilds first lead to attaching the NULL domain to the affected CPUs, which means there will be a window where the static key is set but the
sd_asym_cpucapacityshortcut points to NULL even if asymmetry hasn't been hotplugged out.
Quoted 6 lines
Disable the static key when destroying domains, and let
build_sched_domains()(re) enable it as needed.Cc <[email protected]>Fixes df054e8445a4 ("sched/topology: Add static_key for asymmetric CPU capacity optimizations")Signed-off-by Valentin Schneider <[email protected]>
Quoted 20 lines
--- kernel/sched/topology.c | 2 ++ 1 file changed, 2 insertions(+)kernel/sched/topology.c +2
-1@@ -2123,7 +2123,8 @@ static void detach_destroy_domains(const struct cpumask *cpu_map) {int i; +static_branch_disable_cpuslocked (&sched_asym_cpucapacity); +rcu_read_lock ();for_each_cpu (i, cpu_map)cpu_attach_domain (NULL , &def_root_domain, i); --2.22.0
Hi Valentin,
Quoted 5 lines · Valentin Schneider
While the static key is correctly initialized as being disabled, it will remain forever enabled once turned on. This means that if we start with an asymmetric system and hotplug out enough CPUs to end up with an SMP system, the static key will remain set - which is obviously wrong. We should detect this and turn off things like misfit migration and EAS wakeups.
FWIW we already clear the EAS static key properly (based on the sd pointer, not the static key), so this is really only for the capacity-aware stuff.
Quoted 26 lines
Having that key enabled should also mandate
per_cpu(sd_asym_cpucapacity, cpu)!= NULLfor all CPUs, but this is obviously not true with the above.
On top of that, sched domain rebuilds first lead to attaching the NULL domain to the affected CPUs, which means there will be a window where the static key is set but the
sd_asym_cpucapacityshortcut points to NULL even if asymmetry hasn't been hotplugged out.Disable the static key when destroying domains, and let
build_sched_domains()(re) enable it as needed.Cc <[email protected]>Fixes df054e8445a4 ("sched/topology: Add static_key for asymmetric CPU capacity optimizations")Signed-off-by Valentin Schneider <[email protected]>--- kernel/sched/topology.c | 2 ++ 1 file changed, 2 insertions(+)kernel/sched/topology.c +0
-0
kernel/sched/topology.c static void detach_destroy_domains(const struct cpumask *cpu_map) · line 2130
Quoted 8 lines
kernel/sched/topology.c +2
-0@@ -2123,7 +2123,8 @@ static void detach_destroy_domains(const struct cpumask *cpu_map) {int i; +static_branch_disable_cpuslocked (&sched_asym_cpucapacity); +rcu_read_lock ();for_each_cpu (i, cpu_map)cpu_attach_domain (NULL , &def_root_domain, i);
So what happens it you have mutiple root domains ? You might skip build_sched_domains() for one of them and end up not setting the static key when you should no ?
I suppose an alternative would be to play with static_branch_inc() / static_branch_dec() from build_sched_domains() or something along those lines.
Thanks,
Quentin
Vincent Guittot
maintainerQuoted 43 lines · Quentin Perret
Hi Valentin,
Quoted 5 lines · Valentin Schneider
While the static key is correctly initialized as being disabled, it will remain forever enabled once turned on. This means that if we start with an asymmetric system and hotplug out enough CPUs to end up with an SMP system, the static key will remain set - which is obviously wrong. We should detect this and turn off things like misfit migration and EAS wakeups.
FWIW we already clear the EAS static key properly (based on the sd pointer, not the static key), so this is really only for the capacity-aware stuff.
Quoted 29 lines
Having that key enabled should also mandate
per_cpu(sd_asym_cpucapacity, cpu)!= NULLfor all CPUs, but this is obviously not true with the above.
On top of that, sched domain rebuilds first lead to attaching the NULL domain to the affected CPUs, which means there will be a window where the static key is set but the
sd_asym_cpucapacityshortcut points to NULL even if asymmetry hasn't been hotplugged out.Disable the static key when destroying domains, and let
build_sched_domains()(re) enable it as needed.Cc <[email protected]>Fixes df054e8445a4 ("sched/topology: Add static_key for asymmetric CPU capacity optimizations")Signed-off-by Valentin Schneider <[email protected]>--- kernel/sched/topology.c | 2 ++ 1 file changed, 2 insertions(+)kernel/sched/topology.c +0
-0@@ -2123,7 +2123,8 @@ static void detach_destroy_domains(const struct cpumask *cpu_map) {int i;
Quoted 9 lines
kernel/sched/topology.c static void detach_destroy_domains(const struct cpumask *cpu_map) · line 2130
Quoted 5 lines
kernel/sched/topology.c +2
-0+static_branch_disable_cpuslocked (&sched_asym_cpucapacity); +rcu_read_lock ();for_each_cpu (i, cpu_map)cpu_attach_domain (NULL , &def_root_domain, i);So what happens it you have mutiple root domains ? You might skip
build_sched_domains()for one of them and end up not setting the static key when you should no ?
good point
Quoted 7 lines
I suppose an alternative would be to play with
static_branch_inc()/static_branch_dec()frombuild_sched_domains()or something along those lines.Thanks,
Quentin
(Replying to the reply because for some reason my mail client never got your reply?!)
Quoted 5 lines · Vincent Guittot
Quoted 4 lines · Quentin Perret
FWIW we already clear the EAS static key properly (based on the sd pointer, not the static key), so this is really only for the capacity-aware stuff.
Ah, right.
Quoted 8 lines
Quoted 8 lines
So what happens it you have mutiple root domains ? You might skip
build_sched_domains()for one of them and end up not setting the static key when you should no ?I suppose an alternative would be to play with
static_branch_inc()/static_branch_dec()frombuild_sched_domains()or something along those lines.
Hmph, so I went with the concept that having the key set should mandate having a non-NULL sd_asym_cpucapacity domain, which is why I unset it as soon as one CPU gets attached to a NULL domain.
Sadly as you pointed out, this doesn't work if we have another root domain that sees asymmetry. It also kinda sounds broken to have SDs of a root domain that does not see asymmetry (e.g. LITTLEs only) to see that key being set. Maybe what we want is to have a key per root domain?
Quoted 2 lines
Quoted 2 lines
Thanks,
Quentin
Quoted 21 lines · Valentin Schneider
(Replying to the reply because for some reason my mail client never got your reply?!)
Quoted 5 lines · Vincent Guittot
Quoted 4 lines · Quentin Perret
FWIW we already clear the EAS static key properly (based on the sd pointer, not the static key), so this is really only for the capacity-aware stuff.
Ah, right.
Quoted 8 lines
Quoted 8 lines
So what happens it you have mutiple root domains ? You might skip
build_sched_domains()for one of them and end up not setting the static key when you should no ?I suppose an alternative would be to play with
static_branch_inc()/static_branch_dec()frombuild_sched_domains()or something along those lines.
Quoted 8 lines
Hmph, so I went with the concept that having the key set should mandate having a non-NULL
sd_asym_cpucapacitydomain, which is why I unset it as soon as one CPU gets attached to a NULL domain.Sadly as you pointed out, this doesn't work if we have another root domain that sees asymmetry. It also kinda sounds broken to have SDs of a root domain that does not see asymmetry (e.g. LITTLEs only) to see that key being set. Maybe what we want is to have a key per root domain?
Right, but that's not possible by definition -- static keys aren't variables. The static keys for asym CPUs and for EAS are just to optimize the case when they're disabled, but when they _are_ enabled, you have no choice but do another per-rd check.
And to clarify what I tried to say before, it might be possible to 'count' the number of RDs that have SD_ASYM_CPUCAPACITY set using static_branch_inc()/dec(), like we do for the SMT static key. I remember trying to do something like that for EAS, but that was easier said than done ... :)
Thanks,
Quentin
Quoted 5 lines · Quentin Perret
Right, but that's not possible by definition -- static keys aren't variables. The static keys for asym CPUs and for EAS are just to optimize the case when they're disabled, but when they _are_ enabled, you have no choice but do another per-rd check.
Bleh, right, realized my nonsense after sending the email.
Quoted 6 lines
And to clarify what I tried to say before, it might be possible to 'count' the number of RDs that have
SD_ASYM_CPUCAPACITYset using static_branch_inc()/dec(), like we do for the SMT static key. I remember trying to do something like that for EAS, but that was easier said than done ... :)
Gotcha, the reason I didn't go with this is that I wanted to maintain the relationship between the key and the flag (you either have both or none). It feels icky to have the key set and to have a NULL sd_asym_cpucapacity pointer.
An alternative might be to have a separate counter for asymmetric rd's, always disable the key on domain destruction and use that counter to figure out if we need to restore it. If we don't care about having a NULL SD pointer while the key is set, we could use the included counter as you're suggesting.
Quoted 3 lines
Thanks,
Quentin
Dietmar Eggemann
reviewerQuoted 21 lines · Valentin Schneider
Quoted 5 lines · Quentin Perret
Right, but that's not possible by definition -- static keys aren't variables. The static keys for asym CPUs and for EAS are just to optimize the case when they're disabled, but when they _are_ enabled, you have no choice but do another per-rd check.
Bleh, right, realized my nonsense after sending the email.
Quoted 6 lines
And to clarify what I tried to say before, it might be possible to 'count' the number of RDs that have
SD_ASYM_CPUCAPACITYset using static_branch_inc()/dec(), like we do for the SMT static key. I remember trying to do something like that for EAS, but that was easier said than done ... :)Gotcha, the reason I didn't go with this is that I wanted to maintain the relationship between the key and the flag (you either have both or none). It feels icky to have the key set and to have a NULL
sd_asym_cpucapacitypointer.
Quoted 5 lines
An alternative might be to have a separate counter for asymmetric rd's, always disable the key on domain destruction and use that counter to figure out if we need to restore it. If we don't care about having a NULL SD pointer while the key is set, we could use the included counter as you're suggesting.
I still don't understand the benefit of the counter approach here. sched_smt_present counts the number of cores with SMT. So in case you have 2 SMT cores with 2 HW threads and you CPU hp out one CPU, you still have sched_smt_present, although 1 CPU doesn't have a SMT thread sibling anymore.
Valentin's patch makes sure that sched_asym_cpucapacity is correctly set when the sd hierarchy is rebuild due to CPU hp. Including the unlikely scenario that an asymmetric CPU capacity system (based on DT's capacity-dmips-mhz values) turns normal SMT because of the max frequency values of the CPUs involved.
Systems with a mix of asymmetric and symmetric CPU capacity rd's have to live with the fact that wake_cap and misfit handling is enabled for them. This should be the case already today.
There should be no SD_ASYM_CPUCAPACITY flag on the sd's of the CPUs of the symmetric CPU capacity rd's. I.e. update_top_cache_domain() should set sd_asym_cpucapacity=NULL for those CPUs.
So as a rule we could say even if a static key enables a code path, a derefenced sd still has to be checked against NULL.
Quoted 12 lines · Dietmar Eggemann
I still don't understand the benefit of the counter approach here.
sched_smt_presentcounts the number of cores with SMT. So in case you have 2 SMT cores with 2 HW threads and you CPU hp out one CPU, you still havesched_smt_present, although 1 CPU doesn't have a SMT thread sibling anymore.Valentin's patch makes sure that
sched_asym_cpucapacityis correctly set when the sd hierarchy is rebuild due to CPU hp. Including the unlikely scenario that an asymmetric CPU capacity system (based on DT's capacity-dmips-mhz values) turns normal SMT because of the max frequency values of the CPUs involved.
Quoted 4 lines
Systems with a mix of asymmetric and symmetric CPU capacity rd's have to live with the fact that
wake_capand misfit handling is enabled for them. This should be the case already today.
Good point, that's what I slowly came to realize this morning.
Quoted 7 lines
There should be no
SD_ASYM_CPUCAPACITYflag on the sd's of the CPUs of the symmetric CPU capacity rd's. I.e.update_top_cache_domain()should set sd_asym_cpucapacity=NULL for those CPUs.So as a rule we could say even if a static key enables a code path, a derefenced sd still has to be checked against NULL.
Yeah, I think there's no escaping it. Let me see if I can do something sensible regarding the static key.
Quoted 5 lines · Dietmar Eggemann
I still don't understand the benefit of the counter approach here.
sched_smt_presentcounts the number of cores with SMT. So in case you have 2 SMT cores with 2 HW threads and you CPU hp out one CPU, you still havesched_smt_present, although 1 CPU doesn't have a SMT thread sibling anymore.
Right, and we want something similar for the asym static key I think. That is, it should be enabled if _at least one_ RD is asymmetric.
Quoted 2 lines
Valentin's patch makes sure that
sched_asym_cpucapacityis correctly set when the sd hierarchy is rebuild due to CPU hp.
As mentioned in a previous email, I think this patch is broken in case you have multiple asymmetric RDs, but counting should fix it, though it might not be that easy to implement.
Quoted 4 lines
Including the unlikely
scenario that an asymmetric CPU capacity system (based on DT's capacity-dmips-mhz values) turns normal SMT because of the max frequency values of the CPUs involved.
I'm not sure what you meant here ?
Quoted 10 lines
Systems with a mix of asymmetric and symmetric CPU capacity rd's have to live with the fact that
wake_capand misfit handling is enabled for them. This should be the case already today.There should be no
SD_ASYM_CPUCAPACITYflag on the sd's of the CPUs of the symmetric CPU capacity rd's. I.e.update_top_cache_domain()should set sd_asym_cpucapacity=NULL for those CPUs.So as a rule we could say even if a static key enables a code path, a derefenced sd still has to be checked against NULL.
Right, that's already true today, and I don't see a possible alternative to that. Same thing for the EAS static key and the pd list btw.
Thanks,
Quentin
Dietmar Eggemann
reviewerQuoted 10 lines · Quentin Perret
Quoted 5 lines · Dietmar Eggemann
I still don't understand the benefit of the counter approach here.
sched_smt_presentcounts the number of cores with SMT. So in case you have 2 SMT cores with 2 HW threads and you CPU hp out one CPU, you still havesched_smt_present, although 1 CPU doesn't have a SMT thread sibling anymore.Right, and we want something similar for the asym static key I think. That is, it should be enabled if
_atleastone_RD is asymmetric.
Quoted 6 lines
Quoted 2 lines
Valentin's patch makes sure that
sched_asym_cpucapacityis correctly set when the sd hierarchy is rebuild due to CPU hp.As mentioned in a previous email, I think this patch is broken in case you have multiple asymmetric RDs, but counting should fix it, though it might not be that easy to implement.
Correct, now I see your point! Can be easily recreated on JUNO setting up two exclusive cpusets, each with one big CPU and then hp'ing one of them.
And partition_sched_domains_locked()/build_perf_domains() handles this by distinguishing existing vs. new PDs and or'ing has_eas.
Quoted 6 lines
Quoted 4 lines
Including the unlikely
scenario that an asymmetric CPU capacity system (based on DT's capacity-dmips-mhz values) turns normal SMT because of the max frequency values of the CPUs involved.I'm not sure what you meant here ?
The rebuild of the sd hierarchy after we know the max frequency values of the CPUs from CpuFreq (2. SD hierarchy build on asym CPU capacity systems with CPUfreq).
E.g on Juno when you set capacity-dmips-mhz_{big;little}=791;1024 with max_CPU_freq_{big;little}=1,100,000;850,000.
It's a theoretical case and might not even work due to rounding errors.
Quoted 8 lines
Quoted 8 lines
Systems with a mix of asymmetric and symmetric CPU capacity rd's have to live with the fact that
wake_capand misfit handling is enabled for them. This should be the case already today.There should be no
SD_ASYM_CPUCAPACITYflag on the sd's of the CPUs of the symmetric CPU capacity rd's. I.e.update_top_cache_domain()should set sd_asym_cpucapacity=NULL for those CPUs.
Quoted 5 lines
Quoted 2 lines
So as a rule we could say even if a static key enables a code path, a derefenced sd still has to be checked against NULL.
Right, that's already true today, and I don't see a possible alternative to that. Same thing for the EAS static key and the pd list btw.
Agreed.