Skip to contentSkip to conversation

sched/topology: Disable sched_asym_cpucapacity on domain destruction

PATCH In review Valentin Schneider 4 participants 11 messages Oct 14, 2019 – Oct 15, 2019

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 sched/topology: Disable sched_asym_cpucapacity on domain destruction

While the static key is correctly initialized as being disabled, it will remain forever enabled once turned on. This means that if we start with an asymmetric system and hotplug out enough CPUs to end up with an SMP system, the static key will remain set - which is obviously wrong. We should detect this and turn off things like misfit migration and EAS wakeups.

Having that key enabled should also mandate

per_cpu(sd_asym_cpucapacity, cpu) != NULL

for all CPUs, but this is obviously not true with the above.

On top of that, sched domain rebuilds first lead to attaching the NULL domain to the affected CPUs, which means there will be a window where the static key is set but the sd_asym_cpucapacity shortcut points to NULL even if asymmetry hasn't been hotplugged out.

Disable the static key when destroying domains, and let build_sched_domains() (re) enable it as needed.

Fixes df054e8445a4 ("sched/topology: Add static_key for asymmetric CPU capacity optimizations")
---
 kernel/sched/topology.c | 2 ++
 1 file changed, 2 insertions(+)
kernel/sched/topology.c +2 -1
index b5667a273bf6..c49ae57a0611 100644
@@ -2123,7 +2123,8 @@ static void detach_destroy_domains(const struct cpumask *cpu_map)
 {
 	int i;
 
+	static_branch_disable_cpuslocked(&sched_asym_cpucapacity);
+
 	rcu_read_lock();
 	for_each_cpu(i, cpu_map)
 		cpu_attach_domain(NULL, &def_root_domain, i);
--

2.22.0

maintainer Acked-by on v1 1/1
#1 lore raw
Quoted 18 lines · Valentin Schneider

While the static key is correctly initialized as being disabled, it will remain forever enabled once turned on. This means that if we start with an asymmetric system and hotplug out enough CPUs to end up with an SMP system, the static key will remain set - which is obviously wrong. We should detect this and turn off things like misfit migration and EAS wakeups.

Having that key enabled should also mandate

per_cpu(sd_asym_cpucapacity, cpu) != NULL

for all CPUs, but this is obviously not true with the above.

On top of that, sched domain rebuilds first lead to attaching the NULL domain to the affected CPUs, which means there will be a window where the static key is set but the sd_asym_cpucapacity shortcut points to NULL even if asymmetry hasn't been hotplugged out.

Quoted 6 lines

Disable the static key when destroying domains, and let build_sched_domains() (re) enable it as needed.

Fixes df054e8445a4 ("sched/topology: Add static_key for asymmetric CPU capacity optimizations")
Acked-by Vincent Guittot <vincent [email protected]>
Quoted 20 lines
---
 kernel/sched/topology.c | 2 ++
 1 file changed, 2 insertions(+)
kernel/sched/topology.c +2 -1
index b5667a273bf6..c49ae57a0611 100644
@@ -2123,7 +2123,8 @@ static void detach_destroy_domains(const struct cpumask *cpu_map)
 {
        int i;
 
+       static_branch_disable_cpuslocked(&sched_asym_cpucapacity);
+
        rcu_read_lock();
        for_each_cpu(i, cpu_map)
                cpu_attach_domain(NULL, &def_root_domain, i);
--

2.22.0

#2 lore raw
In reply to Valentin Schneider

Hi Valentin,

Quoted 5 lines · Valentin Schneider

While the static key is correctly initialized as being disabled, it will remain forever enabled once turned on. This means that if we start with an asymmetric system and hotplug out enough CPUs to end up with an SMP system, the static key will remain set - which is obviously wrong. We should detect this and turn off things like misfit migration and EAS wakeups.

FWIW we already clear the EAS static key properly (based on the sd pointer, not the static key), so this is really only for the capacity-aware stuff.

Quoted 26 lines

Having that key enabled should also mandate

per_cpu(sd_asym_cpucapacity, cpu) != NULL

for all CPUs, but this is obviously not true with the above.

On top of that, sched domain rebuilds first lead to attaching the NULL domain to the affected CPUs, which means there will be a window where the static key is set but the sd_asym_cpucapacity shortcut points to NULL even if asymmetry hasn't been hotplugged out.

Disable the static key when destroying domains, and let build_sched_domains() (re) enable it as needed.

Fixes df054e8445a4 ("sched/topology: Add static_key for asymmetric CPU capacity optimizations")
---
 kernel/sched/topology.c | 2 ++
 1 file changed, 2 insertions(+)
kernel/sched/topology.c +0 -0
index b5667a273bf6..c49ae57a0611 100644

kernel/sched/topology.c static void detach_destroy_domains(const struct cpumask *cpu_map) · line 2130

Quoted 8 lines
kernel/sched/topology.c +2 -0
@@ -2123,7 +2123,8 @@ static void detach_destroy_domains(const struct cpumask *cpu_map)
 {
 	int i;
 
+	static_branch_disable_cpuslocked(&sched_asym_cpucapacity);
+
 	rcu_read_lock();
 	for_each_cpu(i, cpu_map)
 		cpu_attach_domain(NULL, &def_root_domain, i);

So what happens it you have mutiple root domains ? You might skip build_sched_domains() for one of them and end up not setting the static key when you should no ?

I suppose an alternative would be to play with static_branch_inc() / static_branch_dec() from build_sched_domains() or something along those lines.

Thanks,
Quentin

maintainer
#3 lore raw
Quoted 43 lines · Quentin Perret

Hi Valentin,

Quoted 5 lines · Valentin Schneider

While the static key is correctly initialized as being disabled, it will remain forever enabled once turned on. This means that if we start with an asymmetric system and hotplug out enough CPUs to end up with an SMP system, the static key will remain set - which is obviously wrong. We should detect this and turn off things like misfit migration and EAS wakeups.

FWIW we already clear the EAS static key properly (based on the sd pointer, not the static key), so this is really only for the capacity-aware stuff.

Quoted 29 lines

Having that key enabled should also mandate

per_cpu(sd_asym_cpucapacity, cpu) != NULL

for all CPUs, but this is obviously not true with the above.

On top of that, sched domain rebuilds first lead to attaching the NULL domain to the affected CPUs, which means there will be a window where the static key is set but the sd_asym_cpucapacity shortcut points to NULL even if asymmetry hasn't been hotplugged out.

Disable the static key when destroying domains, and let build_sched_domains() (re) enable it as needed.

Fixes df054e8445a4 ("sched/topology: Add static_key for asymmetric CPU capacity optimizations")
---
 kernel/sched/topology.c | 2 ++
 1 file changed, 2 insertions(+)
kernel/sched/topology.c +0 -0
index b5667a273bf6..c49ae57a0611 100644
@@ -2123,7 +2123,8 @@ static void detach_destroy_domains(const struct cpumask *cpu_map)
 {
      int i;
 
Quoted 9 lines

kernel/sched/topology.c static void detach_destroy_domains(const struct cpumask *cpu_map) · line 2130

Quoted 5 lines
kernel/sched/topology.c +2 -0
+     static_branch_disable_cpuslocked(&sched_asym_cpucapacity);
+
      rcu_read_lock();
      for_each_cpu(i, cpu_map)
              cpu_attach_domain(NULL, &def_root_domain, i);

So what happens it you have mutiple root domains ? You might skip build_sched_domains() for one of them and end up not setting the static key when you should no ?

good point

Quoted 7 lines

I suppose an alternative would be to play with static_branch_inc() / static_branch_dec() from build_sched_domains() or something along those lines.

Thanks,
Quentin

#4 lore raw

(Replying to the reply because for some reason my mail client never got your reply?!)

Quoted 5 lines · Vincent Guittot
Quoted 4 lines · Quentin Perret

FWIW we already clear the EAS static key properly (based on the sd pointer, not the static key), so this is really only for the capacity-aware stuff.

Ah, right.

Quoted 8 lines
Quoted 8 lines

So what happens it you have mutiple root domains ? You might skip build_sched_domains() for one of them and end up not setting the static key when you should no ?

I suppose an alternative would be to play with static_branch_inc() / static_branch_dec() from build_sched_domains() or something along those lines.

Hmph, so I went with the concept that having the key set should mandate having a non-NULL sd_asym_cpucapacity domain, which is why I unset it as soon as one CPU gets attached to a NULL domain.

Sadly as you pointed out, this doesn't work if we have another root domain that sees asymmetry. It also kinda sounds broken to have SDs of a root domain that does not see asymmetry (e.g. LITTLEs only) to see that key being set. Maybe what we want is to have a key per root domain?

Quoted 2 lines
Quoted 2 lines

Thanks,
Quentin

#5 lore raw
Quoted 21 lines · Valentin Schneider

(Replying to the reply because for some reason my mail client never got your reply?!)

Quoted 5 lines · Vincent Guittot
Quoted 4 lines · Quentin Perret

FWIW we already clear the EAS static key properly (based on the sd pointer, not the static key), so this is really only for the capacity-aware stuff.

Ah, right.

Quoted 8 lines
Quoted 8 lines

So what happens it you have mutiple root domains ? You might skip build_sched_domains() for one of them and end up not setting the static key when you should no ?

I suppose an alternative would be to play with static_branch_inc() / static_branch_dec() from build_sched_domains() or something along those lines.

Quoted 8 lines

Hmph, so I went with the concept that having the key set should mandate having a non-NULL sd_asym_cpucapacity domain, which is why I unset it as soon as one CPU gets attached to a NULL domain.

Sadly as you pointed out, this doesn't work if we have another root domain that sees asymmetry. It also kinda sounds broken to have SDs of a root domain that does not see asymmetry (e.g. LITTLEs only) to see that key being set. Maybe what we want is to have a key per root domain?

Right, but that's not possible by definition -- static keys aren't variables. The static keys for asym CPUs and for EAS are just to optimize the case when they're disabled, but when they _are_ enabled, you have no choice but do another per-rd check.

And to clarify what I tried to say before, it might be possible to 'count' the number of RDs that have SD_ASYM_CPUCAPACITY set using static_branch_inc()/dec(), like we do for the SMT static key. I remember trying to do something like that for EAS, but that was easier said than done ... :)

Thanks,
Quentin

#6 lore raw
Quoted 5 lines · Quentin Perret

Right, but that's not possible by definition -- static keys aren't variables. The static keys for asym CPUs and for EAS are just to optimize the case when they're disabled, but when they _are_ enabled, you have no choice but do another per-rd check.

Bleh, right, realized my nonsense after sending the email.

Quoted 6 lines

And to clarify what I tried to say before, it might be possible to 'count' the number of RDs that have SD_ASYM_CPUCAPACITY set using static_branch_inc()/dec(), like we do for the SMT static key. I remember trying to do something like that for EAS, but that was easier said than done ... :)

Gotcha, the reason I didn't go with this is that I wanted to maintain the relationship between the key and the flag (you either have both or none). It feels icky to have the key set and to have a NULL sd_asym_cpucapacity pointer.

An alternative might be to have a separate counter for asymmetric rd's, always disable the key on domain destruction and use that counter to figure out if we need to restore it. If we don't care about having a NULL SD pointer while the key is set, we could use the included counter as you're suggesting.

Quoted 3 lines

Thanks,
Quentin

#7 lore raw
Quoted 21 lines · Valentin Schneider
Quoted 5 lines · Quentin Perret

Right, but that's not possible by definition -- static keys aren't variables. The static keys for asym CPUs and for EAS are just to optimize the case when they're disabled, but when they _are_ enabled, you have no choice but do another per-rd check.

Bleh, right, realized my nonsense after sending the email.

Quoted 6 lines

And to clarify what I tried to say before, it might be possible to 'count' the number of RDs that have SD_ASYM_CPUCAPACITY set using static_branch_inc()/dec(), like we do for the SMT static key. I remember trying to do something like that for EAS, but that was easier said than done ... :)

Gotcha, the reason I didn't go with this is that I wanted to maintain the relationship between the key and the flag (you either have both or none). It feels icky to have the key set and to have a NULL sd_asym_cpucapacity pointer.

Quoted 5 lines

An alternative might be to have a separate counter for asymmetric rd's, always disable the key on domain destruction and use that counter to figure out if we need to restore it. If we don't care about having a NULL SD pointer while the key is set, we could use the included counter as you're suggesting.

I still don't understand the benefit of the counter approach here. sched_smt_present counts the number of cores with SMT. So in case you have 2 SMT cores with 2 HW threads and you CPU hp out one CPU, you still have sched_smt_present, although 1 CPU doesn't have a SMT thread sibling anymore.

Valentin's patch makes sure that sched_asym_cpucapacity is correctly set when the sd hierarchy is rebuild due to CPU hp. Including the unlikely scenario that an asymmetric CPU capacity system (based on DT's capacity-dmips-mhz values) turns normal SMT because of the max frequency values of the CPUs involved.

Systems with a mix of asymmetric and symmetric CPU capacity rd's have to live with the fact that wake_cap and misfit handling is enabled for them. This should be the case already today.

There should be no SD_ASYM_CPUCAPACITY flag on the sd's of the CPUs of the symmetric CPU capacity rd's. I.e. update_top_cache_domain() should set sd_asym_cpucapacity=NULL for those CPUs.

So as a rule we could say even if a static key enables a code path, a derefenced sd still has to be checked against NULL.

#8 lore raw
Quoted 12 lines · Dietmar Eggemann

I still don't understand the benefit of the counter approach here. sched_smt_present counts the number of cores with SMT. So in case you have 2 SMT cores with 2 HW threads and you CPU hp out one CPU, you still have sched_smt_present, although 1 CPU doesn't have a SMT thread sibling anymore.

Valentin's patch makes sure that sched_asym_cpucapacity is correctly set when the sd hierarchy is rebuild due to CPU hp. Including the unlikely scenario that an asymmetric CPU capacity system (based on DT's capacity-dmips-mhz values) turns normal SMT because of the max frequency values of the CPUs involved.

Quoted 4 lines

Systems with a mix of asymmetric and symmetric CPU capacity rd's have to live with the fact that wake_cap and misfit handling is enabled for them. This should be the case already today.

Good point, that's what I slowly came to realize this morning.

Quoted 7 lines

There should be no SD_ASYM_CPUCAPACITY flag on the sd's of the CPUs of the symmetric CPU capacity rd's. I.e. update_top_cache_domain() should set sd_asym_cpucapacity=NULL for those CPUs.

So as a rule we could say even if a static key enables a code path, a derefenced sd still has to be checked against NULL.

Yeah, I think there's no escaping it. Let me see if I can do something sensible regarding the static key.

#9 lore raw
In reply to Dietmar Eggemann
Quoted 5 lines · Dietmar Eggemann

I still don't understand the benefit of the counter approach here. sched_smt_present counts the number of cores with SMT. So in case you have 2 SMT cores with 2 HW threads and you CPU hp out one CPU, you still have sched_smt_present, although 1 CPU doesn't have a SMT thread sibling anymore.

Right, and we want something similar for the asym static key I think. That is, it should be enabled if _at least one_ RD is asymmetric.

Quoted 2 lines

Valentin's patch makes sure that sched_asym_cpucapacity is correctly set when the sd hierarchy is rebuild due to CPU hp.

As mentioned in a previous email, I think this patch is broken in case you have multiple asymmetric RDs, but counting should fix it, though it might not be that easy to implement.

Quoted 4 lines

Including the unlikely
scenario that an asymmetric CPU capacity system (based on DT's capacity-dmips-mhz values) turns normal SMT because of the max frequency values of the CPUs involved.

I'm not sure what you meant here ?

Quoted 10 lines

Systems with a mix of asymmetric and symmetric CPU capacity rd's have to live with the fact that wake_cap and misfit handling is enabled for them. This should be the case already today.

There should be no SD_ASYM_CPUCAPACITY flag on the sd's of the CPUs of the symmetric CPU capacity rd's. I.e. update_top_cache_domain() should set sd_asym_cpucapacity=NULL for those CPUs.

So as a rule we could say even if a static key enables a code path, a derefenced sd still has to be checked against NULL.

Right, that's already true today, and I don't see a possible alternative to that. Same thing for the EAS static key and the pd list btw.

Thanks,
Quentin

#10 lore raw
Quoted 10 lines · Quentin Perret
Quoted 5 lines · Dietmar Eggemann

I still don't understand the benefit of the counter approach here. sched_smt_present counts the number of cores with SMT. So in case you have 2 SMT cores with 2 HW threads and you CPU hp out one CPU, you still have sched_smt_present, although 1 CPU doesn't have a SMT thread sibling anymore.

Right, and we want something similar for the asym static key I think. That is, it should be enabled if _at least one_ RD is asymmetric.

Quoted 6 lines
Quoted 2 lines

Valentin's patch makes sure that sched_asym_cpucapacity is correctly set when the sd hierarchy is rebuild due to CPU hp.

As mentioned in a previous email, I think this patch is broken in case you have multiple asymmetric RDs, but counting should fix it, though it might not be that easy to implement.

Correct, now I see your point! Can be easily recreated on JUNO setting up two exclusive cpusets, each with one big CPU and then hp'ing one of them.

And partition_sched_domains_locked()/build_perf_domains() handles this by distinguishing existing vs. new PDs and or'ing has_eas.

Quoted 6 lines
Quoted 4 lines

Including the unlikely
scenario that an asymmetric CPU capacity system (based on DT's capacity-dmips-mhz values) turns normal SMT because of the max frequency values of the CPUs involved.

I'm not sure what you meant here ?

The rebuild of the sd hierarchy after we know the max frequency values of the CPUs from CpuFreq (2. SD hierarchy build on asym CPU capacity systems with CPUfreq).

E.g on Juno when you set capacity-dmips-mhz_{big;little}=791;1024 with max_CPU_freq_{big;little}=1,100,000;850,000.

It's a theoretical case and might not even work due to rounding errors.

Quoted 8 lines
Quoted 8 lines

Systems with a mix of asymmetric and symmetric CPU capacity rd's have to live with the fact that wake_cap and misfit handling is enabled for them. This should be the case already today.

There should be no SD_ASYM_CPUCAPACITY flag on the sd's of the CPUs of the symmetric CPU capacity rd's. I.e. update_top_cache_domain() should set sd_asym_cpucapacity=NULL for those CPUs.

Quoted 5 lines
Quoted 2 lines

So as a rule we could say even if a static key enables a code path, a derefenced sd still has to be checked against NULL.

Right, that's already true today, and I don't see a possible alternative to that. Same thing for the EAS static key and the pd list btw.

Agreed.