Skip to contentSkip to conversation

xen/sched: fix get_cpu_idle_time() for smt=0 suspend/resume

PATCH In review Juergen Gross 3 participants 4 messages Aug 18, 2021

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xen/sched: fix get_cpu_idle_time() for smt=0 suspend/resume

With smt=0 during a suspend/resume cycle of the machine the threads which have been parked before will briefly come up again. This can result in problems e.g. with cpufreq driver being active as this will call into get_cpu_idle_time() for a cpu without initialized scheduler data.

Fix that by letting get_cpu_idle_time() deal with this case.

Fixes 132cbe8f35632fb2 ("sched: fix get_cpu_idle_time() with core scheduling")

---
An alternative way to fix the issue would be to keep the sched_resource of offline cpus allocated like we already do with idle vcpus and units. This fix would be more intrusive, but it would avoid similar other bugs like this one.

---
 xen/common/sched/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
xen/common/sched/core.c +1 -1
index 6d34764d38..9ac1b01ca8 100644
@@ -337,7 +337,7 @@ uint64_t get_cpu_idle_time(unsigned int cpu)
     struct vcpu_runstate_info state = { 0 };
     const struct vcpu *v = idle_vcpu[cpu];
 
-    if ( cpu_online(cpu) && v )
+    if ( cpu_online(cpu) && v && get_sched_res(cpu) )
         vcpu_runstate_get(v, &state);
 
     return state.time[RUNSTATE_running];
Signature · 1 line
2.26.2
Reviewed-by on v1 1/1
#1 lore raw
Quoted 6 lines · Juergen Gross

With smt=0 during a suspend/resume cycle of the machine the threads which have been parked before will briefly come up again. This can result in problems e.g. with cpufreq driver being active as this will call into get_cpu_idle_time() for a cpu without initialized scheduler data.

Quoted 6 lines

Fix that by letting get_cpu_idle_time() deal with this case.

Fixes 132cbe8f35632fb2 ("sched: fix get_cpu_idle_time() with core scheduling")

xen/common/sched/core.c uint64_t get_cpu_idle_time(unsigned int cpu) · line 341

Quoted 9 lines
xen/common/sched/core.c +1 -1
@@ -337,7 +337,7 @@ uint64_t get_cpu_idle_time(unsigned int cpu)
     struct vcpu_runstate_info state = { 0 };
     const struct vcpu *v = idle_vcpu[cpu];
 
-    if ( cpu_online(cpu) && v )
+    if ( cpu_online(cpu) && v && get_sched_res(cpu) )
         vcpu_runstate_get(v, &state);

My earlier question was aiming at getting rid of the (now) middle part of the condition; I thought this may be okay to do as a secondary change here. But perhaps you intentionally left it there, so I'm unsure whether to suggest to make the adjustment while committing (awaiting a maintainer ack first anyway).

Jan

#2 lore raw
Quoted 22 lines · Jan Beulich
Quoted 6 lines · Juergen Gross

With smt=0 during a suspend/resume cycle of the machine the threads which have been parked before will briefly come up again. This can result in problems e.g. with cpufreq driver being active as this will call into get_cpu_idle_time() for a cpu without initialized scheduler data.

Quoted 6 lines

Fix that by letting get_cpu_idle_time() deal with this case.

Fixes 132cbe8f35632fb2 ("sched: fix get_cpu_idle_time() with core scheduling")
Quoted 6 lines
xen/common/sched/core.c +0 -0
@@ -337,7 +337,7 @@ uint64_t get_cpu_idle_time(unsigned int cpu)
      struct vcpu_runstate_info state = { 0 };
      const struct vcpu *v = idle_vcpu[cpu];
 
Quoted 9 lines

xen/common/sched/core.c uint64_t get_cpu_idle_time(unsigned int cpu) · line 341

Quoted 3 lines
xen/common/sched/core.c +1 -1
-    if ( cpu_online(cpu) && v )
+    if ( cpu_online(cpu) && v && get_sched_res(cpu) )
          vcpu_runstate_get(v, &state);

My earlier question was aiming at getting rid of the (now) middle part of the condition; I thought this may be okay to do as a secondary change here. But perhaps you intentionally left it there, so I'm unsure whether to suggest to make the adjustment while committing (awaiting a maintainer ack first anyway).

Ah, okay. Yes, I think the test of v being non-NULL can be removed.

Juergen

Acked-by on v1 1/1
#3 lore raw
In reply to Juergen Gross
Quoted 10 lines · Juergen Gross

Fix that by letting get_cpu_idle_time() deal with this case.

Fixes 132cbe8f35632fb2 ("sched: fix get_cpu_idle_time() with core

scheduling")
Reported-by: Marek Marczykowski-Górecki
<[email protected]>

Tested-by Marek Marczykowski-Górecki

<[email protected]>

Mmm... This is an interesting one. In fact, this fix is not only correct, it's also simple, effective and (I guess) easy enough to backport.

Considering all these things together with the fact that we have an open issue, I'm going to provide my:

(and this remains valid with Jan's proposed change done upon committing.)

That said, in the long run...

Quoted 9 lines

---
An alternative way to fix the issue would be to keep the sched_resource
of offline cpus allocated like we already do with idle vcpus and units.
This fix would be more intrusive, but it would avoid similar other bugs
like this one.

... it would be probably interesting to go this route, as it looks both more consistent and future proof (I mean implement it proactively, independently of issues... when/if we have time, of course!)

Thanks and Regards

Signature · 6 lines
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)