xen: Fix some bugs in scheduling
v2 cover xen: Fix some bugs in scheduling
Some problems I found when trying to find a problem with cpu-on/offlining in core scheduling mode.
Juergen Gross (3):
xen/sched: allow rcu work to happen when syncing cpus in core
scheduling
xen/sched: don't call sync_vcpu_execstate() in
sched_unit_migrate_finish()
xen/sched: fix latent races accessing vcpu->dirty_cpu
xen/arch/x86/domain.c | 16 +++++++++++----- xen/common/domain.c | 2 +- xen/common/keyhandler.c | 2 +- xen/common/sched/core.c | 18 ++++++++++-------- xen/include/xen/sched.h | 2 +- xen/include/xen/softirq.h | 2 +- 6 files changed, 25 insertions(+), 17 deletions(-)
Signature · 1 line
2.26.1
PATCH v2 1/3 xen/sched: allow rcu work to happen when syncing cpus in core scheduling
With RCU barriers moved from tasklets to normal RCU processing cpu offlining in core scheduling might deadlock due to cpu synchronization required by RCU processing and core scheduling concurrently.
Fix that by bailing out from core scheduling synchronization in case of pending RCU work. Additionally the RCU softirq is now required to be of higher priority than the scheduling softirqs in order to do RCU processing before entering the scheduler again, as bailing out from the core scheduling synchronization requires to raise another softirq SCHED_SLAVE, which would bypass RCU processing again.
---
V2:
- add BUILD_BUG_ON() and comment (Dario Faggioli)
--- xen/common/sched/core.c | 13 ++++++++++--- xen/include/xen/softirq.h | 2 +- 2 files changed, 11 insertions(+), 4 deletions(-)
xen/common/sched/core.c +10 -3
@@ -2457,13 +2457,20 @@ static struct sched_unit *sched_wait_rendezvous_in(struct sched_unit *prev, v =unit2vcpu_cpu (prev, cpu); }/* -* Coming from idle might need to do tasklet work. +* Check for any work to be done which might need cpu synchronization. +* This is either pending RCU work, or tasklet work when coming from +* idle. It is mandatory that RCU softirqs are of higher priority +* than scheduling ones as otherwise a deadlock might occur. * In order to avoid deadlocks we can't do that here, but have to -* continue the idle loop. +* schedule the previous vcpu again, which will lead to the desired +* processing to be done. * Undo the rendezvous_in_cnt decrement and schedule another call of * sched_slave(). */ -if (is_idle_unit (prev) &&sched_tasklet_check_cpu (cpu) ) +BUILD_BUG_ON (RCU_SOFTIRQ >SCHED_SLAVE_SOFTIRQ || +RCU_SOFTIRQ >SCHEDULE_SOFTIRQ ); +if (rcu_pending (cpu) || + (is_idle_unit (prev) &&sched_tasklet_check_cpu (cpu)) ) {struct vcpu *vprev = current;
xen/include/xen/softirq.h +1 -1
@@ -4,10 +4,10 @@/* Low-latency softirqs come first in the following list. */ enum {TIMER_SOFTIRQ =0 , +RCU_SOFTIRQ ,SCHED_SLAVE_SOFTIRQ ,SCHEDULE_SOFTIRQ ,NEW_TLBFLUSH_CLOCK_PERIOD_SOFTIRQ , -RCU_SOFTIRQ ,TASKLET_SOFTIRQ ,NR_COMMON_SOFTIRQS };
Signature · 1 line
2.26.1
PATCH v2 2/3 xen/sched: don't call sync_vcpu_execstate() in sched_unit_migrate_finish()
In reply to Juergen GrossWith support of core scheduling sched_unit_migrate_finish() gained a call of sync_vcpu_execstate() as it was believed to be called as a result of vcpu migration in any case.
In case of migrating a vcpu away from a physical cpu for a short period of time only this might not be true, so drop the call and let the lazy state syncing do its job.
---
V2:
- new patch
--- xen/common/sched/core.c | 5 ----- 1 file changed, 5 deletions(-)
xen/common/sched/core.c +0 -5
@@ -1078,12 +1078,7 @@ static void sched_unit_migrate_finish(struct sched_unit *unit)sched_spin_unlock_double (old_lock, new_lock, flags);if ( old_cpu != new_cpu ) - { -/* Vcpus are moved to other pcpus, commit their states to memory. */ - for_each_sched_unit_vcpu ( unit, v ) -sync_vcpu_execstate (v);sched_move_irqs (unit); - }/* Wake on new CPU. */ for_each_sched_unit_vcpu ( unit, v )
Signature · 1 line
2.26.1
Re: [PATCH v2 2/3] xen/sched: don't call sync_vcpu_execstate() in sched_unit_migrate_finish()
Quoted 7 lines · Juergen Gross
With support of core scheduling
sched_unit_migrate_finish()gained a call ofsync_vcpu_execstate()as it was believed to be called as a result of vcpu migration in any case.In case of migrating a vcpu away from a physical cpu for a short period of time only this might not be true, so drop the call and let the lazy state syncing do its job.
Replying here instead of on the patch 3 thread (and I'm sorry for mixing up function names there): By saying "for a short period of time only", do you imply without ever getting scheduled on the new (temporary) CPU? If so, I think I understand this change now, but then this could do with saying here. If not, I'm afraid I'm still lost.
Jan
Re: [PATCH v2 2/3] xen/sched: don't call sync_vcpu_execstate() in sched_unit_migrate_finish()
Quoted 9 lines · Jan Beulich
Quoted 7 lines · Juergen Gross
With support of core scheduling
sched_unit_migrate_finish()gained a call ofsync_vcpu_execstate()as it was believed to be called as a result of vcpu migration in any case.In case of migrating a vcpu away from a physical cpu for a short period of time only this might not be true, so drop the call and let the lazy state syncing do its job.
Quoted 6 lines
Replying here instead of on the patch 3 thread (and I'm sorry for mixing up function names there): By saying "for a short period of time only", do you imply without ever getting scheduled on the new (temporary) CPU? If so, I think I understand this change now, but then this could do with saying here. If not, I'm afraid I'm still lost.
I'll change the commit message to:
... for a short period of time only without ever being scheduled on the selected new cpu ...
Juergen
Jan Beulich
Reviewed-byRe: [PATCH v2 2/3] xen/sched: don't call sync_vcpu_execstate() in sched_unit_migrate_finish()
Quoted 17 lines · Jürgen Groß
Quoted 9 lines · Jan Beulich
Quoted 7 lines · Juergen Gross
With support of core scheduling
sched_unit_migrate_finish()gained a call ofsync_vcpu_execstate()as it was believed to be called as a result of vcpu migration in any case.In case of migrating a vcpu away from a physical cpu for a short period of time only this might not be true, so drop the call and let the lazy state syncing do its job.
Quoted 6 lines
Replying here instead of on the patch 3 thread (and I'm sorry for mixing up function names there): By saying "for a short period of time only", do you imply without ever getting scheduled on the new (temporary) CPU? If so, I think I understand this change now, but then this could do with saying here. If not, I'm afraid I'm still lost.
Quoted 4 lines
I'll change the commit message to:
... for a short period of time only without ever being scheduled on the selected new cpu ...
And then
for both this one and patch 3 (ideally with the one unnecessary hunk dropped).
Jan
PATCH v2 3/3 xen/sched: fix latent races accessing vcpu->dirty_cpu
In reply to Juergen GrossThe dirty_cpu field of struct vcpu denotes which cpu still holds data of a vcpu. All accesses to this field should be atomic in case the vcpu could just be running, as it is accessed without any lock held in most cases. Especially sync_local_execstate() and context_switch() for the same vcpu running concurrently have a risk for failing.
There are some instances where accesses are not atomically done, and even worse where multiple accesses are done when a single one would be mandated.
Correct that in order to avoid potential problems.
Add some assertions to verify dirty_cpu is handled properly.
---
V2:
- convert all accesses to v->dirty_cpu to atomic ones (Jan Beulich)
- drop cast (Julien Grall)
--- xen/arch/x86/domain.c | 16 +++++++++++----- xen/common/domain.c | 2 +- xen/common/keyhandler.c | 2 +- xen/include/xen/sched.h | 2 +- 4 files changed, 14 insertions(+), 8 deletions(-)
xen/arch/x86/domain.c +11 -5
@@ -183,7 +183,7 @@ void startup_cpu_idle_loop(void)ASSERT (is_idle_vcpu (v));cpumask_set_cpu (v->processor, v->domain->dirty_cpumask); - v->dirty_cpu = v->processor; +write_atomic (&v->dirty_cpu, v->processor);reset_stack_and_jump (idle_loop); }
@@ -1769,6 +1769,7 @@ static void __context_switch(void)if ( !is_idle_domain (pd) ) { +ASSERT (read_atomic (&p->dirty_cpu) == cpu);memcpy (&p->arch.user_regs, stack_regs,CTXT_SWITCH_STACK_BYTES );vcpu_save_fpu (p); pd->arch.ctxt_switch->from (p);
@@ -1832,7 +1833,7 @@ void context_switch(struct vcpu *prev, struct vcpu *next) {unsigned int cpu =smp_processor_id ();const struct domain *prevd = prev->domain, *nextd = next->domain; -unsigned int dirty_cpu = next->dirty_cpu; +unsigned int dirty_cpu =read_atomic (&next->dirty_cpu);ASSERT (prev != next);ASSERT (local_irq_is_enabled ());
@@ -1844,6 +1845,7 @@ void context_switch(struct vcpu *prev, struct vcpu *next) {/* Remote CPU calls __sync_local_execstate() from flush IPI handler. */ flush_mask (cpumask_of (dirty_cpu),FLUSH_VCPU_STATE ); +ASSERT (!vcpu_cpu_dirty (next)); }_update_runstate_area (prev);
@@ -1956,13 +1958,17 @@ void sync_local_execstate(void)void sync_vcpu_execstate (struct vcpu *v) { -if ( v->dirty_cpu ==smp_processor_id () ) +unsigned int dirty_cpu =read_atomic (&v->dirty_cpu); + +if ( dirty_cpu ==smp_processor_id () )sync_local_execstate (); -else if (vcpu_cpu_dirty (v) ) +else if (is_vcpu_dirty_cpu (dirty_cpu) ) {/* Remote CPU calls __sync_local_execstate() from flush IPI handler. */ -flush_mask (cpumask_of (v->dirty_cpu),FLUSH_VCPU_STATE ); +flush_mask (cpumask_of (dirty_cpu),FLUSH_VCPU_STATE ); } +ASSERT (!is_vcpu_dirty_cpu (dirty_cpu) || +read_atomic (&v->dirty_cpu) != dirty_cpu); } static int relinquish_memory(
xen/common/domain.c +1 -1
@@ -158,7 +158,7 @@ struct vcpu *vcpu_create(struct domain *d, unsigned int vcpu_id) v->domain = d; v->vcpu_id = vcpu_id; - v->dirty_cpu =VCPU_CPU_CLEAN ; +write_atomic (&v->dirty_cpu,VCPU_CPU_CLEAN );spin_lock_init (&v->virq_lock);
xen/common/keyhandler.c +1 -1
@@ -316,7 +316,7 @@ static void dump_domains(unsigned char key)vcpu_info (v, evtchn_upcall_pending), !vcpu_event_delivery_is_enabled(v));if (vcpu_cpu_dirty (v) ) -printk ("dirty_cpu=%u" , v->dirty_cpu); +printk ("dirty_cpu=%u" ,read_atomic (&v->dirty_cpu));printk ("\n" );printk (" pause_count=%d pause_flags=%lx\n" ,atomic_read (&v->pause_count), v->pause_flags);
xen/include/xen/sched.h +1 -1
@@ -844,7 +844,7 @@ static inline bool is_vcpu_dirty_cpu(unsigned int cpu)static inline bool vcpu_cpu_dirty (const struct vcpu *v) { -return is_vcpu_dirty_cpu (v->dirty_cpu); +return is_vcpu_dirty_cpu (read_atomic (&v->dirty_cpu)); }void vcpu_block (void );
Signature · 1 line
2.26.1
Re: [PATCH v2 3/3] xen/sched: fix latent races accessing vcpu->dirty_cpu
Quoted 8 lines · Juergen Gross
xen/arch/x86/domain.c +3
-1@@ -1956,13 +1958,17 @@ void sync_local_execstate(void)void sync_vcpu_execstate (struct vcpu *v) { -if ( v->dirty_cpu ==smp_processor_id () ) +unsigned int dirty_cpu =read_atomic (&v->dirty_cpu); + +if ( dirty_cpu ==smp_processor_id () )
xen/arch/x86/domain.c void sync_local_execstate(void) · line 1971
Quoted 10 lines
xen/arch/x86/domain.c +4
-2sync_local_execstate (); - else if ( vcpu_cpu_dirty(v) ) + else if ( is_vcpu_dirty_cpu(dirty_cpu) ) {/* Remote CPU calls __sync_local_execstate() from flush IPI handler. */ -flush_mask (cpumask_of (v->dirty_cpu),FLUSH_VCPU_STATE ); +flush_mask (cpumask_of (dirty_cpu),FLUSH_VCPU_STATE ); } +ASSERT (!is_vcpu_dirty_cpu (dirty_cpu) || +read_atomic (&v->dirty_cpu) != dirty_cpu);
Repeating my v1.1 comments:
"However, having stared at it for a while now - is this race
free? I can see this being fine in the (initial) case of dirty_cpu == smp_processor_id(), but if this is for a foreign CPU, can't the vCPU have gone back to that same CPU again in the meantime?"
and later
"There is a time window from late in flush_mask() to the assertion
you add. All sorts of things can happen during this window on other CPUs. IOW what guarantees the vCPU not getting unpaused or its affinity getting changed yet another time?"
You did reply that by what is now patch 2 this race can be eliminated, but I have to admit I don't see why this would be. Hence at the very least I'd expect justification in either the description or a code comment as to why there's no race left (and also no race to be expected to be re-introduced by code changes elsewhere - very unlikely races are, by their nature, unlikely to be hit during code development and the associated testing, hence I'd like there to be sufficiently close to a guarantee here).
My reservations here may in part be due to not following the reasoning for patch 2, which therefore I'll have to rely on the scheduler maintainers to judge on.
xen/common/domain.c struct vcpu *vcpu_create(struct domain *d, unsigned int vcpu_id) · line 161
Quoted 8 lines
xen/common/domain.c +1
-1@@ -158,7 +158,7 @@ struct vcpu *vcpu_create(struct domain *d, unsigned int vcpu_id) v->domain = d; v->vcpu_id = vcpu_id; - v->dirty_cpu =VCPU_CPU_CLEAN ; +write_atomic (&v->dirty_cpu,VCPU_CPU_CLEAN );
This is not strictly necessary (the vCPU won't be acted upon by any other entity in the system just yet), and with this I'd like to suggest to drop this change again.
Jan
Re: [PATCH v2 3/3] xen/sched: fix latent races accessing vcpu->dirty_cpu
Quoted 45 lines · Jan Beulich
Quoted 8 lines · Juergen Gross
xen/arch/x86/domain.c +3
-1@@ -1956,13 +1958,17 @@ void sync_local_execstate(void)void sync_vcpu_execstate (struct vcpu *v) { -if ( v->dirty_cpu ==smp_processor_id () ) +unsigned int dirty_cpu =read_atomic (&v->dirty_cpu); + +if ( dirty_cpu ==smp_processor_id () )xen/arch/x86/domain.c void sync_local_execstate(void) · line 1971
Quoted 10 lines
xen/arch/x86/domain.c +4
-2sync_local_execstate (); - else if ( vcpu_cpu_dirty(v) ) + else if ( is_vcpu_dirty_cpu(dirty_cpu) ) {/* Remote CPU calls __sync_local_execstate() from flush IPI handler. */ -flush_mask (cpumask_of (v->dirty_cpu),FLUSH_VCPU_STATE ); +flush_mask (cpumask_of (dirty_cpu),FLUSH_VCPU_STATE ); } +ASSERT (!is_vcpu_dirty_cpu (dirty_cpu) || +read_atomic (&v->dirty_cpu) != dirty_cpu);Repeating my v1.1 comments:
"However, having stared at it for a while now - is this race
free? I can see this being fine in the (initial) case ofdirty_cpu==smp_processor_id(), but if this is for a foreign CPU, can't the vCPU have gone back to that same CPU again in the meantime?"and later
"There is a time window from late in
flush_mask()to the assertion
you add. All sorts of things can happen during this window on other CPUs. IOW what guarantees the vCPU not getting unpaused or its affinity getting changed yet another time?"You did reply that by what is now patch 2 this race can be eliminated, but I have to admit I don't see why this would be. Hence at the very least I'd expect justification in either the description or a code comment as to why there's no race left (and also no race to be expected to be re-introduced by code changes elsewhere - very unlikely races are, by their nature, unlikely to be hit during code development and the associated testing, hence I'd like there to be sufficiently close to a guarantee here).
Quoted 3 lines
My reservations here may in part be due to not following the reasoning for patch 2, which therefore I'll have to rely on the scheduler maintainers to judge on.
sync_vcpu_execstate() isn't called for a running or runnable vcpu any longer. I can add an ASSERT() and a comment explaining it if you like that better.
Quoted 5 lines
Quoted 5 lines
xen/common/domain.c +0
-0@@ -158,7 +158,7 @@ struct vcpu *vcpu_create(struct domain *d, unsigned int vcpu_id)
Quoted 8 lines
xen/common/domain.c struct vcpu *vcpu_create(struct domain *d, unsigned int vcpu_id) · line 161
Quoted 4 lines
xen/common/domain.c +1
-1v->domain = d; v->vcpu_id = vcpu_id; - v->dirty_cpu =VCPU_CPU_CLEAN ; +write_atomic (&v->dirty_cpu,VCPU_CPU_CLEAN );This is not strictly necessary (the vCPU won't be acted upon by any other entity in the system just yet), and with this I'd like to suggest to drop this change again.
Fine with me.
Juergen
Re: [PATCH v2 3/3] xen/sched: fix latent races accessing vcpu->dirty_cpu
Quoted 45 lines · Jürgen Groß
Quoted 44 lines · Jan Beulich
Quoted 7 lines · Juergen Gross
xen/arch/x86/domain.c +3
-1@@ -1956,13 +1958,17 @@ void sync_local_execstate(void) void sync_vcpu_execstate(struct vcpu *v) { - if ( v->dirty_cpu == smp_processor_id() ) + unsigned int dirty_cpu = read_atomic(&v->dirty_cpu); + + if ( dirty_cpu == smp_processor_id() )xen/arch/x86/domain.c void sync_local_execstate(void) · line 1970
Quoted 10 lines
xen/arch/x86/domain.c +4
-2sync_local_execstate(); - else if ( vcpu_cpu_dirty(v) ) + else if ( is_vcpu_dirty_cpu(dirty_cpu) ) { /* Remote CPU calls __sync_local_execstate() from flush IPI handler. */ - flush_mask(cpumask_of(v->dirty_cpu), FLUSH_VCPU_STATE); + flush_mask(cpumask_of(dirty_cpu), FLUSH_VCPU_STATE); } + ASSERT(!is_vcpu_dirty_cpu(dirty_cpu) || + read_atomic(&v->dirty_cpu) != dirty_cpu);Repeating my v1.1 comments:
"However, having stared at it for a while now - is this race free? I can see this being fine in the (initial) case of
dirty_cpu==smp_processor_id(), but if this is for a foreign CPU, can't the vCPU have gone back to that same CPU again in the meantime?"and later
"There is a time window from late in
flush_mask()to the assertion you add. All sorts of things can happen during this window on other CPUs. IOW what guarantees the vCPU not getting unpaused or its affinity getting changed yet another time?"You did reply that by what is now patch 2 this race can be eliminated, but I have to admit I don't see why this would be. Hence at the very least I'd expect justification in either the description or a code comment as to why there's no race left (and also no race to be expected to be re-introduced by code changes elsewhere - very unlikely races are, by their nature, unlikely to be hit during code development and the associated testing, hence I'd like there to be sufficiently close to a guarantee here).
Quoted 7 lines
Quoted 3 lines
My reservations here may in part be due to not following the reasoning for patch 2, which therefore I'll have to rely on the scheduler maintainers to judge on.
sync_vcpu_execstate()isn't called for a running or runnable vcpu any longer. I can add anASSERT()and a comment explaining it if you like that better.
This would help (hopefully people adding new uses of the function would run into this assertion/comment), but for example the uses in mapcache_current_vcpu() or do_tasklet_work() look to be pretty hard to prove they can't happen for a runnable vCPU.
Jan
Re: [PATCH v2 3/3] xen/sched: fix latent races accessing vcpu->dirty_cpu
Quoted 50 lines · Jan Beulich
Quoted 49 lines · Jürgen Groß
Quoted 47 lines · Jan Beulich
Quoted 7 lines · Juergen Gross
xen/arch/x86/domain.c +3
-1@@ -1956,13 +1958,17 @@ void sync_local_execstate(void) void sync_vcpu_execstate(struct vcpu *v) { - if ( v->dirty_cpu == smp_processor_id() ) + unsigned int dirty_cpu = read_atomic(&v->dirty_cpu); + + if ( dirty_cpu == smp_processor_id() )xen/arch/x86/domain.c void sync_local_execstate(void) · line 1970
Quoted 10 lines
xen/arch/x86/domain.c +4
-2sync_local_execstate(); - else if ( vcpu_cpu_dirty(v) ) + else if ( is_vcpu_dirty_cpu(dirty_cpu) ) { /* Remote CPU calls __sync_local_execstate() from flush IPI handler. */ - flush_mask(cpumask_of(v->dirty_cpu), FLUSH_VCPU_STATE); + flush_mask(cpumask_of(dirty_cpu), FLUSH_VCPU_STATE); } + ASSERT(!is_vcpu_dirty_cpu(dirty_cpu) || + read_atomic(&v->dirty_cpu) != dirty_cpu);Repeating my v1.1 comments:
"However, having stared at it for a while now - is this race
free? I can see this being fine in the (initial) case ofdirty_cpu==smp_processor_id(), but if this is for a foreign CPU, can't the vCPU have gone back to that same CPU again in the meantime?"and later
"There is a time window from late in
flush_mask()to the assertion
you add. All sorts of things can happen during this window on other CPUs. IOW what guarantees the vCPU not getting unpaused or its affinity getting changed yet another time?"You did reply that by what is now patch 2 this race can be eliminated, but I have to admit I don't see why this would be. Hence at the very least I'd expect justification in either the description or a code comment as to why there's no race left (and also no race to be expected to be re-introduced by code changes elsewhere - very unlikely races are, by their nature, unlikely to be hit during code development and the associated testing, hence I'd like there to be sufficiently close to a guarantee here).
My reservations here may in part be due to not following the reasoning for patch 2, which therefore I'll have to rely on the scheduler maintainers to judge on.
Quoted 8 lines
Quoted 3 lines
sync_vcpu_execstate()isn't called for a running or runnable vcpu any longer. I can add anASSERT()and a comment explaining it if you like that better.This would help (hopefully people adding new uses of the function would run into this assertion/comment), but for example the uses in
mapcache_current_vcpu()ordo_tasklet_work()look to be pretty hard to prove they can't happen for a runnable vCPU.
Those call sync_local_execstate(), not sync_vcpu_execstate().
Juergen
Re: [PATCH v2 3/3] xen/sched: fix latent races accessing vcpu->dirty_cpu
Quoted 55 lines · Jürgen Groß
Quoted 54 lines · Jan Beulich
Quoted 52 lines · Jürgen Groß
Quoted 44 lines · Jan Beulich
Quoted 7 lines · Juergen Gross
xen/arch/x86/domain.c +3
-1@@ -1956,13 +1958,17 @@ void sync_local_execstate(void) void sync_vcpu_execstate(struct vcpu *v) { - if ( v->dirty_cpu == smp_processor_id() ) + unsigned int dirty_cpu = read_atomic(&v->dirty_cpu); + + if ( dirty_cpu == smp_processor_id() )xen/arch/x86/domain.c void sync_local_execstate(void) · line 1970
Quoted 10 lines
xen/arch/x86/domain.c +4
-2sync_local_execstate(); - else if ( vcpu_cpu_dirty(v) ) + else if ( is_vcpu_dirty_cpu(dirty_cpu) ) { /* Remote CPU calls __sync_local_execstate() from flush IPI handler. */ - flush_mask(cpumask_of(v->dirty_cpu), FLUSH_VCPU_STATE); + flush_mask(cpumask_of(dirty_cpu), FLUSH_VCPU_STATE); } + ASSERT(!is_vcpu_dirty_cpu(dirty_cpu) || + read_atomic(&v->dirty_cpu) != dirty_cpu);Repeating my v1.1 comments:
"However, having stared at it for a while now - is this race free? I can see this being fine in the (initial) case of
dirty_cpu==smp_processor_id(), but if this is for a foreign CPU, can't the vCPU have gone back to that same CPU again in the meantime?"and later
"There is a time window from late in
flush_mask()to the assertion you add. All sorts of things can happen during this window on other CPUs. IOW what guarantees the vCPU not getting unpaused or its affinity getting changed yet another time?"You did reply that by what is now patch 2 this race can be eliminated, but I have to admit I don't see why this would be. Hence at the very least I'd expect justification in either the description or a code comment as to why there's no race left (and also no race to be expected to be re-introduced by code changes elsewhere - very unlikely races are, by their nature, unlikely to be hit during code development and the associated testing, hence I'd like there to be sufficiently close to a guarantee here).
Quoted 3 lines
My reservations here may in part be due to not following the reasoning for patch 2, which therefore I'll have to rely on the scheduler maintainers to judge on.
sync_vcpu_execstate()isn't called for a running or runnable vcpu any longer. I can add anASSERT()and a comment explaining it if you like that better.
Quoted 6 lines
Quoted 4 lines
This would help (hopefully people adding new uses of the function would run into this assertion/comment), but for example the uses in
mapcache_current_vcpu()ordo_tasklet_work()look to be pretty hard to prove they can't happen for a runnable vCPU.Those call
sync_local_execstate(), notsync_vcpu_execstate().
Ouch, as said on the other sub-thread - I'm sorry for mixing those up.
Jan