Skip to contentSkip to conversation

xen/sched: fix theoretical races accessing vcpu->dirty_cpu

PATCH In review Juergen Gross 3 participants 10 messages Apr 30, 2020 – May 4, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 2/3 xen/sched: fix theoretical races accessing vcpu->dirty_cpu

The dirty_cpu field of struct vcpu denotes which cpu still holds data of a vcpu. All accesses to this field should be atomic in case the vcpu could just be running, as it is accessed without any lock held in most cases.

There are some instances where accesses are not atomically done, and even worse where multiple accesses are done when a single one would be mandated.

Correct that in order to avoid potential problems.

Add some assertions to verify dirty_cpu is handled properly.

---
 xen/arch/x86/domain.c   | 14 ++++++++++----
 xen/include/xen/sched.h |  2 +-
 2 files changed, 11 insertions(+), 5 deletions(-)
xen/arch/x86/domain.c +10 -4
index a4428190d5..f0579a56d1 100644
@@ -1769,6 +1769,7 @@ static void __context_switch(void)
 
     if ( !is_idle_domain(pd) )
     {
+        ASSERT(read_atomic(&p->dirty_cpu) == cpu);
         memcpy(&p->arch.user_regs, stack_regs, CTXT_SWITCH_STACK_BYTES);
         vcpu_save_fpu(p);
         pd->arch.ctxt_switch->from(p);
@@ -1832,7 +1833,7 @@ void context_switch(struct vcpu *prev, struct vcpu *next)
 {
     unsigned int cpu = smp_processor_id();
     const struct domain *prevd = prev->domain, *nextd = next->domain;
-    unsigned int dirty_cpu = next->dirty_cpu;
+    unsigned int dirty_cpu = read_atomic(&next->dirty_cpu);
 
     ASSERT(prev != next);
     ASSERT(local_irq_is_enabled());
@@ -1844,6 +1845,7 @@ void context_switch(struct vcpu *prev, struct vcpu *next)
     {
         /* Remote CPU calls __sync_local_execstate() from flush IPI handler. */
         flush_mask(cpumask_of(dirty_cpu), FLUSH_VCPU_STATE);
+        ASSERT(read_atomic(&next->dirty_cpu) == VCPU_CPU_CLEAN);
     }
 
     _update_runstate_area(prev);
@@ -1956,13 +1958,17 @@ void sync_local_execstate(void)
 
 void sync_vcpu_execstate(struct vcpu *v)
 {
-    if ( v->dirty_cpu == smp_processor_id() )
+    unsigned int dirty_cpu = read_atomic(&v->dirty_cpu);
+
+    if ( dirty_cpu == smp_processor_id() )
         sync_local_execstate();
-    else if ( vcpu_cpu_dirty(v) )
+    else if ( is_vcpu_dirty_cpu(dirty_cpu) )
     {
         /* Remote CPU calls __sync_local_execstate() from flush IPI handler. */
-        flush_mask(cpumask_of(v->dirty_cpu), FLUSH_VCPU_STATE);
+        flush_mask(cpumask_of(dirty_cpu), FLUSH_VCPU_STATE);
     }
+    ASSERT(read_atomic(&v->dirty_cpu) != dirty_cpu ||
+           dirty_cpu == VCPU_CPU_CLEAN);
 }
 
 static int relinquish_memory(
xen/include/xen/sched.h +1 -1
index 195e7ee583..81628e2d98 100644
@@ -844,7 +844,7 @@ static inline bool is_vcpu_dirty_cpu(unsigned int cpu)
 
 static inline bool vcpu_cpu_dirty(const struct vcpu *v)
 {
-    return is_vcpu_dirty_cpu(v->dirty_cpu);
+    return is_vcpu_dirty_cpu(read_atomic((unsigned int *)&v->dirty_cpu));
 }
 
 void vcpu_block(void);
Signature · 1 line
2.16.4
#1 lore raw

Hi Juergen,

They are less theoritical than we would want. :/ There was a great series of article on lwn [1] about compiler optimization last year.

There is at least a few TOCTOU in the code where you could end up with cpumask_of(VCPU_CPU_CLEAN).

Quoted 4 lines · Juergen Gross

The dirty_cpu field of struct vcpu denotes which cpu still holds data of a vcpu. All accesses to this field should be atomic in case the vcpu could just be running, as it is accessed without any lock held in most cases.

Looking at the patch below, I am not sure why the issue is happening only when running. For instance, in the case of context_switch(), 'next' should not be running.

Instead, I think, the race would happen if the vCPU state is synchronized (__sync_local_execstate()) at the same time as time context_switch(). Am I correct?

Quoted 69 lines

There are some instances where accesses are not atomically done, and even worse where multiple accesses are done when a single one would be mandated.

Correct that in order to avoid potential problems.

Add some assertions to verify dirty_cpu is handled properly.

---
  xen/arch/x86/domain.c   | 14 ++++++++++----
  xen/include/xen/sched.h |  2 +-
  2 files changed, 11 insertions(+), 5 deletions(-)
xen/arch/x86/domain.c +10 -4
index a4428190d5..f0579a56d1 100644
@@ -1769,6 +1769,7 @@ static void __context_switch(void)
 
      if ( !is_idle_domain(pd) )
      {
+        ASSERT(read_atomic(&p->dirty_cpu) == cpu);
          memcpy(&p->arch.user_regs, stack_regs, CTXT_SWITCH_STACK_BYTES);
          vcpu_save_fpu(p);
          pd->arch.ctxt_switch->from(p);
@@ -1832,7 +1833,7 @@ void context_switch(struct vcpu *prev, struct vcpu *next)
  {
      unsigned int cpu = smp_processor_id();
      const struct domain *prevd = prev->domain, *nextd = next->domain;
-    unsigned int dirty_cpu = next->dirty_cpu;
+    unsigned int dirty_cpu = read_atomic(&next->dirty_cpu);
 
      ASSERT(prev != next);
      ASSERT(local_irq_is_enabled());
@@ -1844,6 +1845,7 @@ void context_switch(struct vcpu *prev, struct vcpu *next)
      {
          /* Remote CPU calls __sync_local_execstate() from flush IPI handler. */
          flush_mask(cpumask_of(dirty_cpu), FLUSH_VCPU_STATE);
+        ASSERT(read_atomic(&next->dirty_cpu) == VCPU_CPU_CLEAN);
      }
 
      _update_runstate_area(prev);
@@ -1956,13 +1958,17 @@ void sync_local_execstate(void)
 
  void sync_vcpu_execstate(struct vcpu *v)
  {
-    if ( v->dirty_cpu == smp_processor_id() )
+    unsigned int dirty_cpu = read_atomic(&v->dirty_cpu);
+
+    if ( dirty_cpu == smp_processor_id() )
          sync_local_execstate();
-    else if ( vcpu_cpu_dirty(v) )
+    else if ( is_vcpu_dirty_cpu(dirty_cpu) )
      {
          /* Remote CPU calls __sync_local_execstate() from flush IPI handler. */
-        flush_mask(cpumask_of(v->dirty_cpu), FLUSH_VCPU_STATE);
+        flush_mask(cpumask_of(dirty_cpu), FLUSH_VCPU_STATE);
      }
+    ASSERT(read_atomic(&v->dirty_cpu) != dirty_cpu ||
+           dirty_cpu == VCPU_CPU_CLEAN); >   }
 
  static int relinquish_memory(
xen/include/xen/sched.h +0 -0
index 195e7ee583..81628e2d98 100644

xen/include/xen/sched.h static inline bool is_vcpu_dirty_cpu(unsigned int cpu) · line 847

Quoted 5 lines
xen/include/xen/sched.h +1 -1
@@ -844,7 +844,7 @@ static inline bool is_vcpu_dirty_cpu(unsigned int cpu)
 
  static inline bool vcpu_cpu_dirty(const struct vcpu *v)
  {
-    return is_vcpu_dirty_cpu(v->dirty_cpu);
+    return is_vcpu_dirty_cpu(read_atomic((unsigned int *)&v->dirty_cpu));

Is the cast necessary?

xen/include/xen/sched.h static inline bool is_vcpu_dirty_cpu(unsigned int cpu) · line 850

Quoted 4 lines
xen/include/xen/sched.h +0 -0
  }
 
  void vcpu_block(void);

Cheers,

[1] https://lwn.net/Articles/793253/

Signature · 1 line
Julien Grall
#2 lore raw
Quoted 5 lines · Julien Grall

Hi Juergen,

They are less theoritical than we would want. :/ There was a great series of article on lwn [1] about compiler optimization last year.

There is at least a few TOCTOU in the code where you could end up with cpumask_of(VCPU_CPU_CLEAN).

It is theoretical in the sense that I don't know of any failure resulting due to this.

Quoted 10 lines · Juergen Gross
Quoted 4 lines

The dirty_cpu field of struct vcpu denotes which cpu still holds data of a vcpu. All accesses to this field should be atomic in case the vcpu could just be running, as it is accessed without any lock held in most cases.

Looking at the patch below, I am not sure why the issue is happening only when running. For instance, in the case of context_switch(), 'next' should not be running.

Instead, I think, the race would happen if the vCPU state is synchronized (__sync_local_execstate()) at the same time as time context_switch(). Am I correct?

Yes.

Quoted 67 lines
Quoted 67 lines

There are some instances where accesses are not atomically done, and even worse where multiple accesses are done when a single one would be mandated.

Correct that in order to avoid potential problems.

Add some assertions to verify dirty_cpu is handled properly.

---
  xen/arch/x86/domain.c   | 14 ++++++++++----
  xen/include/xen/sched.h |  2 +-

  2 files changed, 11 insertions(+), 5 deletions(-)
xen/arch/x86/domain.c +10 -4
index a4428190d5..f0579a56d1 100644
@@ -1769,6 +1769,7 @@ static void __context_switch(void)
       if ( !is_idle_domain(pd) )
       {
+        ASSERT(read_atomic(&p->dirty_cpu) == cpu);
           memcpy(&p->arch.user_regs, stack_regs, CTXT_SWITCH_STACK_BYTES);
           vcpu_save_fpu(p);
           pd->arch.ctxt_switch->from(p);
@@ -1832,7 +1833,7 @@ void context_switch(struct vcpu *prev, struct 
 vcpu *next)
   {
       unsigned int cpu = smp_processor_id();
       const struct domain *prevd = prev->domain, *nextd = next->domain;
-    unsigned int dirty_cpu = next->dirty_cpu;
+    unsigned int dirty_cpu = read_atomic(&next->dirty_cpu);
       ASSERT(prev != next);
       ASSERT(local_irq_is_enabled());
@@ -1844,6 +1845,7 @@ void context_switch(struct vcpu *prev, struct 
 vcpu *next)
       {
           /* Remote CPU calls __sync_local_execstate() from flush IPI handler. */
           flush_mask(cpumask_of(dirty_cpu), FLUSH_VCPU_STATE);
+        ASSERT(read_atomic(&next->dirty_cpu) == VCPU_CPU_CLEAN);
       }
       _update_runstate_area(prev);
@@ -1956,13 +1958,17 @@ void sync_local_execstate(void)
   void sync_vcpu_execstate(struct vcpu *v)
   {
-    if ( v->dirty_cpu == smp_processor_id() )
+    unsigned int dirty_cpu = read_atomic(&v->dirty_cpu);
+
+    if ( dirty_cpu == smp_processor_id() )
           sync_local_execstate();
-    else if ( vcpu_cpu_dirty(v) )
+    else if ( is_vcpu_dirty_cpu(dirty_cpu) )
       {
           /* Remote CPU calls __sync_local_execstate() from flush IPI handler. */
-        flush_mask(cpumask_of(v->dirty_cpu), FLUSH_VCPU_STATE);
+        flush_mask(cpumask_of(dirty_cpu), FLUSH_VCPU_STATE);
       }
+    ASSERT(read_atomic(&v->dirty_cpu) != dirty_cpu ||
+           dirty_cpu == VCPU_CPU_CLEAN); >   }

  static int relinquish_memory(

xen/include/xen/sched.h +0 -0
index 195e7ee583..81628e2d98 100644
@@ -844,7 +844,7 @@ static inline bool is_vcpu_dirty_cpu(unsigned int 
Quoted 8 lines

xen/include/xen/sched.h static inline bool is_vcpu_dirty_cpu(unsigned int cpu)

Quoted 6 lines
xen/include/xen/sched.h +1 -1
 cpu)
   static inline bool vcpu_cpu_dirty(const struct vcpu *v)
   {
-    return is_vcpu_dirty_cpu(v->dirty_cpu);
+    return is_vcpu_dirty_cpu(read_atomic((unsigned int 
 *)&v->dirty_cpu));

Is the cast necessary?

Yes, that was the problem when building for ARM I encountered.

read_atomic() on ARM has a local variable of the same type as the read_atomic() parameter for storing the result. Due to the const attribute of v this results in assignment to a read-only variable.

Juergen

#3 lore raw

Hi,

Quoted 9 lines · Jürgen Groß
Quoted 5 lines · Julien Grall

Hi Juergen,

They are less theoritical than we would want. :/ There was a great series of article on lwn [1] about compiler optimization last year.

There is at least a few TOCTOU in the code where you could end up with cpumask_of(VCPU_CPU_CLEAN).

It is theoretical in the sense that I don't know of any failure resulting due to this.

How about "latent" instead of "theoritical"?

Quoted 8 lines
Quoted 8 lines · Juergen Gross
Quoted 4 lines

The dirty_cpu field of struct vcpu denotes which cpu still holds data of a vcpu. All accesses to this field should be atomic in case the vcpu could just be running, as it is accessed without any lock held in most cases.

Quoted 5 lines
Quoted 3 lines

Looking at the patch below, I am not sure why the issue is happening only when running. For instance, in the case of context_switch(), 'next' should not be running.

Instead, I think, the race would happen if the vCPU state is synchronized (__sync_local_execstate()) at the same time as time context_switch(). Am I correct?

Yes.

Would you mind adding this context in the commit message?

[...]

Quoted 14 lines
Quoted 5 lines
Quoted 5 lines
xen/include/xen/sched.h +0 -0
index 195e7ee583..81628e2d98 100644
@@ -844,7 +844,7 @@ static inline bool is_vcpu_dirty_cpu(unsigned int 
Quoted 8 lines

xen/include/xen/sched.h static inline bool is_vcpu_dirty_cpu(unsigned int cpu)

Quoted 6 lines
xen/include/xen/sched.h +1 -1
 cpu)
   static inline bool vcpu_cpu_dirty(const struct vcpu *v)
   {
-    return is_vcpu_dirty_cpu(v->dirty_cpu);
+    return is_vcpu_dirty_cpu(read_atomic((unsigned int 
 *)&v->dirty_cpu));

Is the cast necessary?

Quoted 5 lines

Yes, that was the problem when building for ARM I encountered.

read_atomic() on ARM has a local variable of the same type as the read_atomic() parameter for storing the result. Due to the const attribute of v this results in assignment to a read-only variable.

Doh, we should be able to read from a const value without. So I would argue this is a bug in the read_atomic() implementation on Arm. I will try to come up with a patch.

Cheers,

Signature · 1 line
Julien Grall
#4 lore raw
Quoted 14 lines · Julien Grall
Quoted 14 lines
Quoted 13 lines

xen/include/xen/sched.h static inline bool is_vcpu_dirty_cpu(unsigned · line 848

Quoted 11 lines
xen/include/xen/sched.h +1 -1
index 195e7ee583..81628e2d98 100644
@@ -844,7 +844,7 @@ static inline bool is_vcpu_dirty_cpu(unsigned 
 int cpu)
   static inline bool vcpu_cpu_dirty(const struct vcpu *v)
   {
-    return is_vcpu_dirty_cpu(v->dirty_cpu);
+    return is_vcpu_dirty_cpu(read_atomic((unsigned int 
 *)&v->dirty_cpu));

Is the cast necessary?

Quoted 7 lines
Quoted 5 lines

Yes, that was the problem when building for ARM I encountered.

read_atomic() on ARM has a local variable of the same type as the read_atomic() parameter for storing the result. Due to the const attribute of v this results in assignment to a read-only variable.

Doh, we should be able to read from a const value without. So I would argue this is a bug in the read_atomic() implementation on Arm. I will try to come up with a patch.

I have just sent a series [1] to address the issue reported here and a few more.

Cheers,

[1] <[email protected]>

Signature · 1 line
Julien Grall
#5 lore raw
Quoted 19 lines · Julien Grall
Quoted 16 lines · Julien Grall
Quoted 16 lines
Quoted 5 lines
Quoted 5 lines
xen/include/xen/sched.h +0 -0
index 195e7ee583..81628e2d98 100644
@@ -844,7 +844,7 @@ static inline bool is_vcpu_dirty_cpu(unsigned 
Quoted 8 lines

xen/include/xen/sched.h static inline bool is_vcpu_dirty_cpu(unsigned int cpu)

Quoted 6 lines
xen/include/xen/sched.h +1 -1
 int cpu)
   static inline bool vcpu_cpu_dirty(const struct vcpu *v)
   {
-    return is_vcpu_dirty_cpu(v->dirty_cpu);
+    return is_vcpu_dirty_cpu(read_atomic((unsigned int 
 *)&v->dirty_cpu));

Is the cast necessary?

Yes, that was the problem when building for ARM I encountered.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

read_atomic() on ARM has a local variable of the same type as the read_atomic() parameter for storing the result. Due to the const attribute of v this results in assignment to a read-only variable.

Doh, we should be able to read from a const value without. So I would argue this is a bug in the read_atomic() implementation on Arm. I will try to come up with a patch.

I have just sent a series [1] to address the issue reported here and a few more.

With that series V1 of this patch is fine again. :-)

Juergen

#6 lore raw
In reply to Juergen Gross
Quoted 10 lines · Juergen Gross

The dirty_cpu field of struct vcpu denotes which cpu still holds data of a vcpu. All accesses to this field should be atomic in case the vcpu could just be running, as it is accessed without any lock held in most cases.

There are some instances where accesses are not atomically done, and even worse where multiple accesses are done when a single one would be mandated.

Correct that in order to avoid potential problems.

Beyond the changes you're making, what about the assignment in startup_cpu_idle_loop()? And while less important, dump_domains() also has a use that I think would better be converted for completeness.

xen/arch/x86/domain.c void context_switch(struct vcpu *prev, struct vcpu *next) · line 1848

Quoted 5 lines
xen/arch/x86/domain.c +1 -0
@@ -1844,6 +1845,7 @@ void context_switch(struct vcpu *prev, struct vcpu *next)
     {
         /* Remote CPU calls __sync_local_execstate() from flush IPI handler. */
         flush_mask(cpumask_of(dirty_cpu), FLUSH_VCPU_STATE);
+        ASSERT(read_atomic(&next->dirty_cpu) == VCPU_CPU_CLEAN);

ASSERT(!is_vcpu_dirty_cpu(read_atomic(&next->dirty_cpu))) ?

Quoted 8 lines
xen/arch/x86/domain.c +3 -1
@@ -1956,13 +1958,17 @@ void sync_local_execstate(void)
 
 void sync_vcpu_execstate(struct vcpu *v)
 {
-    if ( v->dirty_cpu == smp_processor_id() )
+    unsigned int dirty_cpu = read_atomic(&v->dirty_cpu);
+
+    if ( dirty_cpu == smp_processor_id() )

xen/arch/x86/domain.c void sync_local_execstate(void) · line 1971

Quoted 10 lines
xen/arch/x86/domain.c +4 -2
         sync_local_execstate();
-    else if ( vcpu_cpu_dirty(v) )
+    else if ( is_vcpu_dirty_cpu(dirty_cpu) )
     {
         /* Remote CPU calls __sync_local_execstate() from flush IPI handler. */
-        flush_mask(cpumask_of(v->dirty_cpu), FLUSH_VCPU_STATE);
+        flush_mask(cpumask_of(dirty_cpu), FLUSH_VCPU_STATE);
     }
+    ASSERT(read_atomic(&v->dirty_cpu) != dirty_cpu ||
+           dirty_cpu == VCPU_CPU_CLEAN);

!is_vcpu_dirty_cpu(dirty_cpu) again? Also perhaps flip both sides of the || (to have the cheaper one first), and maybe

if ( is_vcpu_dirty_cpu(dirty_cpu) )
ASSERT(read_atomic(&v->dirty_cpu) != dirty_cpu);

as the longer assertion string literal isn't really of that much extra value.

However, having stared at it for a while now - is this race free? I can see this being fine in the (initial) case of dirty_cpu == smp_processor_id(), but if this is for a foreign CPU, can't the vCPU have gone back to that same CPU again in the meantime?

xen/include/xen/sched.h static inline bool is_vcpu_dirty_cpu(unsigned int cpu) · line 847

Quoted 8 lines
xen/include/xen/sched.h +1 -1
@@ -844,7 +844,7 @@ static inline bool is_vcpu_dirty_cpu(unsigned int cpu)
 
 static inline bool vcpu_cpu_dirty(const struct vcpu *v)
 {
-    return is_vcpu_dirty_cpu(v->dirty_cpu);
+    return is_vcpu_dirty_cpu(read_atomic((unsigned int *)&v->dirty_cpu));

As per your communication with Julien I understand the cast will go away again.

Jan

#7 lore raw
Quoted 10 lines · Jan Beulich
Quoted 9 lines · Juergen Gross

The dirty_cpu field of struct vcpu denotes which cpu still holds data of a vcpu. All accesses to this field should be atomic in case the vcpu could just be running, as it is accessed without any lock held in most cases.

There are some instances where accesses are not atomically done, and even worse where multiple accesses are done when a single one would be mandated.

Quoted 6 lines
Quoted 1 line

Correct that in order to avoid potential problems.

Beyond the changes you're making, what about the assignment in startup_cpu_idle_loop()? And while less important, dump_domains() also has a use that I think would better be converted for completeness.

startup_cpu_idle_loop() is not important, as it is set before any scheduling activity might occur on a cpu. But I can change both instances.

Quoted 8 lines

xen/arch/x86/domain.c void context_switch(struct vcpu *prev, struct vcpu *next) · line 1848

Quoted 6 lines
xen/arch/x86/domain.c +1 -0
@@ -1844,6 +1845,7 @@ void context_switch(struct vcpu *prev, struct vcpu *next)
      {
          /* Remote CPU calls __sync_local_execstate() from flush IPI handler. */
          flush_mask(cpumask_of(dirty_cpu), FLUSH_VCPU_STATE);
+        ASSERT(read_atomic(&next->dirty_cpu) == VCPU_CPU_CLEAN);

ASSERT(!is_vcpu_dirty_cpu(read_atomic(&next->dirty_cpu))) ?

Yes, this is better.

Quoted 23 lines
Quoted 9 lines
xen/arch/x86/domain.c +3 -1
@@ -1956,13 +1958,17 @@ void sync_local_execstate(void)
 
  void sync_vcpu_execstate(struct vcpu *v)
  {
-    if ( v->dirty_cpu == smp_processor_id() )
+    unsigned int dirty_cpu = read_atomic(&v->dirty_cpu);
+
+    if ( dirty_cpu == smp_processor_id() )

xen/arch/x86/domain.c void sync_local_execstate(void) · line 1971

Quoted 10 lines
xen/arch/x86/domain.c +4 -2
          sync_local_execstate();
-    else if ( vcpu_cpu_dirty(v) )
+    else if ( is_vcpu_dirty_cpu(dirty_cpu) )
      {
          /* Remote CPU calls __sync_local_execstate() from flush IPI handler. */
-        flush_mask(cpumask_of(v->dirty_cpu), FLUSH_VCPU_STATE);
+        flush_mask(cpumask_of(dirty_cpu), FLUSH_VCPU_STATE);
      }
+    ASSERT(read_atomic(&v->dirty_cpu) != dirty_cpu ||
+           dirty_cpu == VCPU_CPU_CLEAN);

!is_vcpu_dirty_cpu(dirty_cpu) again? Also perhaps flip both sides of the || (to have the cheaper one first), and maybe

Quoted 5 lines

if ( is_vcpu_dirty_cpu(dirty_cpu) )
ASSERT(read_atomic(&v->dirty_cpu) != dirty_cpu);

as the longer assertion string literal isn't really of that much extra value.

I can do that, in case we can be sure the compiler will drop the test in case of a non-debug build.

Quoted 6 lines

However, having stared at it for a while now - is this race free? I can see this being fine in the (initial) case of dirty_cpu == smp_processor_id(), but if this is for a foreign CPU, can't the vCPU have gone back to that same CPU again in the meantime?

This should never happen. Either the vcpu in question is paused, or it has been forced off the cpu due to not being allowed to run there (e.g. affinity has been changed, or cpu is about to be removed from cpupool). I can add a comment explaining it.

Quoted 5 lines
Quoted 5 lines
xen/include/xen/sched.h +0 -0
@@ -844,7 +844,7 @@ static inline bool is_vcpu_dirty_cpu(unsigned int cpu)
 
Quoted 7 lines

xen/include/xen/sched.h static inline bool is_vcpu_dirty_cpu(unsigned int cpu) · line 847

Quoted 4 lines
xen/include/xen/sched.h +1 -1
  static inline bool vcpu_cpu_dirty(const struct vcpu *v)
  {
-    return is_vcpu_dirty_cpu(v->dirty_cpu);
+    return is_vcpu_dirty_cpu(read_atomic((unsigned int *)&v->dirty_cpu));

As per your communication with Julien I understand the cast will go away again.

Yes, I think so.

Juergen

#8 lore raw
Quoted 23 lines · Jürgen Groß
Quoted 22 lines · Jan Beulich
Quoted 7 lines · Juergen Gross
xen/arch/x86/domain.c +3 -1
@@ -1956,13 +1958,17 @@ void sync_local_execstate(void)
     void sync_vcpu_execstate(struct vcpu *v)
   {
-    if ( v->dirty_cpu == smp_processor_id() )
+    unsigned int dirty_cpu = read_atomic(&v->dirty_cpu);
+
+    if ( dirty_cpu == smp_processor_id() )

xen/arch/x86/domain.c void sync_local_execstate(void) · line 1970

Quoted 10 lines
xen/arch/x86/domain.c +4 -2
           sync_local_execstate();
-    else if ( vcpu_cpu_dirty(v) )
+    else if ( is_vcpu_dirty_cpu(dirty_cpu) )
       {
           /* Remote CPU calls __sync_local_execstate() from flush IPI handler. */
-        flush_mask(cpumask_of(v->dirty_cpu), FLUSH_VCPU_STATE);
+        flush_mask(cpumask_of(dirty_cpu), FLUSH_VCPU_STATE);
       }
+    ASSERT(read_atomic(&v->dirty_cpu) != dirty_cpu ||
+           dirty_cpu == VCPU_CPU_CLEAN);

!is_vcpu_dirty_cpu(dirty_cpu) again? Also perhaps flip both sides of the || (to have the cheaper one first), and maybe

Quoted 8 lines
Quoted 5 lines

     if ( is_vcpu_dirty_cpu(dirty_cpu) ) ASSERT(read_atomic(&v->dirty_cpu) != dirty_cpu);

as the longer assertion string literal isn't really of that much extra value.

I can do that, in case we can be sure the compiler will drop the test in case of a non-debug build.

Modern gcc will afaik; no idea about clang though.

Quoted 10 lines
Quoted 5 lines

However, having stared at it for a while now - is this race free? I can see this being fine in the (initial) case of dirty_cpu == smp_processor_id(), but if this is for a foreign CPU, can't the vCPU have gone back to that same CPU again in the meantime?

This should never happen. Either the vcpu in question is paused, or it has been forced off the cpu due to not being allowed to run there (e.g. affinity has been changed, or cpu is about to be removed from cpupool). I can add a comment explaining it.

There is a time window from late in flush_mask() to the assertion you add. All sorts of things can happen during this window on other CPUs. IOW what guarantees the vCPU not getting unpaused or its affinity getting changed yet another time?

Jan

#9 lore raw
Quoted 27 lines · Jan Beulich
Quoted 26 lines · Jürgen Groß
Quoted 25 lines · Jan Beulich
Quoted 7 lines · Juergen Gross
xen/arch/x86/domain.c +3 -1
@@ -1956,13 +1958,17 @@ void sync_local_execstate(void)
     void sync_vcpu_execstate(struct vcpu *v)
   {
-    if ( v->dirty_cpu == smp_processor_id() )
+    unsigned int dirty_cpu = read_atomic(&v->dirty_cpu);
+
+    if ( dirty_cpu == smp_processor_id() )

xen/arch/x86/domain.c void sync_local_execstate(void) · line 1970

Quoted 10 lines
xen/arch/x86/domain.c +4 -2
           sync_local_execstate();
-    else if ( vcpu_cpu_dirty(v) )
+    else if ( is_vcpu_dirty_cpu(dirty_cpu) )
       {
           /* Remote CPU calls __sync_local_execstate() from flush IPI handler. */
-        flush_mask(cpumask_of(v->dirty_cpu), FLUSH_VCPU_STATE);
+        flush_mask(cpumask_of(dirty_cpu), FLUSH_VCPU_STATE);
       }
+    ASSERT(read_atomic(&v->dirty_cpu) != dirty_cpu ||
+           dirty_cpu == VCPU_CPU_CLEAN);

!is_vcpu_dirty_cpu(dirty_cpu) again? Also perhaps flip both sides of the || (to have the cheaper one first), and maybe

     if ( is_vcpu_dirty_cpu(dirty_cpu) ) ASSERT(read_atomic(&v->dirty_cpu) != dirty_cpu);

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

as the longer assertion string literal isn't really of that much extra value.

I can do that, in case we can be sure the compiler will drop the test in case of a non-debug build.

Modern gcc will afaik; no idea about clang though.

I'll go with both tests inside the ASSERT(), as this will hurt debug build only.

Quoted 12 lines
Quoted 11 lines
Quoted 6 lines

However, having stared at it for a while now - is this race free? I can see this being fine in the (initial) case of dirty_cpu == smp_processor_id(), but if this is for a foreign CPU, can't the vCPU have gone back to that same CPU again in the meantime?

This should never happen. Either the vcpu in question is paused, or it has been forced off the cpu due to not being allowed to run there (e.g. affinity has been changed, or cpu is about to be removed from cpupool). I can add a comment explaining it.

Quoted 4 lines

There is a time window from late in flush_mask() to the assertion you add. All sorts of things can happen during this window on other CPUs. IOW what guarantees the vCPU not getting unpaused or its affinity getting changed yet another time?

Hmm, very unlikely, but not impossible (especially in nested virt case).

This makes me wonder whether adding the call to sync_vcpu_execstate() to sched_unit_migrate_finish() was really a good idea. There might be cases where it is not necessary (e.g. in the case you are mentioning, but with a much larger time window), or where it is more expensive than doing it the lazy way.

Dropping this call of sync_vcpu_execstate() will remove the race you are mentioning completely.

Juergen