Skip to contentSkip to conversation

x86/hap: be more selective with assisted TLB flush

PATCH In review Roger Pau Monne 3 participants 6 messages Apr 29, 2020 – Apr 30, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86/hap: be more selective with assisted TLB flush

When doing an assisted flush on HAP the purpose of the on_selected_cpus is just to trigger a vmexit on remote CPUs that are in guest context, and hence just using is_vcpu_dirty_cpu is too lax, also check that the vCPU is running.

While there also pass NULL as the data parameter of on_selected_cpus, the dummy handler doesn't consume the data in any way.

---
 xen/arch/x86/mm/hap/hap.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
xen/arch/x86/mm/hap/hap.c +2 -2
index 580d1c2164..0275cdf5c8 100644
@@ -719,7 +719,7 @@ static bool flush_tlb(bool (*flush_vcpu)(void *ctxt, struct vcpu *v),
         hvm_asid_flush_vcpu(v);
 
         cpu = read_atomic(&v->dirty_cpu);
-        if ( cpu != this_cpu && is_vcpu_dirty_cpu(cpu) )
+        if ( cpu != this_cpu && is_vcpu_dirty_cpu(cpu) && v->is_running )
             __cpumask_set_cpu(cpu, mask);
     }
 
@@ -729,7 +729,7 @@ static bool flush_tlb(bool (*flush_vcpu)(void *ctxt, struct vcpu *v),
      * not currently running will already be flushed when scheduled because of
      * the ASID tickle done in the loop above.
      */
-    on_selected_cpus(mask, dummy_flush, mask, 0);
+    on_selected_cpus(mask, dummy_flush, NULL, 0);
 
     return true;
 }
Signature · 1 line
2.26.0
Reviewed-by on v1 1/1
#1 lore raw
Quoted 4 lines · Roger Pau Monne

When doing an assisted flush on HAP the purpose of the on_selected_cpus is just to trigger a vmexit on remote CPUs that are in guest context, and hence just using is_vcpu_dirty_cpu is too lax, also check that the vCPU is running.

Am I right to understand that the change is relevant only to cover the period of time between ->is_running becoming false and ->dirty_cpu becoming VCPU_CPU_CLEAN? I.e. ...

xen/arch/x86/mm/hap/hap.c static bool flush_tlb(bool (*flush_vcpu)(void *ctxt, struct vcpu *v), · line 722

Quoted 8 lines
xen/arch/x86/mm/hap/hap.c +1 -1
@@ -719,7 +719,7 @@ static bool flush_tlb(bool (*flush_vcpu)(void *ctxt, struct vcpu *v),
         hvm_asid_flush_vcpu(v);
 
         cpu = read_atomic(&v->dirty_cpu);
-        if ( cpu != this_cpu && is_vcpu_dirty_cpu(cpu) )
+        if ( cpu != this_cpu && is_vcpu_dirty_cpu(cpu) && v->is_running )

... the previous logic would have suitably covered the switch-to path, but doesn't properly cover the switch-from one, due to our lazy context switch approach? If so, I agree with the change:

It might be worth mentioning this detail in the description then, though.

Jan

#2 lore raw
Quoted 15 lines · Jan Beulich
Quoted 4 lines · Roger Pau Monne

When doing an assisted flush on HAP the purpose of the on_selected_cpus is just to trigger a vmexit on remote CPUs that are in guest context, and hence just using is_vcpu_dirty_cpu is too lax, also check that the vCPU is running.

Am I right to understand that the change is relevant only to cover the period of time between ->is_running becoming false and ->dirty_cpu becoming VCPU_CPU_CLEAN? I.e. ...

Quoted 5 lines
xen/arch/x86/mm/hap/hap.c +0 -0
@@ -719,7 +719,7 @@ static bool flush_tlb(bool (*flush_vcpu)(void *ctxt, struct vcpu *v),
         hvm_asid_flush_vcpu(v);
 
Quoted 7 lines

xen/arch/x86/mm/hap/hap.c static bool flush_tlb(bool (*flush_vcpu)(void *ctxt, struct vcpu *v), · line 722

Quoted 3 lines
xen/arch/x86/mm/hap/hap.c +1 -1
         cpu = read_atomic(&v->dirty_cpu);
-        if ( cpu != this_cpu && is_vcpu_dirty_cpu(cpu) )
+        if ( cpu != this_cpu && is_vcpu_dirty_cpu(cpu) && v->is_running )

... the previous logic would have suitably covered the switch-to path, but doesn't properly cover the switch-from one, due to our lazy context switch approach?

Yes. Also __context_switch is not called from context_switch when switching to the idle vcpu, and hence dirty_cpu is not cleared.

Quoted 4 lines

If so, I agree with the change:

It might be worth mentioning this detail in the description then, though.

Would you mind adding to the commit message if you agree:

"Due to the lazy context switching done by Xen dirty_cpu won't always be cleared when the guest vCPU is not running, and hence relying on is_running allows more fine grained control of whether the vCPU is actually running."

Thanks, Roger.

#3 lore raw
Quoted 32 lines · Roger Pau Monné
Quoted 15 lines · Jan Beulich
Quoted 4 lines · Roger Pau Monne

When doing an assisted flush on HAP the purpose of the on_selected_cpus is just to trigger a vmexit on remote CPUs that are in guest context, and hence just using is_vcpu_dirty_cpu is too lax, also check that the vCPU is running.

Am I right to understand that the change is relevant only to cover the period of time between ->is_running becoming false and ->dirty_cpu becoming VCPU_CPU_CLEAN? I.e. ...

Quoted 5 lines
xen/arch/x86/mm/hap/hap.c +0 -0
@@ -719,7 +719,7 @@ static bool flush_tlb(bool (*flush_vcpu)(void *ctxt, struct vcpu *v),
         hvm_asid_flush_vcpu(v);
 
Quoted 7 lines

xen/arch/x86/mm/hap/hap.c static bool flush_tlb(bool (*flush_vcpu)(void *ctxt, struct vcpu *v), · line 722

Quoted 3 lines
xen/arch/x86/mm/hap/hap.c +1 -1
         cpu = read_atomic(&v->dirty_cpu);
-        if ( cpu != this_cpu && is_vcpu_dirty_cpu(cpu) )
+        if ( cpu != this_cpu && is_vcpu_dirty_cpu(cpu) && v->is_running )

... the previous logic would have suitably covered the switch-to path, but doesn't properly cover the switch-from one, due to our lazy context switch approach?

Yes. Also __context_switch is not called from context_switch when switching to the idle vcpu, and hence dirty_cpu is not cleared.

Quoted 4 lines

If so, I agree with the change:

It might be worth mentioning this detail in the description then, though.

Quoted 6 lines

Would you mind adding to the commit message if you agree:

"Due to the lazy context switching done by Xen dirty_cpu won't always be cleared when the guest vCPU is not running, and hence relying on is_running allows more fine grained control of whether the vCPU is actually running."

Sure; I'll give it over the weekend though for others to comment, if so desired.

Jan

#4 lore raw
Quoted 29 lines · Jan Beulich
Quoted 28 lines · Roger Pau Monné
Quoted 14 lines · Jan Beulich
Quoted 4 lines · Roger Pau Monne

When doing an assisted flush on HAP the purpose of the on_selected_cpus is just to trigger a vmexit on remote CPUs that are in guest context, and hence just using is_vcpu_dirty_cpu is too lax, also check that the vCPU is running.

Am I right to understand that the change is relevant only to cover the period of time between ->is_running becoming false and ->dirty_cpu becoming VCPU_CPU_CLEAN? I.e. ...

Quoted 5 lines
xen/arch/x86/mm/hap/hap.c +0 -0
@@ -719,7 +719,7 @@ static bool flush_tlb(bool (*flush_vcpu)(void *ctxt, struct vcpu *v),
         hvm_asid_flush_vcpu(v);
 
Quoted 6 lines

xen/arch/x86/mm/hap/hap.c static bool flush_tlb(bool (*flush_vcpu)(void *ctxt, struct vcpu *v), · line 722

Quoted 3 lines
xen/arch/x86/mm/hap/hap.c +1 -1
         cpu = read_atomic(&v->dirty_cpu);
-        if ( cpu != this_cpu && is_vcpu_dirty_cpu(cpu) )
+        if ( cpu != this_cpu && is_vcpu_dirty_cpu(cpu) && v->is_running )

... the previous logic would have suitably covered the switch-to path, but doesn't properly cover the switch-from one, due to our lazy context switch approach?

Yes. Also __context_switch is not called from context_switch when switching to the idle vcpu, and hence dirty_cpu is not cleared.

Quoted 4 lines

If so, I agree with the change:

It might be worth mentioning this detail in the description then, though.

Quoted 8 lines
Quoted 6 lines

Would you mind adding to the commit message if you agree:

"Due to the lazy context switching done by Xen dirty_cpu won't always be cleared when the guest vCPU is not running, and hence relying on is_running allows more fine grained control of whether the vCPU is actually running."

Sure; I'll give it over the weekend though for others to comment, if so desired.

I think it is worth pointing out that this fixes a large perf regression on Nehalem/Westmere systems, where L1 Shim using the enlightened hypercall is 8x slower than unenlightened way.

~Andrew

#5 lore raw
Quoted 36 lines · Andrew Cooper
Quoted 35 lines · Jan Beulich
Quoted 34 lines · Roger Pau Monné
Quoted 14 lines · Jan Beulich
Quoted 4 lines · Roger Pau Monne

When doing an assisted flush on HAP the purpose of the on_selected_cpus is just to trigger a vmexit on remote CPUs that are in guest context, and hence just using is_vcpu_dirty_cpu is too lax, also check that the vCPU is running.

Am I right to understand that the change is relevant only to cover the period of time between ->is_running becoming false and ->dirty_cpu becoming VCPU_CPU_CLEAN? I.e. ...

Quoted 5 lines
xen/arch/x86/mm/hap/hap.c +0 -0
@@ -719,7 +719,7 @@ static bool flush_tlb(bool (*flush_vcpu)(void *ctxt, struct vcpu *v),
         hvm_asid_flush_vcpu(v);
 
Quoted 6 lines

xen/arch/x86/mm/hap/hap.c static bool flush_tlb(bool (*flush_vcpu)(void *ctxt, struct vcpu *v), · line 722

Quoted 3 lines
xen/arch/x86/mm/hap/hap.c +1 -1
         cpu = read_atomic(&v->dirty_cpu);
-        if ( cpu != this_cpu && is_vcpu_dirty_cpu(cpu) )
+        if ( cpu != this_cpu && is_vcpu_dirty_cpu(cpu) && v->is_running )

... the previous logic would have suitably covered the switch-to path, but doesn't properly cover the switch-from one, due to our lazy context switch approach?

Yes. Also __context_switch is not called from context_switch when switching to the idle vcpu, and hence dirty_cpu is not cleared.

Quoted 4 lines

If so, I agree with the change:

It might be worth mentioning this detail in the description then, though.

Would you mind adding to the commit message if you agree:

"Due to the lazy context switching done by Xen dirty_cpu won't always be cleared when the guest vCPU is not running, and hence relying on is_running allows more fine grained control of whether the vCPU is actually running."

Quoted 6 lines
Quoted 2 lines

Sure; I'll give it over the weekend though for others to comment, if so desired.

I think it is worth pointing out that this fixes a large perf regression on Nehalem/Westmere systems, where L1 Shim using the enlightened hypercall is 8x slower than unenlightened way.

I might as well post the actual numbers I have.

I've measured the time of the non-local branch of flush_area_mask inside the shim running with 32vCPUs over 100000 executions and averaged the result on a large Westmere system (80 ways total). The figures where fetched during the boot of a SLES 11 PV guest. The results are as follow (less is better):

Non assisted flush with x2APIC: 112406ns
Assisted flush without this patch: 820450ns
Assisted flush with this patch: 8330ns

I can add the figures to the commit message if deemed interesting to have in the repo. Or the above text can be appended to the commit message if that's fine.

Roger.