Skip to contentSkip to conversation

xen: fix logical error in tlb flushing

PATCH In review Alex Shi 4 participants 8 messages Aug 24, 2012 – Sep 5, 2012

Subsystems (2)
lore
Share & tools
lore

Alex Shi

author
#0 lore raw

PATCH 1/1 xen: fix logical error in tlb flushing

While TLB_FLUSH_ALL gets passed as 'end' argument to flush_tlb_others(), the Xen code was made to check its 'start' parameter. That may give a incorrect op.cmd to MMUEXT_INVLPG_MULTI instead of MMUEXT_TLB_FLUSH_MULTI. Then it causes some page can not be flushed from TLB.

This patch fixed this issue.

---
 arch/x86/xen/mmu.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
arch/x86/xen/mmu.c +1 -1
index b65a761..5141d80 100644
@@ -1283,7 +1283,7 @@ static void xen_flush_tlb_others(const struct cpumask *cpus,
 	cpumask_clear_cpu(smp_processor_id(), to_cpumask(args->mask));
 
 	args->op.cmd = MMUEXT_TLB_FLUSH_MULTI;
-	if (start != TLB_FLUSH_ALL && (end - start) <= PAGE_SIZE) {
+	if (end != TLB_FLUSH_ALL && (end - start) <= PAGE_SIZE) {
 		args->op.cmd = MMUEXT_INVLPG_MULTI;
 		args->op.arg1.linear_addr = start;
 	}
Signature · 1 line
1.7.5.4
Acked-by on v1 1/1
#1 lore raw
Quoted 11 lines
Quoted 1 line
Quoted 1 line · Alex Shi

While TLB_FLUSH_ALL gets passed as 'end' argument to flush_tlb_others(), the Xen code was made to check its 'start' parameter. That may give a incorrect op.cmd to MMUEXT_INVLPG_MULTI instead of MMUEXT_TLB_FLUSH_MULTI. Then it causes some page can not be flushed from TLB.

This patch fixed this issue.

Quoted 22 lines
---
 arch/x86/xen/mmu.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
arch/x86/xen/mmu.c +1 -1
index b65a761..5141d80 100644
@@ -1283,7 +1283,7 @@ static void xen_flush_tlb_others(const struct cpumask 
 *cpus,
 	cpumask_clear_cpu(smp_processor_id(), to_cpumask(args->mask));
 
 	args->op.cmd = MMUEXT_TLB_FLUSH_MULTI;
-	if (start != TLB_FLUSH_ALL && (end - start) <= PAGE_SIZE) {
+	if (end != TLB_FLUSH_ALL && (end - start) <= PAGE_SIZE) {
 		args->op.cmd = MMUEXT_INVLPG_MULTI;
 		args->op.arg1.linear_addr = start;

}

Signature · 3 lines
1.7.5.4

.
#2 lore raw
Quoted 7 lines · Jan Beulich
Quoted 7 lines
Quoted 1 line
Quoted 1 line · Alex Shi

While TLB_FLUSH_ALL gets passed as 'end' argument to flush_tlb_others(), the Xen code was made to check its 'start' parameter. That may give a incorrect op.cmd to MMUEXT_INVLPG_MULTI instead of MMUEXT_TLB_FLUSH_MULTI. Then it causes some page can not be flushed from TLB.

Quoted 6 lines
Quoted 4 lines

This patch fixed this issue.

How can I reproduce this and should this also go to [email protected]?

Quoted 26 lines
Quoted 23 lines
---
 arch/x86/xen/mmu.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
arch/x86/xen/mmu.c +1 -1
index b65a761..5141d80 100644
@@ -1283,7 +1283,7 @@ static void xen_flush_tlb_others(const struct cpumask 
 *cpus,
 	cpumask_clear_cpu(smp_processor_id(), to_cpumask(args->mask));
 
 	args->op.cmd = MMUEXT_TLB_FLUSH_MULTI;
-	if (start != TLB_FLUSH_ALL && (end - start) <= PAGE_SIZE) {
+	if (end != TLB_FLUSH_ALL && (end - start) <= PAGE_SIZE) {
 		args->op.cmd = MMUEXT_INVLPG_MULTI;
 		args->op.arg1.linear_addr = start;

}

Signature · 3 lines
1.7.5.4

.
#3 lore raw
Quoted 9 lines
Quoted 1 line
Quoted 1 line · Konrad Rzeszutek Wilk
Quoted 7 lines · Jan Beulich
Quoted 7 lines
Quoted 1 line
Quoted 1 line · Alex Shi

While TLB_FLUSH_ALL gets passed as 'end' argument to flush_tlb_others(), the Xen code was made to check its 'start' parameter. That may give a incorrect op.cmd to MMUEXT_INVLPG_MULTI instead of MMUEXT_TLB_FLUSH_MULTI. Then it causes some page can not be flushed from TLB.

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

This patch fixed this issue.

How can I reproduce this

I don't know, I spotted this while looking at the code.

Quoted 1 line

and should this also go to [email protected]?

No, the problem got introduced in 3.6-rc1.

Jan

author
#4 lore raw
Quoted 9 lines · Jan Beulich
Quoted 9 lines
Quoted 1 line
Quoted 1 line · Konrad Rzeszutek Wilk
Quoted 7 lines · Jan Beulich
Quoted 7 lines
Quoted 1 line
Quoted 1 line · Alex Shi

While TLB_FLUSH_ALL gets passed as 'end' argument to flush_tlb_others(), the Xen code was made to check its 'start' parameter. That may give a incorrect op.cmd to MMUEXT_INVLPG_MULTI instead of MMUEXT_TLB_FLUSH_MULTI. Then it causes some page can not be flushed from TLB.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

This patch fixed this issue.

CC to Yongjie,
Could you like to test this patch on PV guest

Quoted 6 lines
Quoted 2 lines

How can I reproduce this

I don't know, I spotted this while looking at the code.

Again, since the old buggy code doesn't cause trouble in PV guest, guess the hypercall for MMUEXT_INVLPG_MULTI was translated or treated as MMUEXT_TLB_FLUSH_MULTI. If so, believe correct this will bring a big performance benefit.

Tested-by
#5 lore raw
Quoted 12 lines

-----Original Message-----
From: Shi, Alex
Sent: Wednesday, September 05, 2012 1:35 PM
To: Jan Beulich
Cc: Konrad Rzeszutek Wilk; [email protected]; [email protected]; [email protected]; [email protected]; Ren, Yongjie
Subject: Re: [PATCH] xen: fix logical error in tlb flushing

Quoted 1 line · Jan Beulich
Quoted 1 line
Quoted 1 line
Quoted 1 line

On 24.08.12 at 20:17, Konrad Rzeszutek Wilk

<[email protected]> wrote:

Quoted 17 lines
Quoted 5 lines
Quoted 5 lines · Jan Beulich
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line · Alex Shi

While TLB_FLUSH_ALL gets passed as 'end' argument to flush_tlb_others(), the Xen code was made to check its 'start' parameter. That may give a incorrect op.cmd to

MMUEXT_INVLPG_MULTI

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

instead of MMUEXT_TLB_FLUSH_MULTI. Then it causes some page

can not

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines

be flushed from TLB.

This patch fixed this issue.

Quoted 4 lines

CC to Yongjie,
Could you like to test this patch on PV guest

Xen Dom0 and a PV guest kernel with this patch can boot up correctly.

#6 lore raw
Quoted 29 lines · Yongjie
Quoted 12 lines

-----Original Message-----
From: Shi, Alex
Sent: Wednesday, September 05, 2012 1:35 PM
To: Jan Beulich
Cc: Konrad Rzeszutek Wilk; [email protected]; [email protected]; [email protected]; [email protected]; Ren, Yongjie
Subject: Re: [PATCH] xen: fix logical error in tlb flushing

Quoted 1 line · Jan Beulich
Quoted 1 line
Quoted 1 line
Quoted 1 line

On 24.08.12 at 20:17, Konrad Rzeszutek Wilk

<[email protected]> wrote:

Quoted 17 lines
Quoted 5 lines
Quoted 5 lines · Jan Beulich
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line · Alex Shi

While TLB_FLUSH_ALL gets passed as 'end' argument to flush_tlb_others(), the Xen code was made to check its 'start' parameter. That may give a incorrect op.cmd to

MMUEXT_INVLPG_MULTI

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

instead of MMUEXT_TLB_FLUSH_MULTI. Then it causes some page

can not

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines

be flushed from TLB.

This patch fixed this issue.

Quoted 7 lines
Quoted 4 lines

CC to Yongjie,
Could you like to test this patch on PV guest

Xen Dom0 and a PV guest kernel with this patch can boot up correctly.

OK, applied for v3.6

#7 lore raw
In reply to Alex Shi
Quoted 11 lines
Quoted 1 line
Quoted 1 line · Alex Shi
Quoted 4 lines · Jan Beulich
Quoted 2 lines
Quoted 1 line
Quoted 1 line · Konrad Rzeszutek Wilk

How can I reproduce this

I don't know, I spotted this while looking at the code.

Again, since the old buggy code doesn't cause trouble in PV guest, guess the hypercall for MMUEXT_INVLPG_MULTI was translated or treated as MMUEXT_TLB_FLUSH_MULTI. If so, believe correct this will bring a big performance benefit.

It's not clear to me what was buggy with the code prior to your change. And no, there's no magic widening of the scope of these MMU operations - if you ask the hypervisor for a single page invalidation, that's what it's going to do. But of course, there are cases where extra (full) invalidations need to be done without a guest asking for them. But that's nothing a guest can validly make itself dependent upon.

Jan