Skip to contentSkip to conversation

UV: fix incorrect tlb flush all issue

PATCH In review Alex Shi 3 participants 5 messages Aug 24, 2012 – Sep 10, 2012

Subsystems (2)
lore
Share & tools
lore

Alex Shi

author
#0 lore raw

PATCH 1/1 UV: fix incorrect tlb flush all issue

The flush tlb optimization code has logical issue on UV platform. It doesn't flush the full range at all, since it simply ignores its 'end' parameter (and hence also the "all" indicator) in uv_flush_tlb_others() function.

This patch fixed this issue, but untested due to hardware leaking.

---
 arch/x86/include/asm/uv/uv_bau.h |    6 ++++--
 arch/x86/platform/uv/tlb_uv.c    |   14 ++++++++------
 2 files changed, 12 insertions(+), 8 deletions(-)
arch/x86/include/asm/uv/uv_bau.h +4 -2
index a06983c..ac6f326 100644
@@ -225,8 +225,10 @@ struct bau_local_cpumask {
  * The payload is software-defined for INTD transactions
  */
 struct bau_msg_payload {
-	unsigned long	address;		/* signifies a page or all
-						   TLB's of the cpu */
+	unsigned long	start;			/* start address to flush TLB
+						   of the cpu */
+	unsigned long	end;			/* end address to flush TLB
+						   of the cpu */
 	/* 64 bits */
 	unsigned short	sending_cpu;		/* filled in by sender */
 	/* 16 bits */
arch/x86/platform/uv/tlb_uv.c +8 -6
index b8b3a37..c603d15 100644
@@ -280,12 +280,12 @@ static void bau_process_message(struct msg_desc *mdp, struct bau_control *bcp,
 	/*
 	 * This must be a normal message, or retry of a normal message
 	 */
-	if (msg->address == TLB_FLUSH_ALL) {
+	if (msg->end == 0) {
+		__flush_tlb_one(msg->start);
+		stat->d_onetlb++;
+	} else {
 		local_flush_tlb();
 		stat->d_alltlb++;
-	} else {
-		__flush_tlb_one(msg->address);
-		stat->d_onetlb++;
 	}
 	stat->d_requestee++;
 
@@ -1034,7 +1034,8 @@ static int set_distrib_bits(struct cpumask *flush_mask, struct bau_control *bcp,
  * globally purge translation cache of a virtual address or all TLB's
  * @cpumask: mask of all cpu's in which the address is to be removed
  * @mm: mm_struct containing virtual address range
- * @va: virtual address to be removed (or TLB_FLUSH_ALL for all TLB's on cpu)
+ * @start: start virtual address to be removed from TLB
+ * @end: end virtual address to be remove from TLB
  * @cpu: the current cpu
  *
  * This is the entry point for initiating any UV global TLB shootdown.
@@ -1113,7 +1114,8 @@ const struct cpumask *uv_flush_tlb_others(const struct cpumask *cpumask,
 
 	record_send_statistics(stat, locals, hubs, remotes, bau_desc);
 
-	bau_desc->payload.address = start;
+	bau_desc->payload.start = start;
+	bau_desc->payload.end	= end;
 	bau_desc->payload.sending_cpu = cpu;
 	/*
 	 * uv_flush_send_and_wait returns 0 if all cpu's were messaged,
Signature · 1 line
1.7.5.4
#1 lore raw
Quoted 6 lines · On Fri, 24 Aug 2012 16:57:35 +0800 Alex Shi <[email protected]> wrote:

The flush tlb optimization code has logical issue on UV platform. It doesn't flush the full range at all, since it simply ignores its 'end' parameter (and hence also the "all" indicator) in uv_flush_tlb_others() function.

This patch fixed this issue, but untested due to hardware leaking.

Well, it doesn't really come very close to being compilable:

arch/x86/platform/uv/tlb_uv.c: In function 'bau_process_message':
arch/x86/platform/uv/tlb_uv.c:283: error: 'struct bau_pq_entry' has no member named 'end' arch/x86/platform/uv/tlb_uv.c:284: error: 'struct bau_pq_entry' has no member named 'start'

Cliff, could you please help out here?

author
#2 lore raw
Quoted 10 lines · Andrew Morton
Quoted 6 lines · On Fri, 24 Aug 2012 16:57:35 +0800 Alex Shi <[email protected]> wrote:

The flush tlb optimization code has logical issue on UV platform. It doesn't flush the full range at all, since it simply ignores its 'end' parameter (and hence also the "all" indicator) in uv_flush_tlb_others() function.

This patch fixed this issue, but untested due to hardware leaking.

Quoted 8 lines

Well, it doesn't really come very close to being compilable:

arch/x86/platform/uv/tlb_uv.c: In function 'bau_process_message':
arch/x86/platform/uv/tlb_uv.c:283: error: 'struct bau_pq_entry' has no member named 'end' arch/x86/platform/uv/tlb_uv.c:284: error: 'struct bau_pq_entry' has no member named 'start'

Cliff, could you please help out here?

Sorry, I had found the problem yesterday, and in trying to fix this issue.

here I use the simplest fixing, that just flush single page or flush all tlb. It will great if UV experts like to enable flush_tlb_range() finally.

------

Quoted 1 line

From 1b5e470466d719109770a39b230c074f9d2a1490 Mon Sep 17 00:00:00 2001

From: Alex Shi <[email protected]>
Date: Fri, 7 Sep 2012 10:38:36 +0800
Subject: [PATCH] x86/uv: fix incorrect tlb flush issue

The flush tlb optimization code has logical issue on UV platform. It doesn't flush the full range at all, since it simply ignores its 'end' parameter (and hence also the "all" indicator) in uv_flush_tlb_others() function.

I have tried to embed the start/end on UV code, but finally find the payload struct embedded hardware filled vector. So be lacking the enough knowledge of the hardware, I just check the start/end fair in uv_flush_tlb_others(): except single page flush(end==0), will flush all TLB entries for any other situations.

Cc Jack Steiner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
---
 arch/x86/platform/uv/tlb_uv.c |    8 ++++++--
 1 files changed, 6 insertions(+), 2 deletions(-)
arch/x86/platform/uv/tlb_uv.c +6 -2
index b8b3a37..1b2d49a 100644
@@ -1034,7 +1034,8 @@ static int set_distrib_bits(struct cpumask *flush_mask, struct bau_control *bcp,
  * globally purge translation cache of a virtual address or all TLB's
  * @cpumask: mask of all cpu's in which the address is to be removed
  * @mm: mm_struct containing virtual address range
- * @va: virtual address to be removed (or TLB_FLUSH_ALL for all TLB's on cpu)
+ * @start: start virtual address to be removed from TLB
+ * @end: end virtual address to be remove from TLB
  * @cpu: the current cpu
  *
  * This is the entry point for initiating any UV global TLB shootdown.
@@ -1113,7 +1114,10 @@ const struct cpumask *uv_flush_tlb_others(const struct cpumask *cpumask,
 
 	record_send_statistics(stat, locals, hubs, remotes, bau_desc);
 
-	bau_desc->payload.address = start;
+	if (!end)
+		bau_desc->payload.address = TLB_FLUSH_ALL;
+	else
+		bau_desc->payload.address = start;
 	bau_desc->payload.sending_cpu = cpu;
 	/*
 	 * uv_flush_send_and_wait returns 0 if all cpu's were messaged,
Signature · 1 line
1.7.5.4
#3 lore raw

arch/x86/platform/uv/tlb_uv.c const struct cpumask *uv_flush_tlb_others(const struct · line 1118

Quoted 8 lines
Quoted 1 line
Quoted 1 line · Alex Shi
arch/x86/platform/uv/tlb_uv.c +1 -1
@@ -1113,7 +1114,10 @@ const struct cpumask *uv_flush_tlb_others(const struct 
 cpumask *cpumask,
 
 	record_send_statistics(stat, locals, hubs, remotes, bau_desc);
 
-	bau_desc->payload.address = start;
+	if (!end)

So despite having made a very similar mistake on the Xen side, you're again not following your own code adjustments here:
The indicator for a multi-page flush is not 'end' being zero, nor would it be okay to use the proper indicator (TLB_FLUSH_ALL) here. Instead, to yield correct code, you need to check for the range covering more than one page. Improvements to do finer grained flushing (if worthwhile on UV) could then be done on top of this.

Jan

Quoted 6 lines
arch/x86/platform/uv/tlb_uv.c +3 -0
+		bau_desc->payload.address = TLB_FLUSH_ALL;
+	else
+		bau_desc->payload.address = start;
 	bau_desc->payload.sending_cpu = cpu;
 	/*

* uv_flush_send_and_wait returns 0 if all cpu's were messaged,

author
#4 lore raw
Quoted 6 lines · Jan Beulich
Quoted 6 lines
Quoted 1 line
Quoted 1 line · Alex Shi
(quoted diff) +0 -0
@@ -1113,7 +1114,10 @@ const struct cpumask *uv_flush_tlb_others(const struct 
 cpumask *cpumask,
 
 	record_send_statistics(stat, locals, hubs, remotes, bau_desc);
 
Quoted 12 lines

Review const struct cpumask *uv_flush_tlb_others(const struct · line 1118

Quoted 2 lines
(quoted diff) +1 -1
-	bau_desc->payload.address = start;
+	if (!end)

So despite having made a very similar mistake on the Xen side, you're again not following your own code adjustments here:
The indicator for a multi-page flush is not 'end' being zero, nor would it be okay to use the proper indicator (TLB_FLUSH_ALL) here. Instead, to yield correct code, you need to check for the range covering more than one page. Improvements to do finer grained flushing (if worthwhile on UV) could then be done on top of this.

Thanks for catching this typo. The patch was updated:
==========

Quoted 1 line

From bccb5f34834986fbd36c069640237296a9b1fe37 Mon Sep 17 00:00:00 2001

From: Alex Shi <[email protected]>
Date: Fri, 7 Sep 2012 10:38:36 +0800
Subject: [PATCH] x86/uv: fix incorrect tlb flush issue

The flush tlb optimization code has logical issue on UV platform. It doesn't flush the full range at all, since it simply ignores its 'end' parameter (and hence also the "all" indicator) in uv_flush_tlb_others() function.

I have tried to embed the start/end on UV code, but finally find the 'payload' message format embedded hardware filled vector. So be lacking the enough knowledge of the hardware, I just check the start/end fair in uv_flush_tlb_others(): except single page flush, will flush all TLB entries for other situations.

Cc Jack Steiner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
---
 arch/x86/platform/uv/tlb_uv.c |    8 ++++++--
 1 files changed, 6 insertions(+), 2 deletions(-)
arch/x86/platform/uv/tlb_uv.c +6 -2
index b8b3a37..582836e 100644
@@ -1034,7 +1034,8 @@ static int set_distrib_bits(struct cpumask *flush_mask, struct bau_control *bcp,
  * globally purge translation cache of a virtual address or all TLB's
  * @cpumask: mask of all cpu's in which the address is to be removed
  * @mm: mm_struct containing virtual address range
- * @va: virtual address to be removed (or TLB_FLUSH_ALL for all TLB's on cpu)
+ * @start: start virtual address to be removed from TLB
+ * @end: end virtual address to be remove from TLB
  * @cpu: the current cpu
  *
  * This is the entry point for initiating any UV global TLB shootdown.
@@ -1113,7 +1114,10 @@ const struct cpumask *uv_flush_tlb_others(const struct cpumask *cpumask,
 
 	record_send_statistics(stat, locals, hubs, remotes, bau_desc);
 
-	bau_desc->payload.address = start;
+	if (!end || (end - start) <= PAGE_SIZE)
+		bau_desc->payload.address = start;
+	else
+		bau_desc->payload.address = TLB_FLUSH_ALL;
 	bau_desc->payload.sending_cpu = cpu;
 	/*
 	 * uv_flush_send_and_wait returns 0 if all cpu's were messaged,
Signature · 1 line
1.7.5.4