UV: fix incorrect tlb flush all issue
Subsystems (2)
PATCH 1/1 UV: fix incorrect tlb flush all issue
The flush tlb optimization code has logical issue on UV platform. It doesn't flush the full range at all, since it simply ignores its 'end' parameter (and hence also the "all" indicator) in uv_flush_tlb_others() function.
This patch fixed this issue, but untested due to hardware leaking.
--- arch/x86/include/asm/uv/uv_bau.h | 6 ++++-- arch/x86/platform/uv/tlb_uv.c | 14 ++++++++------ 2 files changed, 12 insertions(+), 8 deletions(-)
arch/x86/include/asm/uv/uv_bau.h +4 -2
@@ -225,8 +225,10 @@ struct bau_local_cpumask {* The payload is software-defined for INTD transactions */ struct bau_msg_payload { -unsigned long address;/* signifies a page or all -TLB's of the cpu */ +unsigned long start;/* start address to flush TLB +of the cpu */ +unsigned long end;/* end address to flush TLB +of the cpu */ /* 64 bits */ unsigned short sending_cpu;/* filled in by sender */ /* 16 bits */
arch/x86/platform/uv/tlb_uv.c +8 -6
@@ -280,12 +280,12 @@ static void bau_process_message(struct msg_desc *mdp, struct bau_control *bcp,/* * This must be a normal message, or retry of a normal message */ -if (msg->address ==TLB_FLUSH_ALL ) { +if (msg->end ==0 ) { +__flush_tlb_one (msg->start); + stat->d_onetlb++; + }else {local_flush_tlb (); stat->d_alltlb++; - }else { -__flush_tlb_one (msg->address); - stat->d_onetlb++; } stat->d_requestee++;
@@ -1034,7 +1034,8 @@ static int set_distrib_bits(struct cpumask *flush_mask, struct bau_control *bcp,* globally purge translation cache of a virtual address or all TLB's * @cpumask: mask of all cpu's in which the address is to be removed * @mm: mm_struct containing virtual address range -* @va: virtual address to be removed (or TLB_FLUSH_ALL for all TLB's on cpu) +* @start: start virtual address to be removed from TLB +* @end: end virtual address to be remove from TLB * @cpu: the current cpu * * This is the entry point for initiating any UV global TLB shootdown.
@@ -1113,7 +1114,8 @@ const struct cpumask *uv_flush_tlb_others(const struct cpumask *cpumask,record_send_statistics (stat, locals, hubs, remotes, bau_desc); - bau_desc->payload.address = start; + bau_desc->payload.start = start; + bau_desc->payload.end = end; bau_desc->payload.sending_cpu = cpu;/* * uv_flush_send_and_wait returns 0 if all cpu's were messaged,
Signature · 1 line
1.7.5.4
Quoted 6 lines · On Fri, 24 Aug 2012 16:57:35 +0800 Alex Shi <[email protected]> wrote:
The flush tlb optimization code has logical issue on UV platform. It doesn't flush the full range at all, since it simply ignores its 'end' parameter (and hence also the "all" indicator) in
uv_flush_tlb_others()function.This patch fixed this issue, but untested due to hardware leaking.
Well, it doesn't really come very close to being compilable:
arch/x86/platform/uv/tlb_uv.c: In function 'bau_process_message':arch/x86/platform/uv/tlb_uv.c:283: error: 'struct bau_pq_entry' has no member named 'end' arch/x86/platform/uv/tlb_uv.c:284: error: 'struct bau_pq_entry' has no member named 'start'
Cliff, could you please help out here?
Quoted 10 lines · Andrew Morton
Quoted 6 lines · On Fri, 24 Aug 2012 16:57:35 +0800 Alex Shi <[email protected]> wrote:
The flush tlb optimization code has logical issue on UV platform. It doesn't flush the full range at all, since it simply ignores its 'end' parameter (and hence also the "all" indicator) in
uv_flush_tlb_others()function.This patch fixed this issue, but untested due to hardware leaking.
Quoted 8 lines
Well, it doesn't really come very close to being compilable:
arch/x86/platform/uv/tlb_uv.c: In function 'bau_process_message':arch/x86/platform/uv/tlb_uv.c:283: error: 'structbau_pq_entry' has no member named 'end'arch/x86/platform/uv/tlb_uv.c:284: error: 'structbau_pq_entry' has no member named 'start'Cliff, could you please help out here?
Sorry, I had found the problem yesterday, and in trying to fix this issue.
here I use the simplest fixing, that just flush single page or flush all tlb. It will great if UV experts like to enable flush_tlb_range() finally.
------
Quoted 1 line
From
1b5e470466d719109770a39b230c074f9d2a1490Mon Sep 17 00:00:00 2001
From: Alex Shi <[email protected]>
Date: Fri, 7 Sep 2012 10:38:36 +0800
Subject: [PATCH] x86/uv: fix incorrect tlb flush issue
The flush tlb optimization code has logical issue on UV platform. It doesn't flush the full range at all, since it simply ignores its 'end' parameter (and hence also the "all" indicator) in uv_flush_tlb_others() function.
I have tried to embed the start/end on UV code, but finally find the payload struct embedded hardware filled vector. So be lacking the enough knowledge of the hardware, I just check the start/end fair in uv_flush_tlb_others(): except single page flush(end==0), will flush all TLB entries for any other situations.
--- arch/x86/platform/uv/tlb_uv.c | 8 ++++++-- 1 files changed, 6 insertions(+), 2 deletions(-)
arch/x86/platform/uv/tlb_uv.c +6 -2
@@ -1034,7 +1034,8 @@ static int set_distrib_bits(struct cpumask *flush_mask, struct bau_control *bcp,* globally purge translation cache of a virtual address or all TLB's * @cpumask: mask of all cpu's in which the address is to be removed * @mm: mm_struct containing virtual address range -* @va: virtual address to be removed (or TLB_FLUSH_ALL for all TLB's on cpu) +* @start: start virtual address to be removed from TLB +* @end: end virtual address to be remove from TLB * @cpu: the current cpu * * This is the entry point for initiating any UV global TLB shootdown.
@@ -1113,7 +1114,10 @@ const struct cpumask *uv_flush_tlb_others(const struct cpumask *cpumask,record_send_statistics (stat, locals, hubs, remotes, bau_desc); - bau_desc->payload.address = start; +if (!end) + bau_desc->payload.address =TLB_FLUSH_ALL ; +else + bau_desc->payload.address = start; bau_desc->payload.sending_cpu = cpu;/* * uv_flush_send_and_wait returns 0 if all cpu's were messaged,
Signature · 1 line
1.7.5.4
arch/x86/platform/uv/tlb_uv.c const struct cpumask *uv_flush_tlb_others(const struct · line 1118
Quoted 8 lines
Quoted 1 line
Quoted 1 line · Alex Shi
arch/x86/platform/uv/tlb_uv.c +1
-1@@ -1113,7 +1114,10 @@ const struct cpumask *uv_flush_tlb_others(const structcpumask *cpumask,record_send_statistics (stat ,locals ,hubs ,remotes ,bau_desc ); - bau_desc->payload.address = start; +if (!end)
So despite having made a very similar mistake on the Xen side, you're again not following your own code adjustments here:
The indicator for a multi-page flush is not 'end' being zero, nor would it be okay to use the proper indicator (TLB_FLUSH_ALL) here. Instead, to yield correct code, you need to check for the range covering more than one page. Improvements to do finer grained flushing (if worthwhile on UV) could then be done on top of this.
Jan
Quoted 6 lines
arch/x86/platform/uv/tlb_uv.c +3
-0+ bau_desc->payload.address =TLB_FLUSH_ALL ; + else + bau_desc->payload.address = start; bau_desc->payload.sending_cpu = cpu;/* *
uv_flush_send_and_waitreturns 0 if all cpu's were messaged,
Quoted 6 lines · Jan Beulich
Quoted 6 lines
Quoted 1 line
Quoted 1 line · Alex Shi
(quoted diff) +0
-0@@ -1113,7 +1114,10 @@ const struct cpumask *uv_flush_tlb_others(const struct cpumask *cpumask, record_send_statistics(stat, locals, hubs, remotes, bau_desc);
Quoted 12 lines
Review const struct cpumask *uv_flush_tlb_others(const struct · line 1118
Quoted 2 lines
(quoted diff) +1
-1- bau_desc->payload.address = start; + if (!end)So despite having made a very similar mistake on the Xen side, you're again not following your own code adjustments here:
The indicator for a multi-page flush is not 'end' being zero, nor would it be okay to use the proper indicator (TLB_FLUSH_ALL) here. Instead, to yield correct code, you need to check for the range covering more than one page. Improvements to do finer grained flushing (if worthwhile on UV) could then be done on top of this.
Thanks for catching this typo. The patch was updated:
==========
Quoted 1 line
From
bccb5f34834986fbd36c069640237296a9b1fe37Mon Sep 17 00:00:00 2001
From: Alex Shi <[email protected]>
Date: Fri, 7 Sep 2012 10:38:36 +0800
Subject: [PATCH] x86/uv: fix incorrect tlb flush issue
The flush tlb optimization code has logical issue on UV platform. It doesn't flush the full range at all, since it simply ignores its 'end' parameter (and hence also the "all" indicator) in uv_flush_tlb_others() function.
I have tried to embed the start/end on UV code, but finally find the 'payload' message format embedded hardware filled vector. So be lacking the enough knowledge of the hardware, I just check the start/end fair in uv_flush_tlb_others(): except single page flush, will flush all TLB entries for other situations.
--- arch/x86/platform/uv/tlb_uv.c | 8 ++++++-- 1 files changed, 6 insertions(+), 2 deletions(-)
arch/x86/platform/uv/tlb_uv.c +6 -2
@@ -1034,7 +1034,8 @@ static int set_distrib_bits(struct cpumask *flush_mask, struct bau_control *bcp,* globally purge translation cache of a virtual address or all TLB's * @cpumask: mask of all cpu's in which the address is to be removed * @mm: mm_struct containing virtual address range -* @va: virtual address to be removed (or TLB_FLUSH_ALL for all TLB's on cpu) +* @start: start virtual address to be removed from TLB +* @end: end virtual address to be remove from TLB * @cpu: the current cpu * * This is the entry point for initiating any UV global TLB shootdown.
@@ -1113,7 +1114,10 @@ const struct cpumask *uv_flush_tlb_others(const struct cpumask *cpumask,record_send_statistics (stat, locals, hubs, remotes, bau_desc); - bau_desc->payload.address = start; +if (!end || (end - start) <=PAGE_SIZE ) + bau_desc->payload.address = start; +else + bau_desc->payload.address =TLB_FLUSH_ALL ; bau_desc->payload.sending_cpu = cpu;/* * uv_flush_send_and_wait returns 0 if all cpu's were messaged,
Signature · 1 line
1.7.5.4