Skip to contentSkip to conversation

x86: Use online node real index in calulate_tbl_offset()

PATCH In review Yinghai Lu 3 participants 3 messages Oct 29, 2010 – Oct 31, 2010

Subsystems (2)
lore
Share & tools
lore

Yinghai Lu

author
#0 lore raw

PATCH 1/1 x86: Use online node real index in calulate_tbl_offset()

Found one numa system that doesn't have ram installed in first socket hang during executing init scripts.

bisect to:

|commit 932967202182743c01a2eee4bdfa2c42697bc586
|Author: Shaohua Li <[email protected]>
|Date:   Wed Oct 20 11:07:03 2010 +0800
|
|    x86: Spread tlb flush vector between nodes

It turns out when first socket is not online could have cpus on node1 tlb_offset set to bigger than NUM_INVALIDATE_TLB_VECTORS.

that could affect systems like 4 sockets, but socket 2 doesn't have installed, sockets 3 will get too big tlb_offset.

Need to use real online node idx.

---
 arch/x86/mm/tlb.c |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
arch/x86/mm/tlb.c +3 -2
Index: linux-2.6/arch/x86/mm/tlb.c
===================================================================
@@ -223,7 +223,7 @@ void native_flush_tlb_others(const struc
 
 static void __cpuinit calculate_tlb_offset(void)
 {
-	int cpu, node, nr_node_vecs;
+	int cpu, node, nr_node_vecs, idx = 0;
 	/*
 	 * we are changing tlb_vector_offset for each CPU in runtime, but this
 	 * will not cause inconsistency, as the write is atomic under X86. we
@@ -239,7 +239,7 @@ static void __cpuinit calculate_tlb_offs
 		nr_node_vecs = NUM_INVALIDATE_TLB_VECTORS/nr_online_nodes;
 
 	for_each_online_node(node) {
-		int node_offset = (node % NUM_INVALIDATE_TLB_VECTORS) *
+		int node_offset = (idx % NUM_INVALIDATE_TLB_VECTORS) *
 			nr_node_vecs;
 		int cpu_offset = 0;
 		for_each_cpu(cpu, cpumask_of_node(node)) {
@@ -248,6 +248,7 @@ static void __cpuinit calculate_tlb_offs
 			cpu_offset++;
 			cpu_offset = cpu_offset % nr_node_vecs;
 		}
+		idx++;
 	}
 }
 
#1 lore raw

Shaohua, does this look right to you?

-hpa

Quoted 56 lines · Yinghai Lu

Found one numa system that doesn't have ram installed in first socket hang during executing init scripts.

bisect to:

|commit 932967202182743c01a2eee4bdfa2c42697bc586
|Author: Shaohua Li <[email protected]>
|Date:   Wed Oct 20 11:07:03 2010 +0800
|
|    x86: Spread tlb flush vector between nodes

It turns out when first socket is not online could have cpus on node1 tlb_offset set to bigger than NUM_INVALIDATE_TLB_VECTORS.

that could affect systems like 4 sockets, but socket 2 doesn't have installed, sockets 3 will get too big tlb_offset.

Need to use real online node idx.

---
 arch/x86/mm/tlb.c |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

Index: linux-2.6/arch/x86/mm/tlb.c
===================================================================

arch/x86/mm/tlb.c +3 -2
@@ -223,7 +223,7 @@ void native_flush_tlb_others(const struc
 
 static void __cpuinit calculate_tlb_offset(void)
 {
-	int cpu, node, nr_node_vecs;
+	int cpu, node, nr_node_vecs, idx = 0;
 	/*
 	 * we are changing tlb_vector_offset for each CPU in runtime, but this
 	 * will not cause inconsistency, as the write is atomic under X86. we
@@ -239,7 +239,7 @@ static void __cpuinit calculate_tlb_offs
 		nr_node_vecs = NUM_INVALIDATE_TLB_VECTORS/nr_online_nodes;
 
 	for_each_online_node(node) {
-		int node_offset = (node % NUM_INVALIDATE_TLB_VECTORS) *
+		int node_offset = (idx % NUM_INVALIDATE_TLB_VECTORS) *
 			nr_node_vecs;
 		int cpu_offset = 0;
 		for_each_cpu(cpu, cpumask_of_node(node)) {
@@ -248,6 +248,7 @@ static void __cpuinit calculate_tlb_offs
 			cpu_offset++;
 			cpu_offset = cpu_offset % nr_node_vecs;
 		}
+		idx++;
 	}
 }
 
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.
Acked-by on v1 1/1
#2 lore raw
In reply to Yinghai Lu
Quoted 11 lines · Yinghai Lu

Found one numa system that doesn't have ram installed in first socket hang during executing init scripts.

bisect to:

|commit 932967202182743c01a2eee4bdfa2c42697bc586
|Author: Shaohua Li <[email protected]>
|Date:   Wed Oct 20 11:07:03 2010 +0800
|
|    x86: Spread tlb flush vector between nodes
Quoted 8 lines

It turns out when first socket is not online could have cpus on node1 tlb_offset set to bigger than NUM_INVALIDATE_TLB_VECTORS.

that could affect systems like 4 sockets, but socket 2 doesn't have installed, sockets 3 will get too big tlb_offset.

Need to use real online node idx.

Thanks for catching it up.