32/64-bit NUMA consolidation behavior regresion
Subsystems (2)
Hi Tejun,
I was bisecting a boot problem on a 32-bit NUMA kernel and it bisected down to commit 8db78cc4. It turns out that, with this patch, pcpu_need_numa() changed its return value on my system from 1 to 0. What that basically meant was that we stopped using the remapped lowmem areas for percpu data.
My system is just qemu booted with:
-smp 8 -m 8192 -numa node,nodeid=0,cpus=0-3 -numa node,nodeid=1,cpus=4-7
Watch the "PERCPU:" line early in boot, and you can see the "Embedded" come and go with or without your patch:
[ 0.000000] PERCPU: Embedded 11 pages/cpu @f3000000 s30592 r0 d14464 vs [ 0.000000] PERCPU: 11 4K pages/cpu @f83fe000 s30592 r0 d14464
I believe this has to do with the hunks in your patch that do:
-#ifdef CONFIG_X86_64
init_cpu_to_node();
-#endif
...
-#ifdef CONFIG_X86_32
-DEFINE_EARLY_PER_CPU(int, x86_cpu_to_node_map, 0); -#else
DEFINE_EARLY_PER_CPU(int, x86_cpu_to_node_map, NUMA_NO_NODE);
-#endif
EXPORT_EARLY_PER_CPU_SYMBOL(x86_cpu_to_node_map);
I don't have a fix handy because I'm working on the original problem, but I just happened to run across this during a bisect.
Hello, Dave.
Quoted 10 lines · Dave Hansen
My system is just qemu booted with:
-smp 8 -m 8192 -numa node,nodeid=0,cpus=0-3 -numa node,nodeid=1,cpus=4-7
Watch the "PERCPU:" line early in boot, and you can see the "Embedded" come and go with or without your patch:
[ 0.000000] PERCPU: Embedded 11 pages/cpu @f3000000 s30592 r0 d14464 vs [ 0.000000] PERCPU: 11 4K pages/cpu @f83fe000 s30592 r0 d14464
...
Quoted 2 lines
I don't have a fix handy because I'm working on the original problem, but I just happened to run across this during a bisect.
Just tested 3.7-rc7 w/ qemu and it works as expected here.
Can you please boot with the following debug patch and report the boot message before and after?
Thanks.
arch/x86/kernel/setup_percpu.c +5 -1
@@ -71,9 +71,13 @@ static bool __init pcpu_need_numa(void)for_each_possible_cpu (cpu) {int node =early_cpu_to_node (cpu); +printk ("XXX pcpu_need_numa: cpu%d@%d online=%d ND=%p\n" , + cpu, node,node_online (node),NODE_DATA (node));if (node_online (node) &&NODE_DATA (node) && - last && last !=NODE_DATA (node)) + last && last !=NODE_DATA (node)) { +printk ("XXX need numa\n" );return true; + } last =NODE_DATA (node); }
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 11 lines · Tejun Heo
Quoted 10 lines · Dave Hansen
My system is just qemu booted with:
-smp 8 -m 8192 -numa node,nodeid=0,cpus=0-3 -numa node,nodeid=1,cpus=4-7
Watch the "PERCPU:" line early in boot, and you can see the "Embedded" come and go with or without your patch:
[ 0.000000] PERCPU: Embedded 11 pages/cpu @f3000000 s30592 r0 d14464 vs [ 0.000000] PERCPU: 11 4K pages/cpu @f83fe000 s30592 r0 d14464
Quoted 8 lines
...
Quoted 2 lines
I don't have a fix handy because I'm working on the original problem, but I just happened to run across this during a bisect.
Just tested 3.7-rc7 w/ qemu and it works as expected here.
Can you please boot with the following debug patch and report the boot message before and after?
Hi Tejun,
I just tested with 3.7-rc7 and I'm seeing the expected behavior now. Looks like it got fixed along the way somewhere. I was bisecting way back in the 2.6.3x's. Sorry of the noise.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>