sparc64 pcpu failures...
David Miller
Acked-byTejun, I just started seeing the following on sparc64:
[ 56.422005] WARNING: at mm/vmalloc.c:1991 pcpu_get_vm_areas+0x1b4/0x5fc() [ 56.430881] Modules linked in: ipv6(+) snd_ali5451(+) snd_ac97_codec snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd soundcore snd_page_alloc ac97_bus tg3 libphy [ 56.450490] Call Trace: [ 56.460021] [00000000004d769c] pcpu_get_vm_areas+0x1b4/0x5fc [ 56.469645] [00000000004ea5d8] pcpu_alloc+0x204/0x878 [ 56.479212] [00000000006d6a10] snmp_mib_init+0x3c/0x68 [ 56.488813] [0000000010184324] ipv6_add_dev+0x174/0x390 [ipv6] [ 56.498383] [00000000101c44e0] addrconf_init+0x4c/0x16c [ipv6] [ 56.507913] [00000000101c42fc] inet6_init+0x1cc/0x34c [ipv6] [ 56.517450] [0000000000426acc] do_one_initcall+0x44/0x154 [ 56.526975] [0000000000490f7c] SyS_init_module+0xbc/0x1f4 [ 56.536466] [0000000000406214] linux_sparc_syscall32+0x34/0x40 [ 56.545943] ---[ end trace 557118e4f3ad7a51 ]--- [ 56.609093] NET: Unregistered protocol family 10 [ 56.872235] NET: Registered protocol family 10
Might this be a result of:
commit bcb2107fdbecef3de55d597d23453747af81ba88
Author: Tejun Heo <[email protected]> Date: Fri Aug 14 15:00:53 2009 +0900
sparc64: use embedding percpu first chunk allocator
sparc64 currently allocates a large page for each cpu and partially remap them into vmalloc area much like what lpage first chunk allocator did. As a 4M page is used for each cpu, this results in very large unit size and also adds TLB pressure due to the double mapping of pages in the first chunk.
This patch converts sparc64 to use the embedding percpu first chunk allocator which now knows how to handle NUMA configurations. This simplifies the code a lot, doesn't incur any extra TLB pressure and results in better utilization of address space.
Do you think?
From: David Miller <[email protected]>
Date: Thu, 17 Sep 2009 22:18:07 -0700 (PDT)
Quoted 4 lines
Tejun, I just started seeing the following on sparc64:
[ 56.422005] WARNING: at mm/vmalloc.c:1991 pcpu_get_vm_areas+0x1b4/0x5fc()
...
Quoted 19 lines
Might this be a result of:
commit
bcb2107fdbecef3de55d597d23453747af81ba88Author: Tejun Heo <[email protected]> Date: Fri Aug 14 15:00:53 2009 +0900sparc64: use embedding percpu first chunk allocator
sparc64 currently allocates a large page for each cpu and partially remap them into vmalloc area much like what lpage first chunk allocator did. As a 4M page is used for each cpu, this results in very large unit size and also adds TLB pressure due to the double mapping of pages in the first chunk.
This patch converts sparc64 to use the embedding percpu first chunk allocator which now knows how to handle NUMA configurations. This simplifies the code a lot, doesn't incur any extra TLB pressure and results in better utilization of address space.
Quoted 4 lines
Signed-off-by Tejun Heo <[email protected]>Acked-by David S. Miller <[email protected]>Do you think?
I've verified that reverting this makes the problem go away.
Hello, David.
Quoted 26 lines · David Miller
From: David Miller <[email protected]>
Date: Thu, 17 Sep 2009 22:18:07 -0700 (PDT)Quoted 3 lines
Tejun, I just started seeing the following on sparc64:
[ 56.422005] WARNING: at mm/vmalloc.c:1991 pcpu_get_vm_areas+0x1b4/0x5fc()...
Quoted 19 lines
Might this be a result of:
commit
bcb2107fdbecef3de55d597d23453747af81ba88Author: Tejun Heo <[email protected]> Date: Fri Aug 14 15:00:53 2009 +0900sparc64: use embedding percpu first chunk allocator
sparc64 currently allocates a large page for each cpu and partially remap them into vmalloc area much like what lpage first chunk allocator did. As a 4M page is used for each cpu, this results in very large unit size and also adds TLB pressure due to the double mapping of pages in the first chunk.
This patch converts sparc64 to use the embedding percpu first chunk allocator which now knows how to handle NUMA configurations. This simplifies the code a lot, doesn't incur any extra TLB pressure and results in better utilization of address space.
Quoted 6 lines
Quoted 4 lines
Signed-off-by Tejun Heo <[email protected]>Acked-by David S. Miller <[email protected]>Do you think?
I've verified that reverting this makes the problem go away.
Yes, this would be the result of the new congruent sparse allocator. Heh... I didn't really expect that WARN_ON() to actually trigger. What it means is that the farthest percpu units were too far to fit into vmalloc area so percpu chunks couldn't be allocated in the vmalloc area. Checking... vmalloc area on sparc64 is only 4G so yeap that is entirely possible.
Hmmm... the congruent sparse allocator assumes vmalloc area to be larger with some margin than the farthest physical memory nodes. On x86, powerpc and ia64, this holds with sufficient margin. Would it be possible to modify sparc64 memory layout so that the assumption can be satisfied? If that's not possible, going back to lpage is an option but in many ways congruent sparse allocator is better.
Thanks.
Signature · 1 line
tejun
From: Tejun Heo <[email protected]>
Date: Fri, 18 Sep 2009 17:38:58 +0900
Quoted 7 lines
Yes, this would be the result of the new congruent sparse allocator. Heh... I didn't really expect that
WARN_ON()to actually trigger. What it means is that the farthest percpu units were too far to fit into vmalloc area so percpu chunks couldn't be allocated in the vmalloc area. Checking... vmalloc area on sparc64 is only 4G so yeap that is entirely possible.
Quoted 6 lines
Hmmm... the congruent sparse allocator assumes vmalloc area to be larger with some margin than the farthest physical memory nodes. On x86, powerpc and ia64, this holds with sufficient margin. Would it be possible to modify sparc64 memory layout so that the assumption can be satisfied? If that's not possible, going back to lpage is an option but in many ways congruent sparse allocator is better.
I'll think about this, thanks.