Skip to contentSkip to conversation

swapper: page allocation failure.

DISCUSSION Haar János 5 participants 28 messages May 21, 2006 – May 23, 2006

lore
Share & tools
lore
#0 lore raw

Hello, list,

I seriously gets this, and dont know why.
This server have 2GB ram, and ~1.1G always free!
Anybody have an idea?

Thanks,
Janos

May 21 09:05:35 st-0003 kernel: swapper: page allocation failure. order:0,
mode:0x20
May 21 09:05:35 st-0003 kernel:  <c013c6ac> __alloc_pages+0x274/0x286
<c014af1d> cache_alloc_refill+0x2a6/0x45c
May 21 09:05:35 st-0003 kernel:  <c014b12b> __kmalloc+0x58/0x61   <c03d5bfc>
__alloc_skb+0x49/0xf5
May 21 09:05:35 st-0003 kernel:  <f88336b1>
e1000_alloc_rx_buffers+0x5c/0x2e5 [e1000]   <f88315de>
e1000_clean_rx_irq+0x4fd/0x50a [e1000]
May 21 09:05:35 st-0003 kernel:  <f88342d2> e1000_intr+0x96/0xf4 [e1000]
<c0136927> handle_IRQ_event+0x20/0x4c
May 21 09:05:35 st-0003 kernel:  <c01369cf> __do_IRQ+0x7c/0xcd   <c0104a65>
do_IRQ+0x44/0x53
May 21 09:05:35 st-0003 kernel:  <c01030ba> common_interrupt+0x1a/0x20
<c01016f5> mwait_idle+0x20/0x34
May 21 09:05:35 st-0003 kernel:  <c01016c0> cpu_idle+0x5a/0x6f   <c05966c9>
start_kernel+0x317/0x31d
May 21 09:05:35 st-0003 kernel: Mem-info:
May 21 09:05:35 st-0003 kernel: DMA per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: DMA32 per-cpu: empty
May 21 09:05:35 st-0003 kernel: Normal per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:30
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:47
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:51
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:55
May 21 09:05:35 st-0003 kernel: HighMem per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:98
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:10
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:183
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:6
May 21 09:05:35 st-0003 kernel: Free pages:     1165384kB (1160424kB
HighMem)
May 21 09:05:35 st-0003 kernel: Active:3002 inactive:212594 dirty:35905
writeback:1 unstable:0 free:291346 slab:6782 mapped:929 pagetables:57
May 21 09:05:35 st-0003 kernel: DMA free:3548kB min:68kB low:84kB high:100kB
active:0kB inactive:5808kB present:16384kB pages_scanned:0
all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: DMA32 free:0kB min:0kB low:0kB high:0kB
active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: Normal free:1412kB min:3756kB low:4692kB
high:5632kB active:8kB inactive:838708kB present:901120kB pages_scanned:0
all_unrecla
imable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 9215
May 21 09:05:35 st-0003 kernel: HighMem free:1160424kB min:512kB low:1740kB
high:2972kB active:12000kB inactive:5860kB present:1179584kB pages_scanned:0
all_u
nreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 0
May 21 09:05:35 st-0003 kernel: DMA: 1*4kB 1*8kB 1*16kB 0*32kB 1*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 0*4096kB = 3548kB
May 21 09:05:35 st-0003 kernel: DMA32: empty
May 21 09:05:35 st-0003 kernel: Normal: 1*4kB 0*8kB 0*16kB 0*32kB 0*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 1412kB
May 21 09:05:35 st-0003 kernel: HighMem: 142*4kB 34*8kB 18*16kB 8*32kB
6*64kB 2*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 282*4096kB = 1160424kB
May 21 09:05:35 st-0003 kernel: Free swap:            0kB
May 21 09:05:35 st-0003 kernel: 524272 pages of RAM
May 21 09:05:35 st-0003 kernel: 294896 pages of HIGHMEM
May 21 09:05:35 st-0003 kernel: 6010 reserved pages
May 21 09:05:35 st-0003 kernel: 213818 pages shared
May 21 09:05:35 st-0003 kernel: 0 pages swap cached
May 21 09:05:35 st-0003 kernel: 35905 pages dirty
May 21 09:05:35 st-0003 kernel: 1 pages writeback
May 21 09:05:35 st-0003 kernel: 929 pages mapped
May 21 09:05:35 st-0003 kernel: 6782 pages slab
May 21 09:05:35 st-0003 kernel: 57 pages pagetables
May 21 09:05:35 st-0003 kernel: swapper: page allocation failure. order:0,
mode:0x20
May 21 09:05:35 st-0003 kernel:  <c013c6ac> __alloc_pages+0x274/0x286
<c014af1d> cache_alloc_refill+0x2a6/0x45c
May 21 09:05:35 st-0003 kernel:  <c014b12b> __kmalloc+0x58/0x61   <c03d5bfc>
__alloc_skb+0x49/0xf5
May 21 09:05:35 st-0003 kernel:  <f88336b1>
e1000_alloc_rx_buffers+0x5c/0x2e5 [e1000]   <f8831570>
e1000_clean_rx_irq+0x48f/0x50a [e1000]
May 21 09:05:35 st-0003 kernel:  <f88342d2> e1000_intr+0x96/0xf4 [e1000]
<c0136927> handle_IRQ_event+0x20/0x4c
May 21 09:05:35 st-0003 kernel:  <c01369cf> __do_IRQ+0x7c/0xcd   <c0104a65>
do_IRQ+0x44/0x53
May 21 09:05:35 st-0003 kernel:  <c01030ba> common_interrupt+0x1a/0x20
<c01016f5> mwait_idle+0x20/0x34
May 21 09:05:35 st-0003 kernel:  <c01016c0> cpu_idle+0x5a/0x6f   <c05966c9>
start_kernel+0x317/0x31d
May 21 09:05:35 st-0003 kernel: Mem-info:
May 21 09:05:35 st-0003 kernel: DMA per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: DMA32 per-cpu: empty
May 21 09:05:35 st-0003 kernel: Normal per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:30
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:47
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:51
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:55
May 21 09:05:35 st-0003 kernel: HighMem per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:98
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:10
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:183
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:6
May 21 09:05:35 st-0003 kernel: Free pages:     1165384kB (1160424kB
HighMem)
May 21 09:05:35 st-0003 kernel: Active:3002 inactive:212594 dirty:35905
writeback:1 unstable:0 free:291346 slab:6782 mapped:929 pagetables:57
May 21 09:05:35 st-0003 kernel: DMA free:3548kB min:68kB low:84kB high:100kB
active:0kB inactive:5808kB present:16384kB pages_scanned:0
all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: DMA32 free:0kB min:0kB low:0kB high:0kB
active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: Normal free:1412kB min:3756kB low:4692kB
high:5632kB active:8kB inactive:838708kB present:901120kB pages_scanned:0
all_unrecla
imable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 9215
May 21 09:05:35 st-0003 kernel: HighMem free:1160424kB min:512kB low:1740kB
high:2972kB active:12000kB inactive:5860kB present:1179584kB pages_scanned:0
all_u
nreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 0
May 21 09:05:35 st-0003 kernel: DMA: 1*4kB 1*8kB 1*16kB 0*32kB 1*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 0*4096kB = 3548kB
May 21 09:05:35 st-0003 kernel: DMA32: empty
May 21 09:05:35 st-0003 kernel: Normal: 1*4kB 0*8kB 0*16kB 0*32kB 0*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 1412kB
May 21 09:05:35 st-0003 kernel: HighMem: 142*4kB 34*8kB 18*16kB 8*32kB
6*64kB 2*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 282*4096kB = 1160424kB
May 21 09:05:35 st-0003 kernel: Free swap:            0kB
May 21 09:05:35 st-0003 kernel: 524272 pages of RAM
May 21 09:05:35 st-0003 kernel: 294896 pages of HIGHMEM
May 21 09:05:35 st-0003 kernel: 6010 reserved pages
May 21 09:05:35 st-0003 kernel: 213818 pages shared
May 21 09:05:35 st-0003 kernel: 0 pages swap cached
May 21 09:05:35 st-0003 kernel: 35905 pages dirty
May 21 09:05:35 st-0003 kernel: 1 pages writeback
May 21 09:05:35 st-0003 kernel: 929 pages mapped
May 21 09:05:35 st-0003 kernel: 6782 pages slab
May 21 09:05:35 st-0003 kernel: 57 pages pagetables
May 21 09:05:35 st-0003 kernel: swapper: page allocation failure. order:0,
mode:0x20
May 21 09:05:35 st-0003 kernel:  <c013c6ac> __alloc_pages+0x274/0x286
<c014af1d> cache_alloc_refill+0x2a6/0x45c
May 21 09:05:35 st-0003 kernel:  <c014b12b> __kmalloc+0x58/0x61   <c03d5bfc>
__alloc_skb+0x49/0xf5
May 21 09:05:35 st-0003 kernel:  <f88336b1>
e1000_alloc_rx_buffers+0x5c/0x2e5 [e1000]   <f8831570>
e1000_clean_rx_irq+0x48f/0x50a [e1000]
May 21 09:05:35 st-0003 kernel:  <f88342d2> e1000_intr+0x96/0xf4 [e1000]
<c0136927> handle_IRQ_event+0x20/0x4c
May 21 09:05:35 st-0003 kernel:  <c01369cf> __do_IRQ+0x7c/0xcd   <c0104a65>
do_IRQ+0x44/0x53
May 21 09:05:35 st-0003 kernel:  <c01030ba> common_interrupt+0x1a/0x20
<c01016f5> mwait_idle+0x20/0x34
May 21 09:05:35 st-0003 kernel:  <c01016c0> cpu_idle+0x5a/0x6f   <c05966c9>
start_kernel+0x317/0x31d
May 21 09:05:35 st-0003 kernel: Mem-info:
May 21 09:05:35 st-0003 kernel: DMA per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: DMA32 per-cpu: empty
May 21 09:05:35 st-0003 kernel: Normal per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:30
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:47
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:51
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:55
May 21 09:05:35 st-0003 kernel: HighMem per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:98
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:10
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:183
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:6
May 21 09:05:35 st-0003 kernel: Free pages:     1165384kB (1160424kB
HighMem)
May 21 09:05:35 st-0003 kernel: Active:3002 inactive:212594 dirty:35905
writeback:1 unstable:0 free:291346 slab:6782 mapped:929 pagetables:57
May 21 09:05:35 st-0003 kernel: DMA free:3548kB min:68kB low:84kB high:100kB
active:0kB inactive:5808kB present:16384kB pages_scanned:0
all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: DMA32 free:0kB min:0kB low:0kB high:0kB
active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: Normal free:1412kB min:3756kB low:4692kB
high:5632kB active:8kB inactive:838708kB present:901120kB pages_scanned:0
all_unrecla
imable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 9215
May 21 09:05:35 st-0003 kernel: HighMem free:1160424kB min:512kB low:1740kB
high:2972kB active:12000kB inactive:5860kB present:1179584kB pages_scanned:0
all_u
nreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 0
May 21 09:05:35 st-0003 kernel: DMA: 1*4kB 1*8kB 1*16kB 0*32kB 1*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 0*4096kB = 3548kB
May 21 09:05:35 st-0003 kernel: DMA32: empty
May 21 09:05:35 st-0003 kernel: Normal: 1*4kB 0*8kB 0*16kB 0*32kB 0*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 1412kB
May 21 09:05:35 st-0003 kernel: HighMem: 142*4kB 34*8kB 18*16kB 8*32kB
6*64kB 2*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 282*4096kB = 1160424kB
May 21 09:05:35 st-0003 kernel: Free swap:            0kB
May 21 09:05:35 st-0003 kernel: 524272 pages of RAM
May 21 09:05:35 st-0003 kernel: 294896 pages of HIGHMEM
May 21 09:05:35 st-0003 kernel: 6010 reserved pages
May 21 09:05:35 st-0003 kernel: 213818 pages shared
May 21 09:05:35 st-0003 kernel: 0 pages swap cached
May 21 09:05:35 st-0003 kernel: 35905 pages dirty
May 21 09:05:35 st-0003 kernel: 1 pages writeback
May 21 09:05:35 st-0003 kernel: 929 pages mapped
May 21 09:05:35 st-0003 kernel: 6782 pages slab
May 21 09:05:35 st-0003 kernel: 57 pages pagetables
May 21 09:05:35 st-0003 kernel: swapper: page allocation failure. order:0,
mode:0x20
May 21 09:05:35 st-0003 kernel:  <c013c6ac> __alloc_pages+0x274/0x286
<c014af1d> cache_alloc_refill+0x2a6/0x45c
May 21 09:05:35 st-0003 kernel:  <c014b12b> __kmalloc+0x58/0x61   <c03d5bfc>
__alloc_skb+0x49/0xf5
May 21 09:05:35 st-0003 kernel:  <f88336b1>
e1000_alloc_rx_buffers+0x5c/0x2e5 [e1000]   <f8831570>
e1000_clean_rx_irq+0x48f/0x50a [e1000]
May 21 09:05:35 st-0003 kernel:  <f88342d2> e1000_intr+0x96/0xf4 [e1000]
<c0136927> handle_IRQ_event+0x20/0x4c
May 21 09:05:35 st-0003 kernel:  <c01369cf> __do_IRQ+0x7c/0xcd   <c0104a65>
do_IRQ+0x44/0x53
May 21 09:05:35 st-0003 kernel:  <c01030ba> common_interrupt+0x1a/0x20
<c01016f5> mwait_idle+0x20/0x34
May 21 09:05:35 st-0003 kernel:  <c01016c0> cpu_idle+0x5a/0x6f   <c05966c9>
start_kernel+0x317/0x31d
May 21 09:05:35 st-0003 kernel: Mem-info:
May 21 09:05:35 st-0003 kernel: DMA per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: DMA32 per-cpu: empty
May 21 09:05:35 st-0003 kernel: Normal per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:30
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:47
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:51
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:55
May 21 09:05:35 st-0003 kernel: HighMem per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:98
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:10
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:183
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:6
May 21 09:05:35 st-0003 kernel: Free pages:     1165384kB (1160424kB
HighMem)
May 21 09:05:35 st-0003 kernel: Active:3002 inactive:212594 dirty:35905
writeback:1 unstable:0 free:291346 slab:6782 mapped:929 pagetables:57
May 21 09:05:35 st-0003 kernel: DMA free:3548kB min:68kB low:84kB high:100kB
active:0kB inactive:5808kB present:16384kB pages_scanned:0
all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: DMA32 free:0kB min:0kB low:0kB high:0kB
active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: Normal free:1412kB min:3756kB low:4692kB
high:5632kB active:8kB inactive:838708kB present:901120kB pages_scanned:0
all_unrecla
imable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 9215
May 21 09:05:35 st-0003 kernel: HighMem free:1160424kB min:512kB low:1740kB
high:2972kB active:12000kB inactive:5860kB present:1179584kB pages_scanned:0
all_u
nreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 0
May 21 09:05:35 st-0003 kernel: DMA: 1*4kB 1*8kB 1*16kB 0*32kB 1*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 0*4096kB = 3548kB
May 21 09:05:35 st-0003 kernel: DMA32: empty
May 21 09:05:35 st-0003 kernel: Normal: 1*4kB 0*8kB 0*16kB 0*32kB 0*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 1412kB
May 21 09:05:35 st-0003 kernel: HighMem: 142*4kB 34*8kB 18*16kB 8*32kB
6*64kB 2*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 282*4096kB = 1160424kB
May 21 09:05:35 st-0003 kernel: Free swap:            0kB
May 21 09:05:35 st-0003 kernel: 524272 pages of RAM
May 21 09:05:35 st-0003 kernel: 294896 pages of HIGHMEM
May 21 09:05:35 st-0003 kernel: 6010 reserved pages
May 21 09:05:35 st-0003 kernel: 213818 pages shared
May 21 09:05:35 st-0003 kernel: 0 pages swap cached
May 21 09:05:35 st-0003 kernel: 35905 pages dirty
May 21 09:05:35 st-0003 kernel: 1 pages writeback
May 21 09:05:35 st-0003 kernel: 929 pages mapped
May 21 09:05:35 st-0003 kernel: 6782 pages slab
May 21 09:05:35 st-0003 kernel: 57 pages pagetables
May 21 09:05:35 st-0003 kernel: swapper: page allocation failure. order:0,
mode:0x20
May 21 09:05:35 st-0003 kernel:  <c013c6ac> __alloc_pages+0x274/0x286
<c014af1d> cache_alloc_refill+0x2a6/0x45c
May 21 09:05:35 st-0003 kernel:  <c014b12b> __kmalloc+0x58/0x61   <c03d5bfc>
__alloc_skb+0x49/0xf5
May 21 09:05:35 st-0003 kernel:  <f88336b1>
e1000_alloc_rx_buffers+0x5c/0x2e5 [e1000]   <f8831570>
e1000_clean_rx_irq+0x48f/0x50a [e1000]
May 21 09:05:35 st-0003 kernel:  <f88342d2> e1000_intr+0x96/0xf4 [e1000]
<c0136927> handle_IRQ_event+0x20/0x4c
May 21 09:05:35 st-0003 kernel:  <c01369cf> __do_IRQ+0x7c/0xcd   <c0104a65>
do_IRQ+0x44/0x53
May 21 09:05:35 st-0003 kernel:  <c01030ba> common_interrupt+0x1a/0x20
<c01016f5> mwait_idle+0x20/0x34
May 21 09:05:35 st-0003 kernel:  <c01016c0> cpu_idle+0x5a/0x6f   <c05966c9>
start_kernel+0x317/0x31d
May 21 09:05:35 st-0003 kernel: Mem-info:
May 21 09:05:35 st-0003 kernel: DMA per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: DMA32 per-cpu: empty
May 21 09:05:35 st-0003 kernel: Normal per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:30
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:47
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:51
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:55
May 21 09:05:35 st-0003 kernel: HighMem per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:98
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:10
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:183
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:6
May 21 09:05:35 st-0003 kernel: Free pages:     1165384kB (1160424kB
HighMem)
May 21 09:05:35 st-0003 kernel: Active:3002 inactive:212594 dirty:35905
writeback:1 unstable:0 free:291346 slab:6782 mapped:929 pagetables:57
May 21 09:05:35 st-0003 kernel: DMA free:3548kB min:68kB low:84kB high:100kB
active:0kB inactive:5808kB present:16384kB pages_scanned:0
all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: DMA32 free:0kB min:0kB low:0kB high:0kB
active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: Normal free:1412kB min:3756kB low:4692kB
high:5632kB active:8kB inactive:838708kB present:901120kB pages_scanned:0
all_unrecla
imable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 9215
May 21 09:05:35 st-0003 kernel: HighMem free:1160424kB min:512kB low:1740kB
high:2972kB active:12000kB inactive:5860kB present:1179584kB pages_scanned:0
all_u
nreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 0
May 21 09:05:35 st-0003 kernel: DMA: 1*4kB 1*8kB 1*16kB 0*32kB 1*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 0*4096kB = 3548kB
May 21 09:05:35 st-0003 kernel: DMA32: empty
May 21 09:05:35 st-0003 kernel: Normal: 1*4kB 0*8kB 0*16kB 0*32kB 0*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 1412kB
May 21 09:05:35 st-0003 kernel: HighMem: 142*4kB 34*8kB 18*16kB 8*32kB
6*64kB 2*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 282*4096kB = 1160424kB
May 21 09:05:35 st-0003 kernel: Free swap:            0kB
May 21 09:05:35 st-0003 kernel: 524272 pages of RAM
May 21 09:05:35 st-0003 kernel: 294896 pages of HIGHMEM
May 21 09:05:35 st-0003 kernel: 6010 reserved pages
May 21 09:05:35 st-0003 kernel: 213818 pages shared
May 21 09:05:35 st-0003 kernel: 0 pages swap cached
May 21 09:05:35 st-0003 kernel: 35905 pages dirty
May 21 09:05:35 st-0003 kernel: 1 pages writeback
May 21 09:05:35 st-0003 kernel: 929 pages mapped
May 21 09:05:35 st-0003 kernel: 6782 pages slab
May 21 09:05:35 st-0003 kernel: 57 pages pagetables
May 21 09:05:35 st-0003 kernel: swapper: page allocation failure. order:0,
mode:0x20
May 21 09:05:35 st-0003 kernel:  <c013c6ac> __alloc_pages+0x274/0x286
<c014af1d> cache_alloc_refill+0x2a6/0x45c
May 21 09:05:35 st-0003 kernel:  <c014b12b> __kmalloc+0x58/0x61   <c03d5bfc>
__alloc_skb+0x49/0xf5
May 21 09:05:35 st-0003 kernel:  <f88336b1>
e1000_alloc_rx_buffers+0x5c/0x2e5 [e1000]   <f8831570>
e1000_clean_rx_irq+0x48f/0x50a [e1000]
May 21 09:05:35 st-0003 kernel:  <f88342d2> e1000_intr+0x96/0xf4 [e1000]
<c0136927> handle_IRQ_event+0x20/0x4c
May 21 09:05:35 st-0003 kernel:  <c01369cf> __do_IRQ+0x7c/0xcd   <c0104a65>
do_IRQ+0x44/0x53
May 21 09:05:35 st-0003 kernel:  <c01030ba> common_interrupt+0x1a/0x20
<c01016f5> mwait_idle+0x20/0x34
May 21 09:05:35 st-0003 kernel:  <c01016c0> cpu_idle+0x5a/0x6f   <c05966c9>
start_kernel+0x317/0x31d
May 21 09:05:35 st-0003 kernel: Mem-info:
May 21 09:05:35 st-0003 kernel: DMA per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: DMA32 per-cpu: empty
May 21 09:05:35 st-0003 kernel: Normal per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:30
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:47
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:51
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:55
May 21 09:05:35 st-0003 kernel: HighMem per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:98
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:10
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:183
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:6
May 21 09:05:35 st-0003 kernel: Free pages:     1165384kB (1160424kB
HighMem)
May 21 09:05:35 st-0003 kernel: Active:3002 inactive:212594 dirty:35905
writeback:1 unstable:0 free:291346 slab:6782 mapped:929 pagetables:57
May 21 09:05:35 st-0003 kernel: DMA free:3548kB min:68kB low:84kB high:100kB
active:0kB inactive:5808kB present:16384kB pages_scanned:0
all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: DMA32 free:0kB min:0kB low:0kB high:0kB
active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: Normal free:1412kB min:3756kB low:4692kB
high:5632kB active:8kB inactive:838708kB present:901120kB pages_scanned:0
all_unrecla
imable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 9215
May 21 09:05:35 st-0003 kernel: HighMem free:1160424kB min:512kB low:1740kB
high:2972kB active:12000kB inactive:5860kB present:1179584kB pages_scanned:0
all_u
nreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 0
May 21 09:05:35 st-0003 kernel: DMA: 1*4kB 1*8kB 1*16kB 0*32kB 1*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 0*4096kB = 3548kB
May 21 09:05:35 st-0003 kernel: DMA32: empty
May 21 09:05:35 st-0003 kernel: Normal: 1*4kB 0*8kB 0*16kB 0*32kB 0*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 1412kB
May 21 09:05:35 st-0003 kernel: HighMem: 142*4kB 34*8kB 18*16kB 8*32kB
6*64kB 2*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 282*4096kB = 1160424kB
May 21 09:05:35 st-0003 kernel: Free swap:            0kB
May 21 09:05:35 st-0003 kernel: 524272 pages of RAM
May 21 09:05:35 st-0003 kernel: 294896 pages of HIGHMEM
May 21 09:05:35 st-0003 kernel: 6010 reserved pages
May 21 09:05:35 st-0003 kernel: 213818 pages shared
May 21 09:05:35 st-0003 kernel: 0 pages swap cached
May 21 09:05:35 st-0003 kernel: 35905 pages dirty
May 21 09:05:35 st-0003 kernel: 1 pages writeback
May 21 09:05:35 st-0003 kernel: 929 pages mapped
May 21 09:05:35 st-0003 kernel: 6782 pages slab
May 21 09:05:35 st-0003 kernel: 57 pages pagetables
#1 lore raw
Quoted 2 lines · Haar J?nos

This server have 2GB ram, and ~1.1G always free!
Anybody have an idea?

you ran out of lowmem?

what kernel is this and how do you trigger it?

#2 lore raw

----- Original Message -----
From: "Chris Wedgwood" <[email protected]>
To: "Haar J?nos" <[email protected]>
Cc: <[email protected]>
Sent: Sunday, May 21, 2006 10:16 AM
Subject: Re: swapper: page allocation failure.

Quoted 8 lines · Haar J?nos
Quoted 3 lines

This server have 2GB ram, and ~1.1G always free!
Anybody have an idea?

you ran out of lowmem?

what kernel is this and how do you trigger it?

I have 4 disk nodes that acts as nbd-server.

The only processes:

  PID TTY      STAT   TIME COMMAND
    1 ?        S      0:12 init [3]
    2 ?        SW     0:00 [migration/0]
    3 ?        SWN    0:19 [ksoftirqd/0]
    4 ?        SW     0:00 [migration/1]
    5 ?        SWN    0:03 [ksoftirqd/1]
    6 ?        SW<    0:00 [events/0]
    7 ?        SW<    0:00 [events/1]
    8 ?        SW<    0:00 [khelper]
    9 ?        SW<    0:00 [kthread]
   12 ?        SW<    0:07  \_ [kblockd/0]
   13 ?        SW<    0:02  \_ [kblockd/1]
   14 ?        SW<    0:00  \_ [kacpid]
  101 ?        SW<    0:00  \_ [khubd]
  103 ?        SW<    0:00  \_ [kseriod]
  247 ?        SW    12:09  \_ [pdflush]
  248 ?        SW    10:04  \_ [pdflush]
  250 ?        SW<    0:00  \_ [aio/0]
  251 ?        SW<    0:00  \_ [aio/1]
  252 ?        SW<    0:00  \_ [xfslogd/0]
  253 ?        SW<    0:00  \_ [xfslogd/1]
  254 ?        SW<    0:00  \_ [xfsdatad/0]
  255 ?        SW<    0:00  \_ [xfsdatad/1]
  921 ?        SW<    0:00  \_ [ata/0]
  922 ?        SW<    0:00  \_ [ata/1]
  926 ?        SW<    0:00  \_ [scsi_eh_0]
  927 ?        SW<    0:00  \_ [scsi_eh_1]
  935 ?        SW<    0:00  \_ [scsi_eh_2]
  936 ?        SW<    0:00  \_ [scsi_eh_3]
  937 ?        SW<    0:00  \_ [scsi_eh_4]
  938 ?        SW<    0:00  \_ [scsi_eh_5]
  951 ?        SW<    0:00  \_ [scsi_eh_6]
  952 ?        SW<    0:00  \_ [scsi_eh_7]
 1045 ?        SW<    0:00  \_ [exec-osm/0]
 1047 ?        SW<    0:00  \_ [exec-osm/1]
 1088 ?        SW<    0:00  \_ [kcryptd/0]
 1089 ?        SW<    0:00  \_ [kcryptd/1]
 1090 ?        SW<    0:00  \_ [kmpathd/0]
 1091 ?        SW<    0:00  \_ [kmpathd/1]
 1092 ?        SW<    0:00  \_ [kmirrord]
 1093 ?        SW<    0:14  \_ [kedac]
 1111 ?        SW<  219:01  \_ [md0_raid5]
 1112 ?        SW<    0:23  \_ [rpciod/0]
 1113 ?        SW<    0:00  \_ [rpciod/1]
  249 ?        SW    51:30 [kswapd0]
 2532 ?        S      0:05 syslogd -m 0
 2536 ?        S      0:00 klogd -x
 2615 ?        S      0:00 /usr/sbin/sshd
21484 ?        S      0:00  \_ /usr/sbin/sshd
21486 pts/2    S      0:00      \_ -bash
22444 pts/2    R      0:00          \_ ps fax
 2629 ?        S      0:03 crond
 2638 ?        S      0:55 /bin/bwbar --text-file

/mnt/DY_SYSTEM/sysinfo/st4-eth0_o.php --png-file /mnt/DY_SYSTEM/sysinfo/st4-eth0_o.png eth0 100 -x 300
2639 ? S 0:54 /bin/bwbar --text-file
/mnt/DY_SYSTEM/sysinfo/st4-eth0_i.php -i --png-file /mnt/DY_SYSTEM/sysinfo/st4-eth0_i.png eth0 100 -x 300

 2674 tty2     S      0:00 /sbin/mingetty tty2
 2675 tty3     S      0:00 /sbin/mingetty tty3
 2676 tty4     S      0:00 /sbin/mingetty tty4
 2677 tty5     S      0:00 /sbin/mingetty tty5
 2678 tty6     S      0:00 /sbin/mingetty tty6
 2925 tty1     S      0:00 /sbin/mingetty tty1 --noclear
 5167 ?        SW     0:00 [lockd]
21290 ?        S<     0:00 /usr/local/bin/nbd-server 1230 /dev/md0
21780 ?        S<     0:38  \_ /usr/local/bin/nbd-server 1230 /dev/md0

The kernel : 2.6.17-rc3-git1, but if i have right, the 2.6.16.1 is the same if i try to swich back. :-)

2 of 4 nodes frequently reboots without any error message. I have turned on some debug option, and catch this message.

At this point i watch the free -l -s 1, and i can see, the nbd-server (or kernel) didn't use the highmem for buffer cache, but why?

total used free shared buffers cached

Mem:       2073048     893360    1179688          0     829092      19820
Low:        893464     868352      25112          0          0          0
High:      1179584      25008    1154576          0          0          0

-/+ buffers/cache: 44448 2028600
Swap: 0 0 0

(min_free_kbytes = 16000 now.)

Cheers,
Janos

#3 lore raw
Quoted 6 lines · Haar J?nos

total used free shared buffers cached

Mem:       2073048     893360    1179688          0     829092      19820
Low:        893464     868352      25112          0          0          0
High:      1179584      25008    1154576          0          0          0

-/+ buffers/cache: 44448 2028600
Swap: 0 0 0

looks bad for lowmem

what does /proc/meminfo say?

what about the output from "slabtop -sc" ?

#4 lore raw

----- Original Message -----
From: "Chris Wedgwood" <[email protected]>
To: "Haar J?nos" <[email protected]>
Cc: <[email protected]>
Sent: Sunday, May 21, 2006 10:47 AM
Subject: Re: swapper: page allocation failure.

Quoted 3 lines · Haar J?nos
Quoted 2 lines

total used free shared buffers

cached

Quoted 1 line
Quoted 1 line

Mem: 2073048 893360 1179688 0 829092

19820

Quoted 1 line
Quoted 1 line

Low: 893464 868352 25112 0 0

0

Quoted 1 line
Quoted 1 line

High: 1179584 25008 1154576 0 0

0

Quoted 6 lines
Quoted 2 lines

-/+ buffers/cache: 44448 2028600
Swap: 0 0 0

looks bad for lowmem

what does /proc/meminfo say?

[root@st-0001 vm]# cat /proc/meminfo

MemTotal:      2073048 kB
MemFree:       1179376 kB
Buffers:        829764 kB
Cached:          19896 kB
SwapCached:          0 kB
Active:          15604 kB
Inactive:       837636 kB
HighTotal:     1179584 kB
HighFree:      1154736 kB
LowTotal:       893464 kB
LowFree:         24640 kB
SwapTotal:           0 kB
SwapFree:            0 kB
Dirty:           21352 kB
Writeback:           0 kB
Mapped:           7000 kB
Slab:            22612 kB
CommitLimit:   1036524 kB
Committed_AS:    10968 kB
PageTables:        284 kB
VmallocTotal:   114680 kB
VmallocUsed:       308 kB
VmallocChunk:   114036 kB
Quoted 2 lines

what about the output from "slabtop -sc" ?

Not installed.
Wich package or where can i find the source? (i use redhat 9.0)

Cheers,
Janos

#5 lore raw
Quoted 2 lines · Haar J?nos
MemTotal:      2073048 kB
MemFree:       1179376 kB

fine

Quoted 1 line

Buffers: 829764 kB

ok

Quoted 5 lines
Cached:          19896 kB
SwapCached:          0 kB
Active:          15604 kB

Inactive: 837636 kB

hrm

Quoted 2 lines
HighTotal:     1179584 kB
HighFree:      1154736 kB

krm

Quoted 2 lines
LowTotal:       893464 kB
LowFree:         24640 kB

bad

Quoted 2 lines
SwapTotal:           0 kB
SwapFree:            0 kB

ok

Quoted 1 line

Dirty: 21352 kB

ok

Quoted 3 lines
Writeback:           0 kB
Mapped:           7000 kB
Slab:            22612 kB

ok

you have very little low

Quoted 1 line

Not installed.

urgh

Quoted 1 line

Wich package or where can i find the source? (i use redhat 9.0)

google i guess, i have very little idea how to drive RH to be honest

anyhow, it's not the slab

something is eating/using/leaking all your lowmemory

what kernel version is this?
how long has the machine been up?
do you see it get worse over time?

#6 lore raw

----- Original Message -----
From: "Chris Wedgwood" <[email protected]>
To: "Haar J?nos" <[email protected]>
Cc: <[email protected]>
Sent: Sunday, May 21, 2006 11:10 AM
Subject: Re: swapper: page allocation failure.

Quoted 52 lines · Haar J?nos
Quoted 3 lines
MemTotal:      2073048 kB
MemFree:       1179376 kB

fine

Quoted 1 line

Buffers: 829764 kB

ok

Quoted 5 lines
Cached:          19896 kB
SwapCached:          0 kB
Active:          15604 kB

Inactive: 837636 kB

hrm

Quoted 2 lines
HighTotal:     1179584 kB
HighFree:      1154736 kB

krm

Quoted 2 lines
LowTotal:       893464 kB
LowFree:         24640 kB

bad

Quoted 2 lines
SwapTotal:           0 kB
SwapFree:            0 kB

ok

Quoted 1 line

Dirty: 21352 kB

ok

Quoted 3 lines
Writeback:           0 kB
Mapped:           7000 kB
Slab:            22612 kB

ok

you have very little low

Quoted 1 line

Not installed.

Quoted 7 lines

urgh

Quoted 1 line

Wich package or where can i find the source? (i use redhat 9.0)

google i guess, i have very little idea how to drive RH to be honest

anyhow, it's not the slab

I found it already, thanks.

Quoted 6 lines

something is eating/using/leaking all your lowmemory

what kernel version is this?

[root@st-0001 /]# uname -a
Linux st-0001 2.6.17-rc3-git1 #2 SMP Sun May 21 01:12:22 CEST 2006 i686 i686 i386 GNU/Linux

Quoted 1 line

how long has the machine been up?

[root@st-0001 /]# uptime
11:22:10 up 2:52, 1 user, load average: 0.35, 0.42, 0.43

Quoted 1 line

do you see it get worse over time?

No.
This is a simple disk node.
It serves the md0 array, and uses mem for buffering-caching. If it reboots, fill the memory on the first couple of minutes, and stay on full, but this is relatively good.

But 2 question is remaining:

1. why don't use highmem for caching?
2. why can not allocate enough lowmem from shared-buffer for the e1000 driver if it needs some memory?

Cheers,
Janos

Quoted 1 line
#7 lore raw
Quoted 3 lines · Haar J?nos

1. why don't use highmem for caching?
2. why can not allocate enough lowmem from shared-buffer for the e1000 driver if it needs some memory?

highmem can't be used as freely as lowmem, it has additional complexity and a slight over head that makes it hard or impossible to use in many places

#8 lore raw

----- Original Message -----
From: "Chris Wedgwood" <[email protected]>
To: "Haar J?nos" <[email protected]>
Cc: <[email protected]>
Sent: Sunday, May 21, 2006 12:24 PM
Subject: Re: swapper: page allocation failure.

Quoted 9 lines · Haar J?nos
Quoted 4 lines

1. why don't use highmem for caching?
2. why can not allocate enough lowmem from shared-buffer for the e1000 driver if it needs some memory?

highmem can't be used as freely as lowmem, it has additional complexity and a slight over head that makes it hard or impossible to use in many places

OK, i understand this, but buffer-cache is an optional thing, and looks like really simple (for me), and i cannot understand why cannot use exactly for buffering the highmem...

On my concentrator the nbd-client uses ~ 3.5GB of ram for PAGE-cache. The buffer cache why can not use the highmem?
Whats the difference?

Cheers,
Janos

#9 lore raw
In reply to Haar János
Quoted 2 lines · Haar J?nos

[root@st-0001 /]# uname -a
Linux st-0001 2.6.17-rc3-git1 #2 SMP Sun May 21 01:12:22 CEST 2006 i686 i686 i386 GNU/Linux

did earlier kernels work OK?

Quoted 2 lines

This is a simple disk node.
It serves the md0 array, and uses mem for buffering-caching.

odd, i looks like you've leaked alot of lowmem but i can't think why

i've got major (induced) brain-fog right now so i'll have to think about it tomorrow sorry

#10 lore raw

----- Original Message -----
From: "Chris Wedgwood" <[email protected]>
To: "Haar J?nos" <[email protected]>
Cc: <[email protected]>
Sent: Sunday, May 21, 2006 12:26 PM
Subject: Re: swapper: page allocation failure.

Quoted 4 lines · Haar J?nos
Quoted 3 lines

[root@st-0001 /]# uname -a
Linux st-0001 2.6.17-rc3-git1 #2 SMP Sun May 21 01:12:22 CEST 2006 i686

i686 i386 GNU/Linux

Quoted 2 lines

did earlier kernels work OK?

No, the my reboot problem starts about january, but my tracking progress is slow because this is the first error message.
(On january i have upgrade the nodes, and few things are changed.)

Quoted 8 lines
Quoted 3 lines

This is a simple disk node.
It serves the md0 array, and uses mem for buffering-caching.

odd, i looks like you've leaked alot of lowmem but i can't think why

i've got major (induced) brain-fog right now so i'll have to think about it tomorrow sorry

OK, thanks.

Cheers,
Janos

#11 lore raw
In reply to Chris Wedgwood
Quoted 10 lines · Chris Wedgwood
Quoted 2 lines · Haar J?nos

[root@st-0001 /]# uname -a
Linux st-0001 2.6.17-rc3-git1 #2 SMP Sun May 21 01:12:22 CEST 2006 i686 i686 i386 GNU/Linux

did earlier kernels work OK?

Quoted 8 lines
Quoted 2 lines

This is a simple disk node.
It serves the md0 array, and uses mem for buffering-caching.

odd, i looks like you've leaked alot of lowmem but i can't think why

i've got major (induced) brain-fog right now so i'll have to think about it tomorrow sorry

The buffers are buffercache rather than the usual pagecache; due to nbd I guess. Buffercache cannot be satisfied by highmem.

This would be a relatively uncommon setup, which explains why it isn't working 100%. I don't know of any reason why reclaim speed should be worse for buffercache, however one notable thing will be that zone_normal's lowmem reserve that is untouchable by pagecache will be eaten by buffercache...

Anyway, increasing /proc/sys/vm/min_free_kbytes should help. Janos, perhaps you could try doubling it and see how you go?

Signature · 2 lines
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com 
#12 lore raw

----- Original Message -----
From: "Nick Piggin" <[email protected]>
To: "Chris Wedgwood" <[email protected]>
Cc: "Haar J?nos" <[email protected]>; <[email protected]>
Sent: Sunday, May 21, 2006 2:01 PM
Subject: Re: swapper: page allocation failure.

Quoted 6 lines
Quoted 5 lines · Chris Wedgwood
Quoted 2 lines · Haar J?nos

[root@st-0001 /]# uname -a
Linux st-0001 2.6.17-rc3-git1 #2 SMP Sun May 21 01:12:22 CEST 2006 i686

i686 i386 GNU/Linux

Quoted 17 lines
Quoted 5 lines

did earlier kernels work OK?

Quoted 8 lines
Quoted 2 lines

This is a simple disk node.
It serves the md0 array, and uses mem for buffering-caching.

odd, i looks like you've leaked alot of lowmem but i can't think why

i've got major (induced) brain-fog right now so i'll have to think about it tomorrow sorry

The buffers are buffercache rather than the usual pagecache; due to nbd I guess. Buffercache cannot be satisfied by highmem.

Quoted 8 lines

This would be a relatively uncommon setup, which explains why it isn't working 100%. I don't know of any reason why reclaim speed should be worse for buffercache, however one notable thing will be that zone_normal's lowmem reserve that is untouchable by pagecache will be eaten by buffercache...

Anyway, increasing /proc/sys/vm/min_free_kbytes should help. Janos, perhaps you could try doubling it and see how you go?

I did it allready, and it looks like solves the problem. Yesterday i have more than 6 random reboots, and after i set from 3800 to 16000 the min free limit, i have none at this point. :-)

15:51:45 up 7:21, 1 user, load average: 0.85, 0.79, 0.67

Anyway, i interested about cache/buffer mechanism, because i have some performance problems too, and i can see, these systems wastes the half of memory instead of speeds up the operation.

Thanks,
Janos

Quoted 4 lines
Signature · 3 lines
-- 
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com
#13 lore raw
Quoted 5 lines · Haar János

I did it allready, and it looks like solves the problem. Yesterday i have more than 6 random reboots, and after i set from 3800 to 16000 the min free limit, i have none at this point. :-)

15:51:45 up 7:21, 1 user, load average: 0.85, 0.79, 0.67

Oh that's good. It's sad that you had random reboots though :(

Quoted 4 lines

Anyway, i interested about cache/buffer mechanism, because i have some performance problems too, and i can see, these systems wastes the half of memory instead of speeds up the operation.

Yeah, as I said, block device's pagecache (aka buffercache) can't use highmem. If nbd can export regular files as block devices, or you use loop devices from regular files, that might help (or slow things down :P).

Signature · 2 lines
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com 
#14 lore raw

----- Original Message -----
From: "Nick Piggin" <[email protected]>
To: "Haar János" <[email protected]>
Cc: "Chris Wedgwood" <[email protected]>; <[email protected]>
Sent: Monday, May 22, 2006 2:09 AM
Subject: Re: swapper: page allocation failure.

Quoted 4 lines
Quoted 2 lines · Haar János

I did it allready, and it looks like solves the problem. Yesterday i have more than 6 random reboots, and after i set from 3800

to

Quoted 5 lines
Quoted 3 lines

16000 the min free limit, i have none at this point. :-)

15:51:45 up 7:21, 1 user, load average: 0.85, 0.79, 0.67

Oh that's good. It's sad that you had random reboots though :(

09:12:44 up 1 day, 43 min, 1 user, load average: 0.48, 0.44, 0.42

It really fixes this issue. :-)

Thanks to you all!

Quoted 4 lines
Quoted 4 lines

Anyway, i interested about cache/buffer mechanism, because i have some performance problems too, and i can see, these systems wastes the half

of

Quoted 6 lines
Quoted 1 line

memory instead of speeds up the operation.

Yeah, as I said, block device's pagecache (aka buffercache) can't use highmem. If nbd can export regular files as block devices, or you use loop devices from regular files, that might help (or slow things down :P).

Hmm.
That sounds bad.
I think, if highmem is unreachable some times that makes lowmem more valuable!
The kernel needs to keep (reserve) it free as much as possible. The buffer-cache is an unimportant thing next to keeping lowmem free, but it is blocks the performance and wastes the systems resources!

It is possible any workaround?

The NBD is finally fixed, and production ready.
The big systems should use it, because it is easy. Additionally the big system usually needs the maximum of performance..... (+ the good cache/buffer extends the hardware lifetime.)

I think it is important to be changed.

(e.g. i use 4 nodes, 4x 3.3TB to use it as one big blockdev.)

Cheers,
Janos

Quoted 9 lines
Signature · 8 lines
-- 
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
#15 lore raw
Quoted 15 lines · Haar János

----- Original Message -----
From: "Nick Piggin" <[email protected]>

Quoted 4 lines

Yeah, as I said, block device's pagecache (aka buffercache) can't use highmem. If nbd can export regular files as block devices, or you use loop devices from regular files, that might help (or slow things down :P).

Hmm.
That sounds bad.
I think, if highmem is unreachable some times that makes lowmem more valuable!
The kernel needs to keep (reserve) it free as much as possible. The buffer-cache is an unimportant thing next to keeping lowmem free, but it is blocks the performance and wastes the systems resources!

Quoted 1 line

It is possible any workaround?

Try with one of the alternative vmsplit options that gives you more lowmem? That might break certain applications though.

Signature · 1 line
-ck
#16 lore raw

----- Original Message -----
From: "Con Kolivas" <[email protected]>
To: <[email protected]>
Cc: "Haar János" <[email protected]>; "Nick Piggin" <[email protected]>; <[email protected]>
Sent: Monday, May 22, 2006 1:17 PM
Subject: Re: swapper: page allocation failure.

Quoted 8 lines · Haar János
Quoted 7 lines

----- Original Message -----
From: "Nick Piggin" <[email protected]>

Quoted 4 lines

Yeah, as I said, block device's pagecache (aka buffercache) can't use highmem. If nbd can export regular files as block devices, or you use loop devices from regular files, that might help (or slow things down :P).

Quoted 6 lines
Quoted 6 lines

Hmm.
That sounds bad.
I think, if highmem is unreachable some times that makes lowmem more valuable!
The kernel needs to keep (reserve) it free as much as possible. The buffer-cache is an unimportant thing next to keeping lowmem free,

but

Quoted 5 lines
Quoted 3 lines

it is blocks the performance and wastes the systems resources!

It is possible any workaround?

Try with one of the alternative vmsplit options that gives you more

lowmem?

Quoted 1 line

That might break certain applications though.

total used free shared buffers cached

Mem:       4049724    4021196      28528          0      16384    3217288
Low:       4049724    4021196      28528
High:            0          0          0

-/+ buffers/cache: 787524 3262200
Swap: 0 0 0

This is an 64 bit machine, the "concentrator".

It looks like use all, the 4G ram as "lowmem".
If i replace the cpu on my nodes to 64bit capable ones, i can use all the memory as buffer-cache? :-)

Cheers,
Janos

Quoted 8 lines
Signature · 1 line
-- 
(quoted diff) +0 -2
-ck
-

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#17 lore raw
Quoted 12 lines · Haar János

----- Original Message -----
From: "Con Kolivas" <[email protected]>

Quoted 1 line

Try with one of the alternative vmsplit options that gives you more

lowmem?

Quoted 1 line

That might break certain applications though.

total used free shared buffers cached

Mem:       4049724    4021196      28528          0      16384    3217288
Low:       4049724    4021196      28528
High:            0          0          0
Quoted 8 lines

-/+ buffers/cache: 787524 3262200
Swap: 0 0 0

This is an 64 bit machine, the "concentrator".

It looks like use all, the 4G ram as "lowmem".
If i replace the cpu on my nodes to 64bit capable ones, i can use all the memory as buffer-cache? :-)

Heh yes indeed. It's only if you're stuck on 32bit for whatever reason that you'd need a different vmsplit. There is no need for highmem when 64bit allows bazillions of bytes of lowmem :)

Signature · 1 line
-ck
#18 lore raw

Re: swapper: page allocation failure. - random reboot problem

----- Original Message -----
From: "Con Kolivas" <[email protected]>
To: "Haar János" <[email protected]>
Cc: <[email protected]>; <[email protected]>; <[email protected]>
Sent: Monday, May 22, 2006 5:12 PM
Subject: Re: swapper: page allocation failure.

Quoted 10 lines · Haar János
Quoted 9 lines

----- Original Message -----
From: "Con Kolivas" <[email protected]>

Quoted 1 line

Try with one of the alternative vmsplit options that gives you more

lowmem?

Quoted 1 line

That might break certain applications though.

total used free shared buffers

cached

Quoted 1 line
Quoted 1 line

Mem: 4049724 4021196 28528 0 16384

3217288

Quoted 9 lines
Quoted 9 lines
Low:       4049724    4021196      28528
High:            0          0          0

-/+ buffers/cache: 787524 3262200
Swap: 0 0 0

This is an 64 bit machine, the "concentrator".

It looks like use all, the 4G ram as "lowmem".
If i replace the cpu on my nodes to 64bit capable ones, i can use all

the

Quoted 3 lines
Quoted 1 line

memory as buffer-cache? :-)

Heh yes indeed. It's only if you're stuck on 32bit for whatever reason

that

Quoted 2 lines

you'd need a different vmsplit. There is no need for highmem when 64bit allows bazillions of bytes of lowmem :)

OK, it is enough, to switch to 64bit, thanks!

But i have a little problem.
My node #3 reboots again.

At this point i have run out of ideas. :-(

This is checked already:

- the complete hardware, except the 12 hdd. (smart reports, no errors at all, 4x ide + 8xSATA all 300GB.)
- the SMP race. (checked with non-smp kernel)
- APIC/ACPI (tested with non... kernel)
- the e1000 driver (tested with realtek gige adapter)
- the complete filesystem, OS (NFS-ROOT, and copy between nodes.)
- the memory allocation proble, (checked with debug-kernel, and rised min_free_kbytes)

The systems only service is nbd. (nbd-server serving md0, raid4 array)

Anybody have an idea?

Please let me know!

Thanks,
Janos

Quoted 8 lines
Signature · 1 line
-- 
(quoted diff) +0 -2
-ck
-

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#19 lore raw

Re: swapper: page allocation failure. - random reboot problem

Quoted 18 lines · Haar János

OK, it is enough, to switch to 64bit, thanks!

But i have a little problem.
My node #3 reboots again.

At this point i have run out of ideas. :-(

This is checked already:

- the complete hardware, except the 12 hdd. (smart reports, no errors at all, 4x ide + 8xSATA all 300GB.)
- the SMP race. (checked with non-smp kernel)
- APIC/ACPI (tested with non... kernel)
- the e1000 driver (tested with realtek gige adapter)
- the complete filesystem, OS (NFS-ROOT, and copy between nodes.)
- the memory allocation proble, (checked with debug-kernel, and rised min_free_kbytes)

Quoted 3 lines

The systems only service is nbd. (nbd-server serving md0, raid4 array)

Anybody have an idea?

Bad hardware. Run memtest overnight. Can your power supply deal with that many drives? etc.

Signature · 2 lines
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com 
#20 lore raw

Re: swapper: page allocation failure. - random reboot problem

----- Original Message -----
From: "Nick Piggin" <[email protected]>
To: "Haar János" <[email protected]>
Cc: "Con Kolivas" <[email protected]>; <[email protected]>; <[email protected]>
Sent: Tuesday, May 23, 2006 11:17 AM
Subject: Re: swapper: page allocation failure. - random reboot problem

Quoted 20 lines
Quoted 18 lines · Haar János

OK, it is enough, to switch to 64bit, thanks!

But i have a little problem.
My node #3 reboots again.

At this point i have run out of ideas. :-(

This is checked already:

- the complete hardware, except the 12 hdd. (smart reports, no errors at all, 4x ide + 8xSATA all 300GB.)
- the SMP race. (checked with non-smp kernel)
- APIC/ACPI (tested with non... kernel)
- the e1000 driver (tested with realtek gige adapter)
- the complete filesystem, OS (NFS-ROOT, and copy between nodes.)
- the memory allocation proble, (checked with debug-kernel, and rised min_free_kbytes)

Quoted 6 lines
Quoted 3 lines

The systems only service is nbd. (nbd-server serving md0, raid4 array)

Anybody have an idea?

Bad hardware. Run memtest overnight. Can your power supply deal with that many drives? etc.

Sorry, i have allready did these things.

The motherboard + CPU + RAM successed the overnight memtest, but anyway i have replaced this group with another pre-tested ones, but no change! (Additionally, i replaced the NIC [e1000 to e1000, and e1000 to realtek], the sata cards [promise to promise], the sata and ide cables, mb, cpu, ram, ps, and the power cable too.
Only the 12hdd is the same, but smart reports no errors at all!) The power supply is the 3rd. and the problem is the same.

This is really a software bug, but i dont know exactly where.

Cheers,
Janos

Quoted 9 lines
Signature · 8 lines
-- 
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
#21 lore raw

Re: swapper: page allocation failure. - random reboot problem

Quoted 9 lines · Haar János

Sorry, i have allready did these things.

The motherboard + CPU + RAM successed the overnight memtest, but anyway i have replaced this group with another pre-tested ones, but no change! (Additionally, i replaced the NIC [e1000 to e1000, and e1000 to realtek], the sata cards [promise to promise], the sata and ide cables, mb, cpu, ram, ps, and the power cable too.
Only the 12hdd is the same, but smart reports no errors at all!) The power supply is the 3rd. and the problem is the same.

But is the power supply rated enough to support all the drives? I have seen random reboots where the power supply wasn't good enough.

Quoted 2 lines

This is really a software bug, but i dont know exactly where.

Could be. memtest doesn't guarantee anything though...

Signature · 2 lines
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com 
#22 lore raw

Re: swapper: page allocation failure. - random reboot problem

----- Original Message -----
From: "Nick Piggin" <[email protected]>
To: "Haar János" <[email protected]>
Cc: <[email protected]>; <[email protected]>; <[email protected]>
Sent: Tuesday, May 23, 2006 12:24 PM
Subject: Re: swapper: page allocation failure. - random reboot problem

Quoted 5 lines
Quoted 3 lines · Haar János

Sorry, i have allready did these things.

The motherboard + CPU + RAM successed the overnight memtest, but anyway

i

Quoted 2 lines
Quoted 2 lines

have replaced this group with another pre-tested ones, but no change! (Additionally, i replaced the NIC [e1000 to e1000, and e1000 to

realtek],

Quoted 1 line
Quoted 1 line

the sata cards [promise to promise], the sata and ide cables, mb, cpu,

ram,

Quoted 7 lines
Quoted 3 lines

ps, and the power cable too.
Only the 12hdd is the same, but smart reports no errors at all!) The power supply is the 3rd. and the problem is the same.

But is the power supply rated enough to support all the drives? I have seen random reboots where the power supply wasn't good enough.

This is the 3rd modell, currently 550W.
The system is P4 3G.

The another 2 stable node uses only 460W, and all hardware is equal. But i tried to swap ps between the stable and unstabe nodes, but nothing is changed....

Cheers,
Janos

Quoted 14 lines
Quoted 3 lines

This is really a software bug, but i dont know exactly where.

Could be. memtest doesn't guarantee anything though...

Signature · 7 lines
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
#23 lore raw

Re: swapper: page allocation failure. - random reboot problem

Quoted 7 lines · Haar János

----- Original Message -----

Quoted 3 lines

But is the power supply rated enough to support all the drives? I have seen random reboots where the power supply wasn't good enough.

Quoted 6 lines

This is the 3rd modell, currently 550W.
The system is P4 3G.

The another 2 stable node uses only 460W, and all hardware is equal. But i tried to swap ps between the stable and unstabe nodes, but nothing is changed....

Not sure then, sorry.

If it is a software problem, then if you can narrow it down further (eg. kernel 2.6.15 worked, 2.6.16 did not), or find a reproducable test case for it, then some more progress might be made.

Signature · 2 lines
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com 
#24 lore raw

Re: swapper: page allocation failure. - random reboot problem

----- Original Message -----
From: "Nick Piggin" <[email protected]>
To: "Haar János" <[email protected]>
Cc: <[email protected]>; <[email protected]>; <[email protected]>
Sent: Tuesday, May 23, 2006 12:45 PM
Subject: Re: swapper: page allocation failure. - random reboot problem

Quoted 8 lines
Quoted 7 lines · Haar János

----- Original Message -----

Quoted 3 lines

But is the power supply rated enough to support all the drives? I have seen random reboots where the power supply wasn't good enough.

Quoted 5 lines
Quoted 5 lines

This is the 3rd modell, currently 550W.
The system is P4 3G.

The another 2 stable node uses only 460W, and all hardware is equal. But i tried to swap ps between the stable and unstabe nodes, but nothing

is

Quoted 3 lines
Quoted 1 line

changed....

Not sure then, sorry.

OK, what do you recommend? :-)

Quoted 4 lines

If it is a software problem, then if you can narrow it down further (eg. kernel 2.6.15 worked, 2.6.16 did not), or find a reproducable test case for it, then some more progress might be made.

Yes, you have right!
I try it allready, but i cannot step back enough, because my sata card works only on 2.6.16+

Before i use the promise sata cards, and the sata 300G hdds, i use 2.6.15 kernel, and all my 4 nodes was really stable.
But this is not enough to exactly find the problem. :-(

Anyway, Hetbert Xu helps me a lot to track down the problem, but we only can close out some thing.
If i have right, the problem is about libata, promise driver, sata-error-handling or similar, but not so sure.

If there is no way to software debug, i will swap all the 12 hdd between 2 disk nodes, and this can show i have right, or not.

Cheers,
Janos

Quoted 4 lines
Signature · 3 lines
-- 
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com
#25 lore raw
In reply to Chris Wedgwood

----- Original Message -----
From: "Chris Wedgwood" <[email protected]>
To: "Haar J?nos" <[email protected]>
Cc: <[email protected]>
Sent: Sunday, May 21, 2006 10:47 AM
Subject: Re: swapper: page allocation failure.

Quoted 3 lines · Haar J?nos
Quoted 2 lines

total used free shared buffers

cached

Quoted 1 line
Quoted 1 line

Mem: 2073048 893360 1179688 0 829092

19820

Quoted 1 line
Quoted 1 line

Low: 893464 868352 25112 0 0

0

Quoted 1 line
Quoted 1 line

High: 1179584 25008 1154576 0 0

0

Quoted 8 lines
Quoted 2 lines

-/+ buffers/cache: 44448 2028600
Swap: 0 0 0

looks bad for lowmem

what does /proc/meminfo say?

what about the output from "slabtop -sc" ?

Here comes:

 Active / Total Objects (% used)    : 243896 / 258956 (94.2%)
 Active / Total Slabs (% used)      : 6243 / 6261 (99.7%)
 Active / Total Caches (% used)     : 76 / 151 (50.3%)
 Active / Total Size (% used)       : 20769.86K / 23348.68K (89.0%)

Minimum / Average / Maximum Object : 0.01K / 0.09K / 128.00K

  OBJS ACTIVE  USE OBJ SIZE  SLABS OBJ/SLAB CACHE SIZE NAME
208872 206768  98%    0.05K   2901       72     11604K buffer_head
  4960   4051  81%    0.68K    992        5      3968K nfs_inode_cache
  9996   6478  64%    0.27K    714       14      2856K radix_tree_node
   566    530  93%    2.00K    283        2      1132K size-2048
  4680   4014  85%    0.12K    156       30       624K dentry_cache
  1430   1409  98%    0.39K    143       10       572K inode_cache
    65     65 100%    8.00K     65        1       520K size-8192
   258    256  99%    1.32K     86        3       344K raid5/md0
  5460   5435  99%    0.04K     65       84       260K sysfs_dir_cache
   945    882  93%    0.25K     63       15       252K size-256
   456    438  96%    0.50K     57        8       228K size-512
  2891   2514  86%    0.06K     49       59       196K size-64
   675    474  70%    0.25K     45       15       180K skbuff_head_cache
   176    165  93%    1.00K     44        4       176K size-1024
  1320    744  56%    0.12K     44       30       176K bio
    42     37  88%    4.00K     42        1       168K size-4096
  1170   1073  91%    0.12K     39       30       156K size-128
   351    316  90%    0.40K     39        9       156K proc_inode_cache
  4068   3298  81%    0.03K     36      113       144K size-32
   660    160  24%    0.19K     33       20       132K filp
    84     64  76%    1.30K     28        3       112K task_struct
  1232    441  35%    0.09K     28       44       112K vm_area_struct
    69     63  91%    1.31K     23        3        92K sighand_cache
    20     20 100%    4.00K     20        1        80K pgd
   396    244  61%    0.17K     18       22        72K blkdev_requests
     1      1 100%   64.00K      1        1        64K size-65536
   160    153  95%    0.38K     16       10        64K skbuff_fclone_cache
    32     32 100%    2.00K     16        2        64K sgpool-128
    15      3  20%    4.00K     15        1        60K names_cache
  1196   1138  95%    0.04K     13       92        52K acpi_operand
  2639    640  24%    0.02K     13      203        52K biovec-1
    12     11  91%    3.00K      6        2        48K biovec-(256)
   180    162  90%    0.25K     12       15        48K sgpool-16
    96     82  85%    0.50K     12        8        48K sgpool-32
   110     64  58%    0.38K     11       10        44K signal_cache
    40     37  92%    0.97K     10        4        40K blkdev_queue
    32     32 100%    1.00K      8        4        32K sgpool-64
    80     80 100%    0.38K      8       10        32K scsi_cmd_cache
   413    263  63%    0.06K      7       59        28K as_arq
   180    150  83%    0.12K      6       30        24K kmem_cache
    54     20  37%    0.44K      6        9        24K mm_struct
    15     15 100%    1.50K      3        5        24K biovec-128
   180    180 100%    0.12K      6       30        24K sgpool-8
    42     36  85%    0.50K      6        7        24K nfs_write_data
   505     64  12%    0.04K      5      101        20K pid
    50     21  42%    0.38K      5       10        20K files_cache
   145    112  77%    0.13K      5       29        20K idr_layer_cache
    35     26  74%    0.56K      5        7        20K bdev_cache
    35     23  65%    0.50K      5        7        20K sock_inode_cache
    25     23  92%    0.75K      5        5        20K biovec-64
    35     32  91%    0.50K      5        7        20K nfs_read_data

Cheers,
Janos

#26 lore raw
In reply to Haar János
Quoted 8 lines · Haar János

I seriously gets this, and dont know why.
This server have 2GB ram, and ~1.1G always free!
Anybody have an idea?

Thanks,
Janos

Quoted 8 lines
 May 21 09:05:35 st-0003 kernel: swapper: page allocation failure. order:0,
 mode:0x20
 May 21 09:05:35 st-0003 kernel:  <c013c6ac> __alloc_pages+0x274/0x286
 <c014af1d> cache_alloc_refill+0x2a6/0x45c
 May 21 09:05:35 st-0003 kernel:  <c014b12b> __kmalloc+0x58/0x61   <c03d5bfc>
 __alloc_skb+0x49/0xf5
 May 21 09:05:35 st-0003 kernel:  <f88336b1>
 e1000_alloc_rx_buffers+0x5c/0x2e5 [e1000]   <f88315de>

e1000 gobbled up all your lowmem memory from interrupt context.

Increasing /proc/sys/vm/min_free_kbytes will help.

#27 lore raw

----- Original Message -----
From: "Andrew Morton" <[email protected]>
To: "Haar János" <[email protected]>
Cc: <[email protected]>
Sent: Sunday, May 21, 2006 2:03 PM
Subject: Re: swapper: page allocation failure.

Quoted 10 lines
Quoted 9 lines · Haar János

I seriously gets this, and dont know why.
This server have 2GB ram, and ~1.1G always free!
Anybody have an idea?

Thanks,
Janos

 May 21 09:05:35 st-0003 kernel: swapper: page allocation failure.

order:0,

Quoted 4 lines
Quoted 4 lines

mode:0x20

 May 21 09:05:35 st-0003 kernel:  <c013c6ac> __alloc_pages+0x274/0x286
 <c014af1d> cache_alloc_refill+0x2a6/0x45c
 May 21 09:05:35 st-0003 kernel:  <c014b12b> __kmalloc+0x58/0x61

<c03d5bfc>

Quoted 7 lines
Quoted 3 lines
 __alloc_skb+0x49/0xf5
 May 21 09:05:35 st-0003 kernel:  <f88336b1>
 e1000_alloc_rx_buffers+0x5c/0x2e5 [e1000]   <f88315de>

e1000 gobbled up all your lowmem memory from interrupt context.

Increasing /proc/sys/vm/min_free_kbytes will help.

Yes, i know. ;-)

Thanks,

Quoted 5 lines

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/