Skip to contentSkip to conversation

2.6.9 - e1000 - page allocation failed

DISCUSSION Lukas Hejtmanek 4 participants 8 messages Oct 21, 2004 – Dec 3, 2004

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hello,

I'm getting this message in kernel log:

swapper: page allocation failure. order:0, mode:0x20
[__alloc_pages+441/862] __alloc_pages+0x1b9/0x35e [__get_free_pages+37/63] __get_free_pages+0x25/0x3f [kmem_getpages+33/201] kmem_getpages+0x21/0xc9 [cache_grow+171/333] cache_grow+0xab/0x14d [cache_alloc_refill+372/537] cache_alloc_refill+0x174/0x219 [__kmalloc+133/140] __kmalloc+0x85/0x8c [alloc_skb+71/224] alloc_skb+0x47/0xe0 [e1000_alloc_rx_buffers+68/227] e1000_alloc_rx_buffers+0x44/0xe3 [e1000_clean_rx_irq+398/1095] e1000_clean_rx_irq+0x18e/0x447 [__kfree_skb+131/253] __kfree_skb+0x83/0xfd [e1000_clean+81/202] e1000_clean+0x51/0xca [net_rx_action+119/246] net_rx_action+0x77/0xf6 [__do_softirq+183/198] __do_softirq+0xb7/0xc6 [do_softirq+45/47] do_softirq+0x2d/0x2f [do_IRQ+274/304] do_IRQ+0x112/0x130 [common_interrupt+24/32] common_interrupt+0x18/0x20 [default_idle+0/44] default_idle+0x0/0x2c [default_idle+41/44] default_idle+0x29/0x2c [cpu_idle+63/88] cpu_idle+0x3f/0x58 [start_kernel+361/388] start_kernel+0x169/0x184 [unknown_bootoption+0/348] unknown_bootoption+0x0/0x15c

That machine has 300MB of free memory of 1GB total and 4GB of free swap. So what is wrong?

Signature · 1 line
Lukáš Hejtmánek
#1 lore raw

Lukas Hejtmanek <[email protected]> :
[page allocation failure with e1000]

If you are using TSO, try patch below by Herbert Xu (available from http://marc.theaimsgroup.com/?l=linux-netdev&m=109799935603132&w=3)

net/ipv4/tcp_output.c +5 -1
@@ -455,8 +455,12 @@
 {
 	struct tcp_opt *tp = tcp_sk(sk);
 	struct sk_buff *buff;
-	int nsize = skb->len - len;
+	int nsize;
 	u16 flags;
+
+	nsize = skb_headlen(skb) - len;
+	if (nsize < 0)
+		nsize = 0;
 
 	if (skb_cloned(skb) &&
 	    skb_is_nonlinear(skb) &&
#2 lore raw
Quoted 13 lines · Francois Romieu

Lukas Hejtmanek <[email protected]> :
[page allocation failure with e1000]

If you are using TSO, try patch below by Herbert Xu (available from http://marc.theaimsgroup.com/?l=linux-netdev&m=109799935603132&w=3)

net/ipv4/tcp_output.c +0 -0
@@ -455,8 +455,12 @@
 {
 	struct tcp_opt *tp = tcp_sk(sk);
 	struct sk_buff *buff;

net/ipv4/tcp_output.c line 466

Quoted 10 lines
net/ipv4/tcp_output.c +5 -1
-	int nsize = skb->len - len;
+	int nsize;
 	u16 flags;
+
+	nsize = skb_headlen(skb) - len;
+	if (nsize < 0)
+		nsize = 0;
 
 	if (skb_cloned(skb) &&
 	    skb_is_nonlinear(skb) &&

I'd be interested in knowing if this fixes it - I don't expect it will, because that's a zero-order allocation failure. He's really out of memory.

The e1000 driver has a default rx ring size of 256 which seems a bit nutty:
a back-to-back GFP_ATOMIC allocation of 256 skbs could easily exhaust the page allocator pools.

Probably this machine needs to increase /proc/sys/vm/min_free_kbytes.

#3 lore raw
Quoted 8 lines · Andrew Morton

I'd be interested in knowing if this fixes it - I don't expect it will, because that's a zero-order allocation failure. He's really out of memory.

The e1000 driver has a default rx ring size of 256 which seems a bit nutty:
a back-to-back GFP_ATOMIC allocation of 256 skbs could easily exhaust the page allocator pools.

Probably this machine needs to increase /proc/sys/vm/min_free_kbytes.

It did not help.

However I tweak network stack a little bit:
/sbin/sysctl -w net/core/rmem_max=8388608 /sbin/sysctl -w net/core/wmem_max=8388608 /sbin/sysctl -w net/core/rmem_default=1048576 /sbin/sysctl -w net/core/wmem_default=1048576 /sbin/sysctl -w net/ipv4/tcp_window_scaling=1 /sbin/sysctl -w net/ipv4/tcp_rmem="4096 1048576 8388608" /sbin/sysctl -w net/ipv4/tcp_wmem="4096 1048576 8388608" /sbin/ifconfig eth0 txqueuelen 1000

this is /proc/meminfo:
cat /proc/meminfo

MemTotal:      1035116 kB
MemFree:        447028 kB
Buffers:             0 kB
Cached:         522444 kB
SwapCached:          0 kB
Active:          98092 kB
Inactive:       460408 kB
HighTotal:      131008 kB
HighFree:          308 kB
LowTotal:       904108 kB
LowFree:        446720 kB
SwapTotal:     4008208 kB
SwapFree:      4008204 kB
Dirty:               0 kB
Writeback:           0 kB
Mapped:          45804 kB
Slab:            21112 kB
Committed_AS:   128148 kB
PageTables:       1528 kB
VmallocTotal:   114680 kB
VmallocUsed:      2964 kB
VmallocChunk:   111700 kB

Note that there is 400MB of _free_ memory.

This is from slabinfo (hope that it is relevant info..)

size-131072(DMA)       0      0 131072    1   32 : tunables
8    4    0 : slabdata      0      0      0
size-131072            0      0 131072    1   32 : tunables
8    4    0 : slabdata      0      0      0
size-65536(DMA)        0      0  65536    1   16 : tunables
8    4    0 : slabdata      0      0      0
size-65536             1      1  65536    1   16 : tunables
8    4    0 : slabdata      1      1      0
size-32768(DMA)        0      0  32768    1    8 : tunables
8    4    0 : slabdata      0      0      0
size-32768            16     16  32768    1    8 : tunables
8    4    0 : slabdata     16     16      0
size-16384(DMA)        0      0  16384    1    4 : tunables
8    4    0 : slabdata      0      0      0
size-16384             2      2  16384    1    4 : tunables
8    4    0 : slabdata      2      2      0
size-8192(DMA)         0      0   8192    1    2 : tunables
8    4    0 : slabdata      0      0      0
size-8192            120    120   8192    1    2 : tunables
8    4    0 : slabdata    120    120      0
size-4096(DMA)         0      0   4096    1    1 : tunables   24   12

8 : slabdata 0 0 0
size-4096 300 300 4096 1 1 : tunables 24 12
8 : slabdata 300 300 0
size-2048(DMA) 0 0 2048 2 1 : tunables 24 12
8 : slabdata 0 0 0
size-2048 106 106 2048 2 1 : tunables 24 12
8 : slabdata 53 53 0
size-1024(DMA) 0 0 1024 4 1 : tunables 54 27
8 : slabdata 0 0 0
size-1024 1052 1052 1024 4 1 : tunables 54 27
8 : slabdata 263 263 0
size-512(DMA) 0 0 512 8 1 : tunables 54 27
8 : slabdata 0 0 0
size-512 294 1240 512 8 1 : tunables 54 27
8 : slabdata 155 155 0
size-256(DMA) 0 0 256 15 1 : tunables 120 60
8 : slabdata 0 0 0
size-256 211 1170 256 15 1 : tunables 120 60
8 : slabdata 78 78 0
size-128(DMA) 0 0 128 31 1 : tunables 120 60
8 : slabdata 0 0 0
size-128 1688 2728 128 31 1 : tunables 120 60
8 : slabdata 88 88 0
size-64(DMA) 0 0 64 61 1 : tunables 120 60
8 : slabdata 0 0 0
size-64 610 2318 64 61 1 : tunables 120 60
8 : slabdata 38 38 0
size-32(DMA) 0 0 32 119 1 : tunables 120 60
8 : slabdata 0 0 0
size-32 1381 4641 32 119 1 : tunables 120 60
8 : slabdata 39 39 15
kmem_cache 165 165 256 15 1 : tunables 120 60
8 : slabdata 11 11 0

Signature · 1 line
Lukáš Hejtmánek
#4 lore raw
Quoted 5 lines · Lukas Hejtmanek
Quoted 3 lines · Andrew Morton

I'd be interested in knowing if this fixes it - I don't expect it will, because that's a zero-order allocation failure. He's really out of memory.

Quoted 9 lines
Quoted 5 lines

The e1000 driver has a default rx ring size of 256 which seems a bit nutty:
a back-to-back GFP_ATOMIC allocation of 256 skbs could easily exhaust the page allocator pools.

Probably this machine needs to increase /proc/sys/vm/min_free_kbytes.

It did not help.

What did you increase it to? What was the allocation failure message?

#5 lore raw
Quoted 10 lines · Nick Piggin
Quoted 8 lines
Quoted 4 lines

a back-to-back GFP_ATOMIC allocation of 256 skbs could easily exhaust the page allocator pools.

Probably this machine needs to increase /proc/sys/vm/min_free_kbytes.

It did not help.

What did you increase it to? What was the allocation failure message?

Sorry for late answer I missed this mail. I increased it to 10MB and failuer message was the same...

Signature · 1 line
Lukáš Hejtmánek
#6 lore raw
In reply to Andrew Morton
Quoted 8 lines · Andrew Morton

I'd be interested in knowing if this fixes it - I don't expect it will, because that's a zero-order allocation failure. He's really out of memory.

The e1000 driver has a default rx ring size of 256 which seems a bit nutty:
a back-to-back GFP_ATOMIC allocation of 256 skbs could easily exhaust the page allocator pools.

Probably this machine needs to increase /proc/sys/vm/min_free_kbytes.

Well is there some possiblility to solve it? I have lots of free memory (300MB) and it still complains about not enough free memory.

I'm using this:
/sbin/sysctl -w net/core/rmem_max=8388608 /sbin/sysctl -w net/core/wmem_max=8388608 /sbin/sysctl -w net/core/rmem_default=1048576 /sbin/sysctl -w net/core/wmem_default=1048576 /sbin/sysctl -w net/ipv4/tcp_window_scaling=1 /sbin/sysctl -w net/ipv4/tcp_rmem="4096 1048576 8388608" /sbin/sysctl -w net/ipv4/tcp_wmem="4096 1048576 8388608" /sbin/ifconfig eth0 txqueuelen 1000

Signature · 1 line
Lukáš Hejtmánek
#7 lore raw
In reply to Andrew Morton
Quoted 8 lines · Andrew Morton

I'd be interested in knowing if this fixes it - I don't expect it will, because that's a zero-order allocation failure. He's really out of memory.

The e1000 driver has a default rx ring size of 256 which seems a bit nutty:
a back-to-back GFP_ATOMIC allocation of 256 skbs could easily exhaust the page allocator pools.

Probably this machine needs to increase /proc/sys/vm/min_free_kbytes.

However. I have two machines very similar and one opses and one does not. I wondered why. I've found out that if I turn whole netfilter to module and disable:
CONFIG_NET_IPGRE=y
CONFIG_IP_MROUTE=y
CONFIG_IP_PIMSM_V1=y
CONFIG_IP_PIMSM_V2=y

Then problem seems to disappeare on both...

(I do not use iptables on any machine)

Signature · 1 line
Lukáš Hejtmánek