New: swapper: page allocation failure. order:0, mode:0x20
Subsystems (2)
A message that is not in the archive (sent off-list or not yet received): [email protected]/
(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).
Quoted 37 lines · On Wed, 17 Jun 2009 18:41:24 GMT [email protected] wrote:
http://bugzilla.kernel.org/show_bug.cgi?id=13561
Summary: swapper: page allocation failure. order:0, mode:0x20
Product: Memory Management...
[ 1884.639134] swapper: page allocation failure. order:0, mode:0x20 [ 1884.639136] Pid: 0, comm: swapper Tainted: P 2.6.29.4-1.0 #1 [ 1884.639137] Call Trace: [ 1884.639139] <IRQ> [<ffffffff8028828b>] __alloc_pages_internal+0x41e/0x43f [ 1884.639146] [<ffffffff802ac183>] alloc_pages_current+0xb9/0xc2 [ 1884.639149] [<ffffffff802b08eb>] new_slab+0xcf/0x28c [ 1884.639151] [<ffffffff802b0d14>] __slab_alloc+0x200/0x3e2 [ 1884.639154] [<ffffffff803d9e3b>] ? __alloc_skb+0x42/0x131 [ 1884.639157] [<ffffffff804974cc>] ? _spin_lock_irqsave+0x28/0x31 [ 1884.639160] [<ffffffff803d9e3b>] ? __alloc_skb+0x42/0x131 [ 1884.639162] [<ffffffff802b12cf>] kmem_cache_alloc_node+0x7c/0xc6 [ 1884.639165] [<ffffffff8024b8a4>] ? __mod_timer+0xb3/0xc5 [ 1884.639168] [<ffffffff803d9e3b>] __alloc_skb+0x42/0x131 [ 1884.639171] [<ffffffff8041892e>] tcp_send_ack+0x2b/0x112 [ 1884.639173] [<ffffffff80415c53>] __tcp_ack_snd_check+0x65/0x7d [ 1884.639176] [<ffffffff80416906>] tcp_rcv_established+0x7d7/0x926 [ 1884.639179] [<ffffffff8041dfb3>] tcp_v4_do_rcv+0x1b1/0x35e [ 1884.639181] [<ffffffff8041e606>] tcp_v4_rcv+0x4a6/0x785 [ 1884.639185] [<ffffffff80402a2d>] ip_local_deliver_finish+0x177/0x25a [ 1884.639187] [<ffffffff80402b82>] ip_local_deliver+0x72/0x7a [ 1884.639190] [<ffffffff804025e3>] ip_rcv_finish+0x32b/0x345 [ 1884.639192] [<ffffffff80402879>] ip_rcv+0x27c/0x2b9 [ 1884.639195] [<ffffffff803e05b0>] netif_receive_skb+0x471/0x496 [ 1884.639201] [<ffffffffa0ca07c0>] rtl8169_rx_interrupt+0x362/0x43d [r8169] [ 1884.639205] [<ffffffffa0ca38b3>] rtl8169_poll+0x3f/0x1fe [r8169] [ 1884.639208] [<ffffffff803de82c>] net_rx_action+0xae/0x19c [ 1884.639212] [<ffffffff8024745d>] __do_softirq+0x8a/0x139 [ 1884.639214] [<ffffffff8021259c>] call_softirq+0x1c/0x30 [ 1884.639217] [<ffffffff80213d78>] do_softirq+0x44/0x8f [ 1884.639219] [<ffffffff80247153>] irq_exit+0x3f/0x7e
Quoted 8 lines
[ 1884.639222] [<ffffffff80213ff3>] do_IRQ+0xc3/0xe4 [ 1884.639224] [<ffffffff80211d13>] ret_from_intr+0x0/0x29 [ 1884.639225] <EOI> [<ffffffff8022704e>] ? native_safe_halt+0x6/0x8 [ 1884.639230] [<ffffffff802180c2>] ? default_idle+0x2e/0x4b [ 1884.639233] [<ffffffff8021830c>] ? c1e_idle+0x109/0x110 [ 1884.639235] [<ffffffff802590c9>] ? atomic_notifier_call_chain+0x13/0x15 [ 1884.639239] [<ffffffff8021022e>] ? cpu_idle+0x59/0x9a [ 1884.639242] [<ffffffff804925d6>] ? start_secondary+0x254/0x25b
yep, that's OK. The kernel was excessively low on memory and the network driver was unable to allocate a page for a received packet. The packet will just be dropped and everything should recover. If you get a lot of these warnings (one per minute?) then increasing the value in /proc/sys/vm/min_free_kbytes should help.
Dave, we get quite a few reports of this nature, especially from e1000 (grr). Do you think we could/should suppress the warning, by sprinkling a few __GFP_NOWARNs in the right places? It doesn't seem like it's being very useful?
Quoted 53 lines
Quoted 37 lines · On Wed, 17 Jun 2009 18:41:24 GMT [email protected] wrote:
http://bugzilla.kernel.org/show_bug.cgi?id=13561
Summary: swapper: page allocation failure. order:0, mode:0x20
Product: Memory Management...
[ 1884.639134] swapper: page allocation failure. order:0, mode:0x20 [ 1884.639136] Pid: 0, comm: swapper Tainted: P 2.6.29.4-1.0 #1 [ 1884.639137] Call Trace: [ 1884.639139] <IRQ> [<ffffffff8028828b>] __alloc_pages_internal+0x41e/0x43f [ 1884.639146] [<ffffffff802ac183>] alloc_pages_current+0xb9/0xc2 [ 1884.639149] [<ffffffff802b08eb>] new_slab+0xcf/0x28c [ 1884.639151] [<ffffffff802b0d14>] __slab_alloc+0x200/0x3e2 [ 1884.639154] [<ffffffff803d9e3b>] ? __alloc_skb+0x42/0x131 [ 1884.639157] [<ffffffff804974cc>] ? _spin_lock_irqsave+0x28/0x31 [ 1884.639160] [<ffffffff803d9e3b>] ? __alloc_skb+0x42/0x131 [ 1884.639162] [<ffffffff802b12cf>] kmem_cache_alloc_node+0x7c/0xc6 [ 1884.639165] [<ffffffff8024b8a4>] ? __mod_timer+0xb3/0xc5 [ 1884.639168] [<ffffffff803d9e3b>] __alloc_skb+0x42/0x131 [ 1884.639171] [<ffffffff8041892e>] tcp_send_ack+0x2b/0x112 [ 1884.639173] [<ffffffff80415c53>] __tcp_ack_snd_check+0x65/0x7d [ 1884.639176] [<ffffffff80416906>] tcp_rcv_established+0x7d7/0x926 [ 1884.639179] [<ffffffff8041dfb3>] tcp_v4_do_rcv+0x1b1/0x35e [ 1884.639181] [<ffffffff8041e606>] tcp_v4_rcv+0x4a6/0x785 [ 1884.639185] [<ffffffff80402a2d>] ip_local_deliver_finish+0x177/0x25a [ 1884.639187] [<ffffffff80402b82>] ip_local_deliver+0x72/0x7a [ 1884.639190] [<ffffffff804025e3>] ip_rcv_finish+0x32b/0x345 [ 1884.639192] [<ffffffff80402879>] ip_rcv+0x27c/0x2b9 [ 1884.639195] [<ffffffff803e05b0>] netif_receive_skb+0x471/0x496 [ 1884.639201] [<ffffffffa0ca07c0>] rtl8169_rx_interrupt+0x362/0x43d [r8169] [ 1884.639205] [<ffffffffa0ca38b3>] rtl8169_poll+0x3f/0x1fe [r8169] [ 1884.639208] [<ffffffff803de82c>] net_rx_action+0xae/0x19c [ 1884.639212] [<ffffffff8024745d>] __do_softirq+0x8a/0x139 [ 1884.639214] [<ffffffff8021259c>] call_softirq+0x1c/0x30 [ 1884.639217] [<ffffffff80213d78>] do_softirq+0x44/0x8f [ 1884.639219] [<ffffffff80247153>] irq_exit+0x3f/0x7eQuoted 9 lines
[ 1884.639222] [<ffffffff80213ff3>] do_IRQ+0xc3/0xe4 [ 1884.639224] [<ffffffff80211d13>] ret_from_intr+0x0/0x29 [ 1884.639225] <EOI> [<ffffffff8022704e>] ? native_safe_halt+0x6/0x8 [ 1884.639230] [<ffffffff802180c2>] ? default_idle+0x2e/0x4b [ 1884.639233] [<ffffffff8021830c>] ? c1e_idle+0x109/0x110 [ 1884.639235] [<ffffffff802590c9>] ? atomic_notifier_call_chain+0x13/0x15 [ 1884.639239] [<ffffffff8021022e>] ? cpu_idle+0x59/0x9a [ 1884.639242] [<ffffffff804925d6>] ? start_secondary+0x254/0x25b
Quoted 3 lines
yep, that's OK. The kernel was excessively low on memory and the network driver was unable to allocate a page for a received packet.
Even though the box had 4Gbyte RAM with nothing running but console? maybe I"m mistaking the amount of RAM for the amount of memory in a buffer used for network IO?
-N
Quoted 12 lines
The packet will just be dropped and everything should recover. If you get a lot of these warnings (one per minute?) then increasing the value in
/proc/sys/vm/min_free_kbytesshould help.Dave, we get quite a few reports of this nature, especially from e1000 (grr). Do you think we could/should suppress the warning, by sprinkling a few
__GFP_NOWARNsin the right places? It doesn't seem like it's being very useful?
Quoted 5 lines · On Wed, 17 Jun 2009 19:29:25 +0000 Nigel Kukard <[email protected]> wrote:
Quoted 3 lines
yep, that's OK. The kernel was excessively low on memory and the network driver was unable to allocate a page for a received packet.
Even though the box had 4Gbyte RAM with nothing running but console?
yep.
Quoted 2 lines
maybe I"m mistaking the amount of RAM for the amount of memory in a buffer used for network IO?
[ 1884.639264] Node 0 DMA free:11692kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:15284kB pages_scanned:0 all_unreclaimable? yes
[ 1884.639271] Node 0 DMA32 free:5412kB min:6560kB low:8200kB high:9840kB active_anon:60476kB inactive_anon:12224kB active_file:28720kB inactive_file:2987624kB unevictable:0kB present:3333216kB pages_scanned:0 all_unreclaimable? no
[ 1884.639279] Node 0 Normal free:444kB min:1524kB low:1904kB high:2284kB active_anon:12436kB inactive_anon:13984kB active_file:16356kB inactive_file:666568kB unevictable:480kB present:775680kB pages_scanned:0 all_unreclaimable? no
Pretty much all your memory was used by pagecache. The DMA32 zone is below the minimum threshold. The VM has decided that there's no memory left for networking. It has retained a little bit of free memory so the block device drivers can still perform writeout, to free up additional memory.
What is _supposed_ to happen (and usually does) it that kswapd will see that the machine is getting short on memory and will work to free some up, thus making it available to networking receive. But sometimes kswapd can't keep up.
Quoted 18 lines
Quoted 5 lines
maybe I"m mistaking the amount of RAM for the amount of memory in a buffer used for network IO?
[ 1884.639264] Node 0 DMA free:11692kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:15284kB pages_scanned:0 all_unreclaimable? yes[ 1884.639271] Node 0 DMA32 free:5412kB min:6560kB low:8200kB high:9840kB active_anon:60476kB inactive_anon:12224kB active_file:28720kB inactive_file:2987624kB unevictable:0kB present:3333216kB pages_scanned:0 all_unreclaimable? no[ 1884.639279] Node 0 Normal free:444kB min:1524kB low:1904kB high:2284kB active_anon:12436kB inactive_anon:13984kB active_file:16356kB inactive_file:666568kB unevictable:480kB present:775680kB pages_scanned:0 all_unreclaimable? noPretty much all your memory was used by pagecache. The DMA32 zone is below the minimum threshold. The VM has decided that there's no memory left for networking. It has retained a little bit of free memory so the block device drivers can still perform writeout, to free up additional memory.
Quoted 5 lines
What is _supposed_ to happen (and usually does) it that kswapd will see that the machine is getting short on memory and will work to free some up, thus making it available to networking receive. But sometimes kswapd can't keep up.
Understood .... I did copy about 2.5Tb of files from 1 SATA drive to another and was busy trasferring 50Gb to the box from Gbit ethernet.
Cool, thanks again guys. Just thought I'd report to make sure its something trivial and not a serious issue. :)
-N