Skip to contentSkip to conversation

2.6.10-rc1-bk page allocation failure. order:2, mode:0x20

DISCUSSION Brad Campbell 2 participants 4 messages Nov 2, 2004 1 message not in the archive

lore
Share & tools
lore
#0 lore raw

G'day all,

I'm still getting quite a lot of these come up in the logs when the system is under mild load.
I suspect it might have something to do with running an MTU of 9000 on the main ethernet port which is directly feeding a workstation with an NFS root (and thus gets quite a high load at times)

It's not so much an issue but it does cause the workstation to stall for up to a second while it waits for data every time it occurs.

The loaded ethernet port is this one on an PCI card

0000:00:0d.0 Ethernet controller: Marvell Technology Group Ltd. Yukon Gigabit Ethernet 10/100/1000Base-T Adapter (rev 12)

This started rearing its ugly head when I moved from 2.6.5 to 2.6.9-preX and persists with BK as of about 2 days ago.

Regards,
Brad

srv:/home/brad# uname -a
Linux srv 2.6.10-rc1 #2 Sun Oct 31 10:56:30 GST 2004 i686 GNU/Linux
srv:/home/brad# free
             total       used       free     shared    buffers     cached
Mem:        514676     251088     263588          0      13280     156328
-/+ buffers/cache:      81480     433196
Swap:       987988        284     987704
srv:/home/brad# lspci
0000:00:00.0 Host bridge: VIA Technologies, Inc. VT8377 [KT400/KT600 AGP] Host Bridge (rev 80)
0000:00:01.0 PCI bridge: VIA Technologies, Inc. VT8237 PCI Bridge
0000:00:09.0 Ethernet controller: 3Com Corporation 3c940 10/100/1000Base-T [Marvell] (rev 12)
0000:00:0b.0 Unknown mass storage controller: Promise Technology, Inc. PDC20318 (SATA150 TX4) (rev 02)
0000:00:0c.0 FireWire (IEEE 1394): VIA Technologies, Inc. IEEE 1394 Host Controller (rev 46)
0000:00:0d.0 Ethernet controller: Marvell Technology Group Ltd. Yukon Gigabit Ethernet 10/100/1000Base-T Adapter (rev 12)
0000:00:0e.0 Unknown mass storage controller: Promise Technology, Inc. PDC20318 (SATA150 TX4) (rev 02)
0000:00:0f.0 RAID bus controller: VIA Technologies, Inc. VIA VT6420 SATA RAID Controller (rev 80)
0000:00:0f.1 IDE interface: VIA Technologies, Inc. VT82C586A/B/VT82C686/A/B/VT823x/A/C PIPC Bus Master IDE (rev 06)
0000:00:10.0 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81)
0000:00:10.1 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81)
0000:00:10.2 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81)
0000:00:10.3 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81)
0000:00:10.4 USB Controller: VIA Technologies, Inc. USB 2.0 (rev 86)
0000:00:11.0 ISA bridge: VIA Technologies, Inc. VT8237 ISA bridge [K8T800 South]
0000:00:13.0 Unknown mass storage controller: Promise Technology, Inc. PDC20318 (SATA150 TX4) (rev 02)
0000:01:00.0 VGA compatible controller: Silicon Integrated Systems [SiS] 315PRO PCI/AGP VGA Display Adapter
srv:/home/brad# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0E:A6:41:45:94
          inet addr:192.168.3.82  Bcast:192.168.3.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
          Interrupt:18 Memory:de800000-0

eth1 Link encap:Ethernet HWaddr 00:04:E2:8E:1E:AD

          inet addr:192.168.2.82  Bcast:192.168.2.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1
          RX packets:5207091 errors:430 dropped:430 overruns:0 frame:0
          TX packets:4805338 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:322428903 (307.4 MiB)  TX bytes:2838601552 (2.6 GiB)
          Interrupt:16 Memory:dc800000-0

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:138 errors:0 dropped:0 overruns:0 frame:0 TX packets:138 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0
RX bytes:9378 (9.1 KiB) TX bytes:9378 (9.1 KiB)

srv:/home/brad# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/hda1              19G  9.4G  8.1G  54% /
tmpfs                 252M     0  252M   0% /dev/shm
/dev/hda3             165G   38G  119G  24% /raid0
/dev/md0              2.1T  2.1T   12G 100% /raid
/dev/md2              459G  315G  145G  69% /raid2

swapper: page allocation failure. order:2, mode:0x20

 [<c0138a98>] __alloc_pages+0x1c8/0x390
 [<c0138c7f>] __get_free_pages+0x1f/0x40
 [<c013c18d>] kmem_getpages+0x1d/0xb0
 [<c013ce26>] cache_grow+0xb6/0x170
 [<c013d03e>] cache_alloc_refill+0x15e/0x220
 [<c013d400>] __kmalloc+0x80/0xa0
 [<c02e61e7>] alloc_skb+0x47/0xe0
 [<e08bc871>] FillRxDescriptor+0x31/0xc0 [sk98lin]
 [<e08bc824>] FillRxRing+0x54/0x70 [sk98lin]
 [<e08bb89a>] SkGeIsrOnePort+0x17a/0x190 [sk98lin]
 [<c0132b24>] handle_IRQ_event+0x34/0x70
 [<c0132c42>] __do_IRQ+0xe2/0x160
 [<c0106666>] do_IRQ+0x26/0x40
 [<c0104aa8>] common_interrupt+0x18/0x20
 [<c0101fb0>] default_idle+0x0/0x30
 [<c0101fd3>] default_idle+0x23/0x30
 [<c010204a>] cpu_idle+0x3a/0x60
 [<c047e7c2>] start_kernel+0x142/0x160
 [<c047e3b0>] unknown_bootoption+0x0/0x1b0
syslogd: page allocation failure. order:2, mode:0x20
 [<c0138a98>] __alloc_pages+0x1c8/0x390
 [<c0138c7f>] __get_free_pages+0x1f/0x40
 [<c013c18d>] kmem_getpages+0x1d/0xb0
 [<c013ce26>] cache_grow+0xb6/0x170
 [<c013d03e>] cache_alloc_refill+0x15e/0x220
 [<c013d400>] __kmalloc+0x80/0xa0
 [<c02e61e7>] alloc_skb+0x47/0xe0
 [<e08bc871>] FillRxDescriptor+0x31/0xc0 [sk98lin]
 [<e08bc824>] FillRxRing+0x54/0x70 [sk98lin]
 [<e08bb89a>] SkGeIsrOnePort+0x17a/0x190 [sk98lin]
 [<c0132b24>] handle_IRQ_event+0x34/0x70
 [<c0132c42>] __do_IRQ+0xe2/0x160
 [<c0106666>] do_IRQ+0x26/0x40
 [<c0104aa8>] common_interrupt+0x18/0x20
syslogd: page allocation failure. order:2, mode:0x20
 [<c0138a98>] __alloc_pages+0x1c8/0x390
 [<c0104e15>] show_trace+0x35/0x90
 [<c0138c7f>] __get_free_pages+0x1f/0x40
 [<c013c18d>] kmem_getpages+0x1d/0xb0
 [<c013ce26>] cache_grow+0xb6/0x170
 [<c013d03e>] cache_alloc_refill+0x15e/0x220
 [<c0138c7f>] __get_free_pages+0x1f/0x40
 [<c013d400>] __kmalloc+0x80/0xa0
 [<c02e61e7>] alloc_skb+0x47/0xe0
 [<e08bc871>] FillRxDescriptor+0x31/0xc0 [sk98lin]
 [<e08bc824>] FillRxRing+0x54/0x70 [sk98lin]
 [<e08c0584>] SkDrvEvent+0xa44/0xaa0 [sk98lin]
 [<e08cd953>] SkGeSirqIsr+0x1d3/0x890 [sk98lin]
 [<e08d06e4>] SkEventDispatcher+0xc4/0x160 [sk98lin]
 [<e08bb7cc>] SkGeIsrOnePort+0xac/0x190 [sk98lin]
 [<c0132b24>] handle_IRQ_event+0x34/0x70
 [<c0132c42>] __do_IRQ+0xe2/0x160
 [<c0106666>] do_IRQ+0x26/0x40
 [<c0104aa8>] common_interrupt+0x18/0x20
#1 lore raw
Quoted 5 lines · Brad Campbell

G'day all,

I'm still getting quite a lot of these come up in the logs when the system is under mild load.
I suspect it might have something to do with running an MTU of 9000 on the main ethernet port which is directly feeding a workstation with an NFS root (and thus gets quite a high load at times)

Quoted 8 lines

It's not so much an issue but it does cause the workstation to stall for up to a second while it waits for data every time it occurs.

The loaded ethernet port is this one on an PCI card

0000:00:0d.0 Ethernet controller: Marvell Technology Group Ltd. Yukon Gigabit Ethernet 10/100/1000Base-T Adapter (rev 12)

This started rearing its ugly head when I moved from 2.6.5 to 2.6.9-preX and persists with BK as of about 2 days ago.

There are patches in the newest -mm kernels that should help the problem. If you're willing to test them, the feedback would be welcome.

Thanks
Nick

#2 lore raw
Quoted 11 lines · Nick Piggin
Quoted 9 lines · Brad Campbell

G'day all,

I'm still getting quite a lot of these come up in the logs when the system is under mild load.
I suspect it might have something to do with running an MTU of 9000 on the main ethernet port which is directly feeding a workstation with an NFS root (and thus gets quite a high load at times)

It's not so much an issue but it does cause the workstation to stall for up to a second while it waits for data every time it occurs.

The loaded ethernet port is this one on an PCI card

Quoted 8 lines
Quoted 4 lines

0000:00:0d.0 Ethernet controller: Marvell Technology Group Ltd. Yukon Gigabit Ethernet 10/100/1000Base-T Adapter (rev 12)

This started rearing its ugly head when I moved from 2.6.5 to 2.6.9-preX and persists with BK as of about 2 days ago.

There are patches in the newest -mm kernels that should help the problem. If you're willing to test them, the feedback would be welcome.

Always willing to test specific patches. Can I just grab the broken out patches, or pull some specific csets from a bk tree? I'm not particularly keen on running an -mm kernel on this box if I can avoid it (It's a server in 24hr use with 2.5TB of data where the backup media is 7,000km away).

Regards,
Brad

A message that is not in the archive (sent off-list or not yet received): [email protected]

#4 lore raw
Quoted 12 lines · Nick Piggin
Quoted 2 lines

Always willing to test specific patches. Can I just grab the broken out patches, or pull some specific csets from a bk tree? I'm not particularly keen on running an -mm kernel on this box if I can avoid it (It's a server in 24hr use with 2.5TB of data where the backup media is 7,000km away).

OK fair enough.

Here is a rollup of the 3 patches that are supposed to help with the problem. It is diffed against 2.6.10-rc1-bk8, which you probably wouldn't want to run either.

Not sure how cleanly it will apply onto 2.6.9... shouldn't be too bad I think.

I'm actually running a reasonably recent BK pull of 2.6.10-rc1 as of a couple of days ago, but I did some pretty severe testing and evaluation with my raid disks removed and replaced with spares before I let it loose on the real media. I have applied those patches and I'll beat on it for a few hours and see how it goes. I have some pretty defined cron jobs that make it easy to reproduce.

Regards,
Brad