vs-8345: get_mem_for_virtual_node: kmalloc failed.

This machine is has a 400MHz PII, Adaptec AIC-7895, 631 MB of memory, and about 512 Meg of swap).

When compiling the kernel (2.2.19) I changed __FD_SETSIZE in /usr/include/bits/types.h from 1024 to 8192. (Increased number of file descriptors)

The following is executed at boot time:

# More file descriptors
echo 8192 > /proc/sys/fs/file-max

# Inodes 4x the file-max
echo 32768 > /proc/sys/fs/inode-max
# More user ports
echo 1024 32768 > /proc/sys/net/ipv4/ip_local_port_range

echo 768 > /proc/sys/net/ipv4/tcp_max_syn_backlog

echo 120 > /proc/sys/net/ipv4/tcp_fin_timeout

And I started to see this recenty :

Apr 19 11:54:24 netcache2 kernel: eth0: Memory squeeze, deferring packet.
Apr 19 11:54:24 netcache2 kernel: eth0: Memory squeeze, deferring packet.
Apr 19 11:54:33 netcache2 last message repeated 1113 times
Apr 19 11:54:33 netcache2 kernel: mem_for_virtual_node: kmalloc failed.  There were 272 allocations
Apr 19 11:54:33 netcache2 last message repeated 1113 times
Apr 19 11:54:33 netcache2 kernel: mem_for_virtual_node: kmalloc failed.  There were 272 allocations
Apr 19 11:54:33 netcache2 kernel: vs-8345: get_mem_for_virtual_node: kmalloc failed.  There were 272 allocations
Apr 19 11:54:33 netcache2 kernel: vs-8345: get_mem_for_virtual_node: kmalloc failed.  There were 272 allocations

...
...
This goes on until ...

Apr 19 11:56:45 netcache2 kernel: vs-8345: get_mem_for_virtual_node: kmalloc failed.  There were 272 allocations
Apr 19 11:56:45 netcache2 last message repeated 198 times
Apr 19 11:56:53 netcache2 squid[8377]: Squid Parent: child process 8439 exited due to signal 6
Apr 19 11:56:45 netcache2 last message repeated 198 times
Apr 19 11:56:53 netcache2 squid[8377]: Squid Parent: child process 8439 exited due to signal 6
Apr 19 11:56:56 netcache2 squid[8377]: Squid Parent: child process 13195 started
Apr 19 11:56:56 netcache2 squid[8377]: Squid Parent: child process 13195 started

Is there anything I can do to prevent this from happening ?

This (squid) server, serves about 50 requests/sec, so 2 minutes of downtime means a lot of unhappy "customers".

 Ríkharður Egilsson - Networking/Security EXD/ITN/CCO
 OECD/OCDE - Organisation for Economic Co-operation and Development

The atomic allocations for incoming packets probably overwhelm the memory system. Try to increase the values (*2,*3) in /proc/sys/vm/freepages, that'll keep more memory free for interrupts and flush more aggressively. It may also help a bit to change the rx_copybreak argument of the network driver.

-Andi