kernel freeze

After I reported a kernel freeze in this mail list("2.4.18 freeze on
4G memory."),
I try 2.4.19 & 2.4.19+rmap14a patch, all of those have been frozen. I try to enable kernel profiling to detect what is the problem and found

the function _text_lock_vmscan make the cpu busy. Any comment? The result below:
Normal state:

# uptime
12:20pm  up  2:00,  6 users,  load average: 8.18, 10.13, 8.28
# vmstat 2 2
   procs                      memory    swap          io     system
cpu
 r  b  w   swpd   free   buff  cache  si  so    bi    bo   in    cs  us
sy  id
 1  0  4      0 182988 408272 1919636   0   0    89   164  853   499  11
9  79
 0  1  0      0 184288 408516 1919840   0   0   332    12 4869  2587  20
30  50
# readprofile | sort -g
   ...
    36 _text_lock_vmscan                          0.2727
   ...
  4666 _text_lock_read_write                     95.2245
  5027 _text_lock_namei                           2.8465
  5113 ip_packet_match                           13.8940
  6124 _text_lock_inode                          12.6529
  6342 prune_icache                              11.3250
  7549 shrink_cache                               8.1347
 10010 __constant_memcpy                         36.8015
 11099 _text_lock_swap                          194.7193
1242957 default_idle                             15536.9625
1414706 total                                      1.0008
 after freeze:
#uptime
12:44pm  up  2:25,  6 users,  load average: 292.05, 528.62, 117.95
#vmstat 2 2
   procs                      memory    swap          io     system
cpu
 r  b  w   swpd   free   buff  cache  si  so    bi    bo   in    cs  us
sy  id
 9 207  4      0 565776 337108 1948476   0   0    77   144  751   433
10  22  68
 3 239  1      0 548152 337312 1949624   0   0   718  3666 2256  2789
34  38  27
# readprofile | sort -g
  ...
21999 _text_lock_swap                          385.9474
 84239 prune_icache                             150.4268
101555 _text_lock_inode                         209.8244
122285 shrink_cache                             131.7726
173019 _text_lock_vmscan                        1310.7500
1311668 default_idle                             16395.8500
1992753 total                                      1.4098

I'll lay you a large bet that lowmem is full of garbage. Probably buffer_heads, inodes or PTEs. Output of /proc/meminfo and /proc/slabinfo as you approach oblivion would be useful. As would a description of the workload that triggers it.

M.

I think it's the reason. But I have to decreased memory from 4G to 2G
because of the complain from my customers, so I can't give you the /proc/meminfo or /proc/slabinfo now. :(
Why the linux-vm can't manage lowmem correctly? I have seen some
articles talking about the LRU pre zone patch, but why 2.4.19+rmap14a patch also has this problem?

The patches don't seem to have wended their way back to mainline yet, probably because people have been concentrating on 2.5 recently. I think Andrea's -aa kernel has fixes for most, if not all of these problems.

rmap won't help you in this instance, as the stuff you're filling ZONE_NORMAL with is unswappable. In fact, it may make it worse, due the overhead of pte_chains ... but this will probably only hurt you if you have many large tasks sharing memory.

Changing PAGE_OFFSET from 3Gb to 2Gb may be a better workaround than decreasing from 4Gb to 2Gb of RAM, if you don't have any huge single process.

M.