Getting total memory used by processes

(note: using 2.4.22)

I patched the kernel with rmap and i2c, now I'm seeing weirdness where top reports 500-900 megs used by processes((mem used + swap used) - cached) but I know that can't be right. So it seems that memory is magically being used and I don't know who is using it.

I'd like to get the total memory used by processes using a better method. So I created a perl script which does a ps listing, pulls out VSZ, sums that and spits out the result. Here it is in all its glory:

open(PSOUT, "/bin/ps -auxwwt | /bin/awk '{print \$5}' |") || die "Error opening file";

my $size = 0;

while(<PSOUT>)

{
    chomp;
    $size += $_;
}

When I run this I get:

# perl getmem.pl
Total size = 2748144

yet I don't have 2.7 gigs in physical memory + swap:

# cat /proc/meminfo
        total:    used:    free:  shared: buffers:  cached:
Mem:  1588834304 1562914816 25919488        0 11993088 1055232000
Swap: 1077501952 538742784 538759168
MemTotal:      1551596 kB
MemFree:         25312 kB
MemShared:           0 kB
Buffers:         11712 kB
Cached:         952588 kB
SwapCached:      77912 kB
Active:         813644 kB
Inactive:       582436 kB
HighTotal:      655360 kB
HighFree:         2044 kB
LowTotal:       896236 kB
LowFree:         23268 kB
SwapTotal:     1052248 kB
SwapFree:       526132 kB

(btw, notice the ridiculous 972 megs cached? that's another issue.)

I'm sure linux doesn't have redundant copies of program executables and shared libraries for each process. Many processes are invoked using the same executable. If ps reports the program/shared library memory within VSZ then this might explain the large number I'm getting.

Could I parse /proc/<pid>/maps, going through each process and finding only those sections which aren't shared libraries/executables? maps looks like this:

08048000-080f3000 r-xp 00000000 03:04 293805 /usr/bin/perl
080f3000-080fb000 rw-p 000aa000 03:04 293805 /usr/bin/perl
080fb000-0d37e000 rwxp 00000000 00:00 0
40000000-40013000 r-xp 00000000 03:04 81603 /lib/ld-2.2.5.so
40013000-40014000 rw-p 00013000 03:04 81603 /lib/ld-2.2.5.so
40014000-40015000 rw-p 00000000 00:00 0
40015000-40017000 r-xp 00000000 03:04 539158 /usr/lib/perl5/site_perl/5.6.1/i386-linux/auto/Time/HiRes/HiRes.so 40017000-40018000 rw-p 00001000 03:04 539158 /usr/lib/perl5/site_perl/5.6.1/i386-linux/auto/Time/HiRes/HiRes.so
...

So I could sum up the sections where inode=0? Or should I look at the permissions and not count shared? If it's a copy on write page then I don't know how many bytes are used by the parent process.

Anyone know if this is the right track/should I do something else/maybe there's another tool to help me? thanks for any information.

Thanks in advance,

Brett

This isn't -rmap. I would like to help you debug your problem, but it would be nice if you could at least record your debugging info running the kernel you want to have debugged ;)

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan

Sorry, I tried to be sneaky but you caught me! :D I should have said that I'm seeing the same thing with both the -rmap and stock kernel, the data I collected was from a stock kernel.

Here's the data from an 2.4.22 -rmap kernel:

# getmem.pl
Total size = 4967032

# cat /proc/meminfo
        total:    used:    free:  shared: buffers:  cached:
Mem:  1056083968 1047187456  8896512        0 12722176 592035840
Swap: 1077501952 88113152 989388800
MemTotal:      1031332 kB
MemFree:          8688 kB
MemShared:           0 kB
Buffers:         12424 kB
Cached:         519428 kB
SwapCached:      58732 kB
Active:         696796 kB
ActiveAnon:     335180 kB
ActiveCache:    361616 kB
Inact_dirty:        76 kB
Inact_laundry:  147524 kB
Inact_clean:     22068 kB
Inact_target:   173292 kB
HighTotal:      131072 kB
HighFree:         1112 kB
LowTotal:       900260 kB
LowFree:          7576 kB
SwapTotal:     1052248 kB
SwapFree:       966200 kB

maps file:

# cat /proc/21234/maps
08048000-080f3000 r-xp 00000000 03:04 293805     /usr/bin/perl
080f3000-080fb000 rw-p 000aa000 03:04 293805     /usr/bin/perl
080fb000-0d423000 rwxp 00000000 00:00 0
40000000-40013000 r-xp 00000000 03:04 81603      /lib/ld-2.2.5.so
40013000-40014000 rw-p 00013000 03:04 81603      /lib/ld-2.2.5.so
40015000-40017000 r-xp 00000000 03:04 539158 /usr/lib/perl5/site_perl/5.6.1/i386-linux/auto/Time/HiRes/HiRes.so
40017000-40018000 rw-p 00001000 03:04 539158 /usr/lib/perl5/site_perl/5.6.1/
...