Skip to contentSkip to conversation

negative anon_rss?

DISCUSSION Pádraig Brady 2 participants 2 messages Nov 15, 2006

lore
Share & tools
lore
#0 lore raw

Sorry for not sending this to linux-mm,
but I can't subscribe at present? Anyway...

I wrote the following script to try and accurately determine how much RAM a particular program uses:
http://www.pixelbeat.org/scripts/ps_mem.py

A user reported an issue on debian with kernel 2.6.8-2-386 where many processes were being reported as using a negative amount of memory.

I asked him to run the following:

(
echo total rss shared trs - drs -
for pid in pidof apache2; do
cat /proc/$pid/statm
done
) | column -t

the output of which was:

  total  rss   shared  trs  -  drs   -
  6580   2306  5273    95   0  6485  0
  6580   2313  5273    95   0  6485  0
  6119   1717  5269    95   0  6024  0
  6630   2371  5273    95   0  6535  0
  6735   2503  5273    95   0  6640  0
  6773   2546  5273    95   0  6678  0
  5845   1146  5198    95   0  5750  0

Notice the large values for the shared column.
Also notice that the shared column is larger than rss!? I had assumed that shared was a subset of rss from
the following (pseudo) code from fs/proc/task_mmu.c::task_statm()

*total = mm->total_em
*shared = get_mm_counter(mm, file_rss)

  *rss    = *shared + get_mm_counter(mm, anon_rss)
  *trs    = mm->end_code - mm->start_code
  *drs    = mm->total_vm - mm->shared_vm

Therefore anon_rss must be negative in the kernel? So there is a mismatch between pages being marked as anonymous and anon_rss being updated appropriately?

cheers,
Pádraig.

#1 lore raw
Quoted 38 lines · Pádraig Brady

Sorry for not sending this to linux-mm,
but I can't subscribe at present? Anyway...

I wrote the following script to try and accurately determine how much RAM a particular program uses:
http://www.pixelbeat.org/scripts/ps_mem.py

A user reported an issue on debian with kernel 2.6.8-2-386 where many processes were being reported as using a negative amount of memory.

I asked him to run the following:

(
echo total rss shared trs - drs -
for pid in pidof apache2; do
cat /proc/$pid/statm
done
) | column -t

the output of which was:

  total  rss   shared  trs  -  drs   -
  6580   2306  5273    95   0  6485  0
  6580   2313  5273    95   0  6485  0
  6119   1717  5269    95   0  6024  0
  6630   2371  5273    95   0  6535  0
  6735   2503  5273    95   0  6640  0
  6773   2546  5273    95   0  6678  0
  5845   1146  5198    95   0  5750  0

Notice the large values for the shared column.
Also notice that the shared column is larger than rss!? I had assumed that shared was a subset of rss from
the following (pseudo) code from fs/proc/task_mmu.c::task_statm()

*total = mm->total_em
*shared = get_mm_counter(mm, file_rss)

Quoted 7 lines
  *rss    = *shared + get_mm_counter(mm, anon_rss)
  *trs    = mm->end_code - mm->start_code
  *drs    = mm->total_vm - mm->shared_vm

Therefore anon_rss must be negative in the kernel? So there is a mismatch between pages being marked as anonymous and anon_rss being updated appropriately?

You're looking at recent source to explain what happened in 2.6.8:
not a good strategy. 2.6.8 didn't even record file_rss and anon_rss separately. We split off anon_rss in 2.6.10, and changed the meaning of "shared" then too - here's my ChangeLog comment justifying it.

From: Hugh Dickins <[email protected]>
Date: Thu, 28 Oct 2004 01:17:23 +0000 (-0700)
Subject: [PATCH] statm: shared = rss - anon_rss
X-Git-Tag: v2.6.10~147^2~103
X-Git-Url: http://127.0.0.1:1234/?p=.git;a=commitdiff_plain;h=8586febd93ed02668b9d0ad87963610425e44977;hp=da63671a5c506729abb8fe383704c264967c53c0

[PATCH] statm: shared = rss - anon_rss

The third "shared" field of /proc/$pid/statm in 2.4 was a count of pages in the mm whose page_count is more than 1 (oddly, including pages shared just with swapcache). That's too costly to calculate each time, so 2.6 changed it to the total file-backed extent. But Andrea knows apps and users surprised when (rss - shared) goes negative: we need to provide an rss-like statistic, close to the 2.4 interpretation.

Something that's quick and easy to maintain accurately is mm->anon_rss, the count of anonymous pages in the mm. Then shared = rss - anon_rss gives a pretty good and meaningful approximation to 2.4's intention: wli confirms that this will be useful to Oracle too.

Where to show it? I think it's best to treat this as a bugfix and show it in the third field of /proc/$pid/statm, after resident, as before - there's no evidence that the total file-backed extent was found useful.

Albert would like other fields to revert to page counts, but that's a lot
harder: if mprotect can change the category of a page, then it can't be accounted as simply as this. Only go that route if real need shown.

Hugh