negative anon_rss?
Sorry for not sending this to linux-mm,
but I can't subscribe at present? Anyway...
I wrote the following script to try and accurately determine how much RAM a particular program uses:
http://www.pixelbeat.org/scripts/ps_mem.py
A user reported an issue on debian with kernel 2.6.8-2-386 where many processes were being reported as using a negative amount of memory.
I asked him to run the following:
(
echo total rss shared trs - drs -
for pid in pidof apache2; do
cat /proc/$pid/statm
done
) | column -t
the output of which was:
total rss shared trs - drs - 6580 2306 5273 95 0 6485 0 6580 2313 5273 95 0 6485 0 6119 1717 5269 95 0 6024 0 6630 2371 5273 95 0 6535 0 6735 2503 5273 95 0 6640 0 6773 2546 5273 95 0 6678 0 5845 1146 5198 95 0 5750 0
Notice the large values for the shared column.
Also notice that the shared column is larger than rss!? I had assumed that shared was a subset of rss from
the following (pseudo) code from fs/proc/task_mmu.c::task_statm()
*total = mm->total_em
*shared = get_mm_counter(mm, file_rss)
*rss = *shared + get_mm_counter(mm, anon_rss) *trs = mm->end_code - mm->start_code *drs = mm->total_vm - mm->shared_vm
Therefore anon_rss must be negative in the kernel? So there is a mismatch between pages being marked as anonymous and anon_rss being updated appropriately?
cheers,
Pádraig.
Quoted 38 lines · Pádraig Brady
Sorry for not sending this to linux-mm,
but I can't subscribe at present? Anyway...I wrote the following script to try and accurately determine how much RAM a particular program uses:
http://www.pixelbeat.org/scripts/ps_mem.pyA user reported an issue on debian with kernel 2.6.8-2-386 where many processes were being reported as using a negative amount of memory.
I asked him to run the following:
(
echo total rss shared trs - drs -
for pid inpidof apache2; do
cat/proc/$pid/statm
done
) | column -tthe output of which was:
total rss shared trs - drs - 6580 2306 5273 95 0 6485 0 6580 2313 5273 95 0 6485 0 6119 1717 5269 95 0 6024 0 6630 2371 5273 95 0 6535 0 6735 2503 5273 95 0 6640 0 6773 2546 5273 95 0 6678 0 5845 1146 5198 95 0 5750 0Notice the large values for the shared column.
Also notice that the shared column is larger than rss!? I had assumed that shared was a subset of rss from
the following (pseudo) code from fs/proc/task_mmu.c::task_statm()*total =
mm->total_em
*shared =get_mm_counter(mm, file_rss)
Quoted 7 lines
*rss = *shared + get_mm_counter(mm, anon_rss) *trs = mm->end_code - mm->start_code *drs = mm->total_vm - mm->shared_vmTherefore
anon_rssmust be negative in the kernel? So there is a mismatch between pages being marked as anonymous andanon_rssbeing updated appropriately?
You're looking at recent source to explain what happened in 2.6.8:
not a good strategy. 2.6.8 didn't even record file_rss and anon_rss separately. We split off anon_rss in 2.6.10, and changed the meaning of "shared" then too - here's my ChangeLog comment justifying it.
From: Hugh Dickins <[email protected]>
Date: Thu, 28 Oct 2004 01:17:23 +0000 (-0700)
Subject: [PATCH] statm: shared = rss - anon_rss
X-Git-Tag: v2.6.10~147^2~103
X-Git-Url: http://127.0.0.1:1234/?p=.git;a=commitdiff_plain;h=8586febd93ed02668b9d0ad87963610425e44977;hp=da63671a5c506729abb8fe383704c264967c53c0
[PATCH] statm: shared = rss - anon_rss
The third "shared" field of /proc/$pid/statm in 2.4 was a count of pages in the mm whose page_count is more than 1 (oddly, including pages shared just with swapcache). That's too costly to calculate each time, so 2.6 changed it to the total file-backed extent. But Andrea knows apps and users surprised when (rss - shared) goes negative: we need to provide an rss-like statistic, close to the 2.4 interpretation.
Something that's quick and easy to maintain accurately is mm->anon_rss, the count of anonymous pages in the mm. Then shared = rss - anon_rss gives a pretty good and meaningful approximation to 2.4's intention: wli confirms that this will be useful to Oracle too.
Where to show it? I think it's best to treat this as a bugfix and show it in the third field of /proc/$pid/statm, after resident, as before - there's no evidence that the total file-backed extent was found useful.
Albert would like other fields to revert to page counts, but that's a lot
harder: if mprotect can change the category of a page, then it can't be accounted as simply as this. Only go that route if real need shown.
Hugh