weird memory related problems, negative memory usage or fake memory usage?
Subsystems (2)
Hi all,
after upgrade from 2.4.10pre8 to 2.4.10 I have weird problems, Xfree sometimes shows up with 99.9% memory in top (on a box with 512 mb), and in ps axl it has 4294989036 in the RSS column. When this happens the box starts to kill some processes, starts heavily swapping (top reports > 400MB in the cache, but the machine is heavily swapping!!!) and is completely unusable.
The problems is triggered when I start edonkey, some filesharing program that creates checksums of large files (a couple of > 700Mb files).
Since this makes the machine completely unusable, and since it is not happening on 2.4.10pre8 I guess it is a bug ;-)
This is a top snapshot:
46 processes: 43 sleeping, 3 running, 0 zombie, 0 stopped
CPU states: 3.8% user, 1.2% system, 94.6% nice, 0.4% idle Mem: 514032K total, 511436K used, 2596K free, 1244K buffers Swap: 358808K total, 10928K used, 347880K free, 466072K cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
253 olivier 19 18 300 196 124 R N 52.9 0.0 119:50 dnetc
1274 root 14 0 5940 5940 872 R 33.7 1.1 0:00 xsetbg
1262 root 5 -10 50764 -1M 1320 S < 2.7 99.9 0:01 XFree86
1263 root 9 0 1528 1492 1204 S 2.7 0.2 0:00 xdm
1267 root 11 0 980 976 776 R 2.7 0.1 0:00 top
1270 root 9 0 1016 1012 820 S 0.9 0.1 0:00 Xsetup_0
1 root 8 0 104 56 36 S 0.0 0.0 0:04 init
2 root 9 0 0 0 0 SW 0.0 0.0 0:00 keventd
3 root 9 0 0 0 0 SW 0.0 0.0 0:00 kapm-idled
4 root 19 19 0 0 0 SWN 0.0 0.0 0:00 ksoftirqd_CPU0
5 root 9 0 0 0 0 SW 0.0 0.0 0:01 kswapd
6 root 9 0 0 0 0 SW 0.0 0.0 0:00 bdflush
7 root 9 0 0 0 0 SW 0.0 0.0 0:00 kupdated
8 root 9 0 0 0 0 SW 0.0 0.0 0:00 kreiserfsdthe ps axl for X then is
100 0 1262 1260 5 -10 52408 4294965424 select S< ? 0:01/usr/X11R6/bin/X vt7 -dpi 100 -nolisten tcp -auth /var/lib/xdm/authdir
regards,
Olivier
[Andrea, please read this bugreport ...]
Quoted 7 lines · Olivier Sessink
after upgrade from 2.4.10pre8 to 2.4.10 I have weird problems, Xfree sometimes shows up with 99.9% memory in top (on a box with 512 mb), and in ps axl it has 4294989036 in the RSS column. When this happens the box starts to kill some processes, starts heavily swapping (top reports > 400MB in the cache, but the machine is heavily swapping!!!) and is completely unusable.
Quoted 5 lines
Since this makes the machine completely unusable, and since it is not happening on 2.4.10pre8 I guess it is a bug ;-)
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 1262 root 5 -10 50764 -1M 1320 S < 2.7 99.9 0:01 XFree86
It seems Andrea wasn't careful with the merge and
backed out some of the locking wrt mm->rss.
Andrea, you may want to spend some time auditing
your VM like has been done with the other 2.4 VM.
cheers,
Rik
--
IA64: a worthy successor to the i860.
http://www.surriel.com/
Quoted 14 lines · Rik van Riel
[Andrea, please read this bugreport ...]
Quoted 9 lines · Olivier Sessink
after upgrade from 2.4.10pre8 to 2.4.10 I have weird problems, Xfree sometimes shows up with 99.9% memory in top (on a box with 512 mb), and in ps axl it has 4294989036 in the RSS column. When this happens the box starts to kill some processes, starts heavily swapping (top reports > 400MB in the cache, but the machine is heavily swapping!!!) and is completely unusable.
Since this makes the machine completely unusable, and since it is not happening on 2.4.10pre8 I guess it is a bug ;-)
Quoted 6 lines
Quoted 2 lines
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 1262 root 5 -10 50764 -1M 1320 S < 2.7 99.9 0:01 XFree86It seems Andrea wasn't careful with the merge and
backed out some of the locking wrtmm->rss.
thanks for forwarding this report, actually I just noticed this here and that's good so I can reproduce :)
it is possible it is my mistake, but I don't think so, infact I don't recall to have changed rss stuff or locking around it. Incidentally the first time I reproduced it here was after the tlb shootdown patch from Ben was introduced, never reproduced it here previously with only my changes. However it is possibly just a coincidence.
Quoted 4 lines
Andrea, you may want to spend some time auditing
your VM like has been done with the other 2.4 VM.
Quoted 10 lines
cheers,
Rik
--
IA64: a worthy successor to the i860.http://www.surriel.com/
Andrea
Quoted 6 lines · Andrea Arcangeli
Quoted 5 lines · Rik van Riel
Quoted 2 lines · Olivier Sessink
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 1262 root 5 -10 50764 -1M 1320 S < 2.7 99.9 0:01 XFree86
Quoted 8 lines
Quoted 2 lines
It seems Andrea wasn't careful with the merge and
backed out some of the locking wrtmm->rss.thanks for forwarding this report, actually I just noticed this here and that's good so I can reproduce :)
it is possible it is my mistake, but I don't think so, infact I don't recall to have changed rss stuff or locking around it.
Mmm, then it could also be one of the bugs which got fixed in -ac but where Linus never reacted to the patch, IIRC the RSS thing was indeed fixed around the time where Linus was in the habbit of silently dropping half of the patches sent to him...
regards,
Rik
--
IA64: a worthy successor to the i860.
http://www.surriel.com/
Quoted 10 lines · Rik van Riel
Quoted 9 lines · Andrea Arcangeli
Quoted 7 lines · Rik van Riel
Quoted 2 lines · Olivier Sessink
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 1262 root 5 -10 50764 -1M 1320 S < 2.7 99.9 0:01 XFree86It seems Andrea wasn't careful with the merge and
backed out some of the locking wrtmm->rss.
Quoted 8 lines
Quoted 5 lines
thanks for forwarding this report, actually I just noticed this here and that's good so I can reproduce :)
it is possible it is my mistake, but I don't think so, infact I don't recall to have changed rss stuff or locking around it.
Mmm, then it could also be one of the bugs which got fixed in -ac but where Linus never reacted to the
possibly yes but maybe not, dunno right now or I would be just sending the fix inline in this email :). As said I never seen it before Ben's tlb shootdown was merged into mainline, but again I repeat it can _really_ be just an unlucky coincidence. But I guess because of this coincidence the tlb shootdown will be the first things I will audit tomorrow.
Andrea
Quoted 6 lines · Andrea Arcangeli
possibly yes but maybe not, dunno right now or I would be just sending the fix inline in this email :). As said I never seen it before Ben's tlb shootdown was merged into mainline, but again I repeat it can _really_ be just an unlucky coincidence. But I guess because of this coincidence the tlb shootdown will be the first things I will audit tomorrow.
This should fix it.
-ben
... v2.4.10-rss.diff ...
mm/memory.c +3 -1
@@ -319,7 +319,9 @@if (pte_none (pte))continue ;if (pte_present (pte)) { - freed ++; +struct page *page =pte_page (pte); +if (!PageReserved (page) &&VALID_PAGE (page)) + freed ++;/* This will eventually call __free_pte on the pte. */ tlb_remove_page (tlb, ptep, address + offset); }else {
Review
Quoted 7 lines · Benjamin LaHaise
This should fix it.
(quoted diff) +3
-1- freed ++; + struct page *page = pte_page(pte); + if (!PageReserved(page) && VALID_PAGE(page)) + freed ++;
NO! I expect that fixes the rss, but crashes my system:
must test VALID_PAGE(page) _before_ !PageReserved(page).
Hugh
mm/memory.c +3 -1
@@ -319,7 +319,9 @@if (pte_none (pte))continue ;if (pte_present (pte)) { - freed ++; +struct page *page =pte_page (pte); +if (VALID_PAGE (page) && !PageReserved (page)) + freed ++;/* This will eventually call __free_pte on the pte. */ tlb_remove_page (tlb, ptep, address + offset); }else {