Skip to contentSkip to conversation

weird memory related problems, negative memory usage or fake memory usage?

DISCUSSION Olivier Sessink 5 participants 7 messages Sep 24, 2001 – Sep 27, 2001

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi all,

after upgrade from 2.4.10pre8 to 2.4.10 I have weird problems, Xfree sometimes shows up with 99.9% memory in top (on a box with 512 mb), and in ps axl it has 4294989036 in the RSS column. When this happens the box starts to kill some processes, starts heavily swapping (top reports > 400MB in the cache, but the machine is heavily swapping!!!) and is completely unusable.

The problems is triggered when I start edonkey, some filesharing program that creates checksums of large files (a couple of > 700Mb files).

Since this makes the machine completely unusable, and since it is not happening on 2.4.10pre8 I guess it is a bug ;-)

This is a top snapshot:

46 processes: 43 sleeping, 3 running, 0 zombie, 0 stopped

CPU states:   3.8% user,   1.2% system,  94.6% nice,   0.4% idle
Mem:    514032K total,   511436K used,     2596K free,     1244K buffers
Swap:   358808K total,    10928K used,   347880K free,   466072K cached
  PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
  253 olivier   19  18   300  196   124 R N  52.9  0.0 119:50 dnetc
 1274 root      14   0  5940 5940   872 R    33.7  1.1   0:00 xsetbg
 1262 root       5 -10 50764  -1M  1320 S <   2.7 99.9   0:01 XFree86
 1263 root       9   0  1528 1492  1204 S     2.7  0.2   0:00 xdm
 1267 root      11   0   980  976   776 R     2.7  0.1   0:00 top
 1270 root       9   0  1016 1012   820 S     0.9  0.1   0:00 Xsetup_0
    1 root       8   0   104   56    36 S     0.0  0.0   0:04 init
    2 root       9   0     0    0     0 SW    0.0  0.0   0:00 keventd
    3 root       9   0     0    0     0 SW    0.0  0.0   0:00 kapm-idled
    4 root      19  19     0    0     0 SWN   0.0  0.0   0:00 ksoftirqd_CPU0
    5 root       9   0     0    0     0 SW    0.0  0.0   0:01 kswapd
    6 root       9   0     0    0     0 SW    0.0  0.0   0:00 bdflush
    7 root       9   0     0    0     0 SW    0.0  0.0   0:00 kupdated
    8 root       9   0     0    0     0 SW    0.0  0.0   0:00 kreiserfsd

the ps axl for X then is

100 0 1262 1260 5 -10 52408 4294965424 select S< ? 0:01
/usr/X11R6/bin/X vt7 -dpi 100 -nolisten tcp -auth /var/lib/xdm/authdir

regards,
Olivier

#1 lore raw

[Andrea, please read this bugreport ...]

Quoted 7 lines · Olivier Sessink

after upgrade from 2.4.10pre8 to 2.4.10 I have weird problems, Xfree sometimes shows up with 99.9% memory in top (on a box with 512 mb), and in ps axl it has 4294989036 in the RSS column. When this happens the box starts to kill some processes, starts heavily swapping (top reports > 400MB in the cache, but the machine is heavily swapping!!!) and is completely unusable.

Quoted 5 lines

Since this makes the machine completely unusable, and since it is not happening on 2.4.10pre8 I guess it is a bug ;-)

  PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
 1262 root       5 -10 50764  -1M  1320 S <   2.7 99.9   0:01 XFree86

It seems Andrea wasn't careful with the merge and
backed out some of the locking wrt mm->rss.

Andrea, you may want to spend some time auditing
your VM like has been done with the other 2.4 VM.

cheers,

Rik
--
IA64: a worthy successor to the i860.

		http://www.surriel.com/

http://www.conectiva.com/ http://distro.conectiva.com/

#2 lore raw
In reply to Olivier Sessink
Quoted 14 lines · Rik van Riel

[Andrea, please read this bugreport ...]

Quoted 9 lines · Olivier Sessink

after upgrade from 2.4.10pre8 to 2.4.10 I have weird problems, Xfree sometimes shows up with 99.9% memory in top (on a box with 512 mb), and in ps axl it has 4294989036 in the RSS column. When this happens the box starts to kill some processes, starts heavily swapping (top reports > 400MB in the cache, but the machine is heavily swapping!!!) and is completely unusable.

Since this makes the machine completely unusable, and since it is not happening on 2.4.10pre8 I guess it is a bug ;-)

Quoted 6 lines
Quoted 2 lines
  PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
 1262 root       5 -10 50764  -1M  1320 S <   2.7 99.9   0:01 XFree86

It seems Andrea wasn't careful with the merge and
backed out some of the locking wrt mm->rss.

thanks for forwarding this report, actually I just noticed this here and that's good so I can reproduce :)

it is possible it is my mistake, but I don't think so, infact I don't recall to have changed rss stuff or locking around it. Incidentally the first time I reproduced it here was after the tlb shootdown patch from Ben was introduced, never reproduced it here previously with only my changes. However it is possibly just a coincidence.

Quoted 4 lines

Andrea, you may want to spend some time auditing
your VM like has been done with the other 2.4 VM.

Quoted 10 lines

cheers,

Rik
--
IA64: a worthy successor to the i860.

		http://www.surriel.com/

http://www.conectiva.com/ http://distro.conectiva.com/

Andrea

#3 lore raw
Quoted 6 lines · Andrea Arcangeli
Quoted 5 lines · Rik van Riel
Quoted 2 lines · Olivier Sessink
  PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
 1262 root       5 -10 50764  -1M  1320 S <   2.7 99.9   0:01 XFree86
Quoted 8 lines
Quoted 2 lines

It seems Andrea wasn't careful with the merge and
backed out some of the locking wrt mm->rss.

thanks for forwarding this report, actually I just noticed this here and that's good so I can reproduce :)

it is possible it is my mistake, but I don't think so, infact I don't recall to have changed rss stuff or locking around it.

Mmm, then it could also be one of the bugs which got fixed in -ac but where Linus never reacted to the patch, IIRC the RSS thing was indeed fixed around the time where Linus was in the habbit of silently dropping half of the patches sent to him...

regards,

Rik
--
IA64: a worthy successor to the i860.

		http://www.surriel.com/

http://www.conectiva.com/ http://distro.conectiva.com/

#4 lore raw
In reply to Andrea Arcangeli
Quoted 10 lines · Rik van Riel
Quoted 9 lines · Andrea Arcangeli
Quoted 7 lines · Rik van Riel
Quoted 2 lines · Olivier Sessink
  PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
 1262 root       5 -10 50764  -1M  1320 S <   2.7 99.9   0:01 XFree86

It seems Andrea wasn't careful with the merge and
backed out some of the locking wrt mm->rss.

Quoted 8 lines
Quoted 5 lines

thanks for forwarding this report, actually I just noticed this here and that's good so I can reproduce :)

it is possible it is my mistake, but I don't think so, infact I don't recall to have changed rss stuff or locking around it.

Mmm, then it could also be one of the bugs which got fixed in -ac but where Linus never reacted to the

possibly yes but maybe not, dunno right now or I would be just sending the fix inline in this email :). As said I never seen it before Ben's tlb shootdown was merged into mainline, but again I repeat it can _really_ be just an unlucky coincidence. But I guess because of this coincidence the tlb shootdown will be the first things I will audit tomorrow.

Andrea

#5 lore raw
In reply to Andrea Arcangeli
Quoted 6 lines · Andrea Arcangeli

possibly yes but maybe not, dunno right now or I would be just sending the fix inline in this email :). As said I never seen it before Ben's tlb shootdown was merged into mainline, but again I repeat it can _really_ be just an unlucky coincidence. But I guess because of this coincidence the tlb shootdown will be the first things I will audit tomorrow.

This should fix it.

-ben

... v2.4.10-rss.diff ...

mm/memory.c +3 -1
diff -urN v2.4.10/mm/memory.c foo/mm/memory.c
@@ -319,7 +319,9 @@
 		if (pte_none(pte))
 			continue;
 		if (pte_present(pte)) {
-			freed ++;
+			struct page *page = pte_page(pte);
+			if (!PageReserved(page) && VALID_PAGE(page))
+				freed ++;
 			/* This will eventually call __free_pte on the pte. */
 			tlb_remove_page(tlb, ptep, address + offset);
 		} else {
#6 lore raw

Review

Quoted 7 lines · Benjamin LaHaise

This should fix it.

(quoted diff) +3 -1
-			freed ++;
+			struct page *page = pte_page(pte);
+			if (!PageReserved(page) && VALID_PAGE(page))
+				freed ++;

NO! I expect that fixes the rss, but crashes my system:
must test VALID_PAGE(page) _before_ !PageReserved(page).

Hugh

mm/memory.c +3 -1
@@ -319,7 +319,9 @@
 		if (pte_none(pte))
 			continue;
 		if (pte_present(pte)) {
-			freed ++;
+			struct page *page = pte_page(pte);
+			if (VALID_PAGE(page) && !PageReserved(page))
+				freed ++;
 			/* This will eventually call __free_pte on the pte. */
 			tlb_remove_page(tlb, ptep, address + offset);
 		} else {