Odd swap behavior

DISCUSSION Rajagopal Ananthanarayanan 2 participants 9 messages Oct 4, 2000 – Oct 5, 2000 Memory management

Share & tools

Rajagopal Ananthanarayanan

author

2000-10-04 00:23 UTC #0 lore raw

I'm running fairly stressful tests like
dbench with lots of clients. Since the new
VM changes (now in test9), I haven't noticed _any_ swap activity, in spite of the enormous memory pressures. I have lots of processes in the system, like 8 httpd's, 4 getty's, etc. most of which should be "idle" ... Why aren't the
pages (eg. mapped stacks) from these processes being swapped out?

--------------------------------------------------------------------------
Rajagopal Ananthanarayanan ("ananth")
Member Technical Staff, SGI.
--------------------------------------------------------------------------
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Rik van Riel

2000-10-04 15:14 UTC #1 lore raw

Quoted 7 lines · Rajagopal Ananthanarayanan

I'm running fairly stressful tests like dbench with lots of clients. Since the new VM changes (now in test9), I haven't noticed _any_ swap activity, in spite of the enormous memory pressures. I have lots of processes in the system, like 8 httpd's, 4 getty's, etc. most of which should be "idle" ... Why aren't the pages (eg. mapped stacks) from these processes being swapped out?

That's an interesting one. Most "complaints" I've had about test9 is that it swaps more than previous versions ;)

But let me give you the answer...

Small code changes in deactivate_page() have caused the drop_behind() code to actually WORK AS ADVERTISED right now, and because of that streaming IO doesn't put any memory pressure on the system.

regards,

Rik
--
"What you're running that piece of shit Gnome?!?!"
-- Miguel de Icaza, UKUUG 2000

http://www.conectiva.com/ http://www.surriel.com/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Rajagopal Ananthanarayanan

author

2000-10-04 21:39 UTC #2 lore raw

Quoted 14 lines · Rik van Riel

Quoted 7 lines · Rajagopal Ananthanarayanan
I'm running fairly stressful tests like dbench with lots of clients. Since the new VM changes (now in test9), I haven't noticed _any_ swap activity, in spite of the enormous memory pressures. I have lots of processes in the system, like 8 httpd's, 4 getty's, etc. most of which should be "idle" ... Why aren't the pages (eg. mapped stacks) from these processes being swapped out?
That's an interesting one. Most "complaints" I've had about test9 is that it swaps more than previous versions ;)

Quoted 6 lines

But let me give you the answer...
Small code changes in deactivate_page() have caused the drop_behind() code to actually WORK AS ADVERTISED right now, and because of that streaming IO doesn't put any memory pressure on the system.

Agreed. And since the introduction of drop_behind & the deactivate_page() in generic_file_write, streaming I/O performance has become pretty good.

However, in the above I was particularly talking about swap behaviour on running dbench. Dbench is write intensive, and also has fair amount of re-writes. So, the I'm not sure why we still do not swap out _really_ old processes.

If old pages are not swapped out, then dbench itself will get less than optimal amount of the page-cache during its run. I believe this is one of the reasons for dbench's poor showing with the new VM.

Rik van Riel

2000-10-04 21:46 UTC #3 lore raw

Quoted 12 lines · Rajagopal Ananthanarayanan

Quoted 10 lines · Rik van Riel
Quoted 3 lines · Rajagopal Ananthanarayanan
I'm running fairly stressful tests like dbench with lots of clients. Since the new VM changes (now in test9), I haven't noticed _any_ swap activity, in spite of the enormous memory
Small code changes in deactivate_page() have caused the drop_behind() code to actually WORK AS ADVERTISED right now, and because of that streaming IO doesn't put any memory pressure on the system.

Quoted 8 lines

Agreed. And since the introduction of drop_behind & the deactivate_page() in generic_file_write, streaming I/O performance has become pretty good.
However, in the above I was particularly talking about swap behaviour on running dbench. Dbench is write intensive, and also has fair amount of re-writes. So, the I'm not sure why we still do not swap out _really_ old processes.

Please take a look at vmscan.c::refill_inactive()

Furthermore, we don't do background scanning on all active pages, only on the unmapped ones.

This is one of the things we'll be able to fix in
2.5...

Quoted 4 lines

If old pages are not swapped out, then dbench itself will get less than optimal amount of the page-cache during its run. I believe this is one of the reasons for dbench's poor showing with the new VM.

Agreed, but I don't see an "easy" solution for 2.4.

regards,

Rik
--
"What you're running that piece of shit Gnome?!?!"
-- Miguel de Icaza, UKUUG 2000

http://www.conectiva.com/ http://www.surriel.com/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Rajagopal Ananthanarayanan

author

2000-10-04 22:05 UTC #4 lore raw

Rik van Riel wrote:

	[ ... ]

Quoted 5 lines

Please take a look at vmscan.c::refill_inactive()
Furthermore, we don't do background scanning on all active pages, only on the unmapped ones.

Does that mean stack pages of processes are not included? Non-aggressive swap can hurt performance.

Quoted 3 lines

Agreed, but I don't see an "easy" solution for 2.4.

Ok, I have another suggestion. Suppose you had a situation where a page is read from disk. It has buffers. Initially the page is active, and then aged. Where does the page go at age = 0? In reading the current code it seems that it would go to inactive_dirty. See how deactivate_page() chooses the dirty list to add a page which has buffers. Of course, later page_launder() would do try_to_free_buffers() which discards (clean) buffer heads, and at that point the page is put on free_list/reclaimed. Would it not be more efficient to bung clean (read) pages directly to inactive_clean on age = 0?

Rik van Riel

2000-10-04 22:13 UTC #5 lore raw

Quoted 10 lines · Rajagopal Ananthanarayanan

Rik van Riel wrote:
	[ ... ]
Quoted 5 lines
Please take a look at vmscan.c::refill_inactive()
Furthermore, we don't do background scanning on all active pages, only on the unmapped ones.
Does that mean stack pages of processes are not included? Non-aggressive swap can hurt performance.

I don't really see a clean way to do that in 2.4 ...

Quoted 12 lines

Quoted 1 line
Agreed, but I don't see an "easy" solution for 2.4.
Ok, I have another suggestion. Suppose you had a situation where a page is read from disk. It has buffers. Initially the page is active, and then aged. Where does the page go at age = 0? In reading the current code it seems that it would go to inactive_dirty. See how deactivate_page() chooses the dirty list to add a page which has buffers. Of course, later page_launder() would do try_to_free_buffers() which discards (clean) buffer heads, and at that point the page is put on free_list/reclaimed. Would it not be more efficient to bung clean (read) pages directly to inactive_clean on age = 0?

I don't know if this would make any difference...

And in fact, I'm contemplating adding /all/ pages
that are deactivated to the inactive_dirty list,
since that way we'll reclaim all inactive pages
in FIFO order.

Currently we may "skip" some pages that were put
on the inactive_dirty list but were cleaned up
subsequently because we can find enough active
pages that can be moved to the inactive_clean
list immediately ...

regards,

Rik
--
"What you're running that piece of shit Gnome?!?!"
-- Miguel de Icaza, UKUUG 2000

http://www.conectiva.com/ http://www.surriel.com/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Rajagopal Ananthanarayanan

author

2000-10-04 23:03 UTC #6 lore raw

Quoted 4 lines · Rik van Riel

Quoted 2 lines
Does that mean stack pages of processes are not included? Non-aggressive swap can hurt performance.
I don't really see a clean way to do that in 2.4 ...

We can perhaps talk about this at the Storage Workshop ...

	[ ... ]

Quoted 4 lines

Quoted 2 lines
Would it not be more efficient to bung clean (read) pages directly to inactive_clean on age = 0?
I don't know if this would make any difference...

I think it would. Consider steady state where pages are all "in use". Any new allocation has to start with pushing a page from active -> inactive, and then inactive -> inactive_clean, if necessary, and then reclaim. Now, if we had pages which _are_ clean, then the path taken is simply active -> inactive_clean -> reclaim.

Quoted 6 lines

And in fact, I'm contemplating adding /all/ pages that are deactivated to the inactive_dirty list, since that way we'll reclaim all inactive pages in FIFO order.

Quoted 6 lines

Currently we may "skip" some pages that were put on the inactive_dirty list but were cleaned up subsequently because we can find enough active pages that can be moved to the inactive_clean list immediately ...

This is an interesting idea, although it seems antithetical to what I said above. I think pure FIFO has its merits in accomodating longer locality of reference; it can help dbench.

If you have a patch (to always deactivate to inactive_dirty), I can help you gauge it with the benchmarks ...

Rik van Riel

2000-10-05 00:07 UTC #7 lore raw

Quoted 8 lines · Rajagopal Ananthanarayanan

Quoted 4 lines · Rik van Riel
Quoted 2 lines
Does that mean stack pages of processes are not included? Non-aggressive swap can hurt performance.
I don't really see a clean way to do that in 2.4 ...
We can perhaps talk about this at the Storage Workshop ...

That would be a great idea. Maybe over dinner or lunch, with some of the other people present as well? ;)

[I'll also bring guarana, to facilitate all-night hacking sessions]

Quoted 11 lines

Quoted 4 lines
Quoted 2 lines
Would it not be more efficient to bung clean (read) pages directly to inactive_clean on age = 0?
I don't know if this would make any difference...
I think it would. Consider steady state where pages are all "in use". Any new allocation has to start with pushing a page from active -> inactive, and then inactive -> inactive_clean, if necessary, and then reclaim. Now, if we had pages which _are_ clean, then the path taken is simply active -> inactive_clean -> reclaim.

Indeed. Then again, we probably want to clear the unused buffer heads off of active pages anyway, and in that case these pages will be cleaned for us.

This is something I'm looking into at the moment...

[But it has lower priority than out of memory and the high memory lockup problem ... if that still exists]

Quoted 12 lines

Quoted 11 lines
And in fact, I'm contemplating adding /all/ pages that are deactivated to the inactive_dirty list, since that way we'll reclaim all inactive pages in FIFO order.
Currently we may "skip" some pages that were put on the inactive_dirty list but were cleaned up subsequently because we can find enough active pages that can be moved to the inactive_clean list immediately ...

Quoted 4 lines

This is an interesting idea, although it seems antithetical to what I said above. I think pure FIFO has its merits in accomodating longer locality of reference; it can help dbench.

I have no idea if it would help or not; only one
way to find out I guess...

Quoted 2 lines

If you have a patch (to always deactivate to inactive_dirty), I can help you gauge it with the benchmarks ...

Quick and dirty patch below ;)

regards,

Rik
--
"What you're running that piece of shit Gnome?!?!"
-- Miguel de Icaza, UKUUG 2000

http://www.conectiva.com/ http://www.surriel.com/

swap.c renamed from swap.c.orig +1 -1

@@ -201,7 +201,7 @@
 		} else if (page->mapping && !PageDirty(page) &&
 							!PageLocked(page)) {
 			del_page_from_active_list(page);
-			add_page_to_inactive_clean_list(page);
+			add_page_to_inactive_dirty_list(page);
 		}
 		/*
 		 * OK, we cannot free the page. Leave it alone.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Rajagopal Ananthanarayanan

author

2000-10-05 02:31 UTC #8 lore raw

Rik van Riel wrote:

	[ ... ]

Quoted 6 lines

Quoted 3 lines
If you have a patch (to always deactivate to inactive_dirty), I can help you gauge it with the benchmarks ...
Quick and dirty patch below ;)

	[ ... ]

Now, I'm not sure whether it helps either. It seemed to help on 16 & 32 clients, but has worse effects w/ 48 clients. In reality, i guess having a single inactive (dirty + clean) might help; but seeing good end results probably requires other things (such as the swap thing) to be in place.

Back to looking at code.

cheers,