MM fix & improvement
PATCH 1/1 MM fix & improvement
OK, here it goes. Patch is unbelievably small, and improvement is BIG.
Two things:
1) Til' now, writing to swap files & partitions was clustered in 128kb chunks which is too small, especially now when we have swapin readahead (default 64kb). Fragmentation of swap file with such a value is big, so swapin readahead hit rate is probably small. Thus first improvement is in increasing on-disk cluster size to much bigger value. I chose 512, and it works very well, indeed (see below). All this is completely safe.
2) There was an artificial limit in swapin readahead code, that is completely unnecessary, and also kills performance a big time. It also makes trouble with two thrashing tasks, because swapin readahead doesn't work very well in such low memory condition. I removed it alltogether, and swapping quite improved.
If you don't believe, look at benchmarks I made for your eyes only (swap cache statistics is after both runs):
pre6 + MM cleanup (needed for swap cache hit rate)
hogmem 100 3 - 10.75 MB/sec
2 x hogmem 50 3 - 2.01 + 1.97 MB/sec (disk thrashing)
swap cache - add 194431 find 13315/194300 (6.9% hit rate)
pre6 + MM cleanup + patch below
hogmem 100 3 - 13.27 MB/sec
2 x hogmem 50 3 - 6.15 + 5.77 MB/sec (perfect)
swap cache - add 175887 find 76003/237711 (32% hit rate)
Notice how swap cache done it's job much better with changes applied!!!
Both tests were run in single user mode, after reboot, on 64MB machine. Don't be disappointed if you get smaller numbers, I have two swap partitions on different disks and transports (IDE + SCSI). :)
Signature · 2 lines
-- Zlatko
Quoted 8 lines · Zlatko Calusic
1) Til' now, writing to swap files & partitions was clustered in 128kb chunks which is too small, especially now when we have swapin readahead (default 64kb). Fragmentation of swap file with such a value is big, so swapin readahead hit rate is probably small. Thus first improvement is in increasing on-disk cluster size to much bigger value. I chose 512, and it works very well, indeed (see below). All this is completely safe.
Oops, in fact 256 is in patch. I used 512 in my preliminary tests. But, either value will work well.
Signature · 4 lines
Zlatko -- This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Linus Torvalds
AppliedQuoted 3 lines · Zlatko Calusic
OK, here it goes. Patch is unbelievably small, and improvement is BIG.
Looks good. Especially the fact that once again performance got a lot better by _removing_ some silly heuristics that didn't actually work.
Applied,
Linus
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Quoted 8 lines · Linus Torvalds
Quoted 3 lines · Zlatko Calusic
OK, here it goes. Patch is unbelievably small, and improvement is BIG.
Looks good. Especially the fact that once again performance got a lot better by _removing_ some silly heuristics that didn't actually work.
Yes, that's the thing that I like with changes, too. Let's remove anything that can be removed. :)
I'm just looking into buffer reference issue you mentioned.
I already noticed that part of code that was dealing with buffer reference in shrink_mmap() disappeared some time ago. But I didn't have a slightest clue that fs/buffer.c survived such mass deletia, as one I just found in there. :)
I'll implement simple buffer->page reference functionality, benchmark it and let you know what I've found.
At this point, I think it will improve things, but let's wait and see.
Regards,
Signature · 4 lines
Zlatko -- This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Not so fast.... :(
Here are the numbers (P2-400, 128MB SDRAM, one IDE drive), but the interesting part is below:
./hogmem 200 3
Memory speed: 6.53 MB/sec
./hogmem 100 3 & ./hogmem 100 3 &
Memory speed: 1.74 MB/sec
Memory speed: 1.71 MB/sec
With pre6, Zlatko cleanup patch and MM fix and Linus's one line patch (try_to_free_pages), while swapping with either the one hogmem, or both running at the same time, the system is basically unusable for interactive use. Keyboard and mouse input is lagged several minutes, and switching virtual desktops in X take forever.
With plain vanilla Pre6, the two hogmems would take a very long time to complete, and the speed was < 1 MB/sec, however, the system was pretty usuable. I could switch virtual desktops in about 5-6 seconds, and mouse input was slightly lagged, but typing in an xterm had no noticeable lag.
This kernel MM stuff is way way beyond my head, but I can test stuff. :)
Dax Kelson
Internet Connect, Inc.
Quoted 12 lines · Linus Torvalds
Quoted 3 lines · Zlatko Calusic
OK, here it goes. Patch is unbelievably small, and improvement is BIG.
Looks good. Especially the fact that once again performance got a lot better by _removing_ some silly heuristics that didn't actually work.
Applied,
Quoted 8 lines
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Hi,
Quoted 3 lines · Zlatko Calusic
1) Til' now, writing to swap files & partitions was clustered in 128kb chunks which is too small, especially now when we have swapin readahead (default 64kb).
Right --- it's not actually the write clustering, but the allocation clustering which was the problem. Well spotted.
--Stephen
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Quoted 2 lines · Zlatko Calusic
OK, here it goes. Patch is unbelievably small, and improvement is BIG.
[Snip]
Quoted 6 lines
pre6 + MM cleanup (needed for swap cache hit rate)
hogmem 100 3 - 10.75 MB/sec2 x hogmem 50 3 - 2.01 + 1.97 MB/sec (disk thrashing)
swap cache - add 194431 find 13315/194300 (6.9% hit rate)
Quoted 7 lines
pre6 + MM cleanup + patch below
hogmem 100 3 - 13.27 MB/sec2 x hogmem 50 3 - 6.15 + 5.77 MB/sec (perfect)
swap cache - add 175887 find 76003/237711 (32% hit rate)Notice how swap cache done it's job much better with changes applied!!!
Looks REALLY nice...
Quoted 3 lines
Both tests were run in single user mode, after reboot, on 64MB machine. Don't be disappointed if you get smaller numbers, I have two swap partitions on different disks and transports (IDE + SCSI). :)
Have you tried your patch on a low-memory machine and/or a low-capacity processor, ie a 386 with say 4 MB's of memory?!
/David Weinehall
_ _
// David Weinehall <[email protected]> /> Northern lights wander \\
// Project MCA Linux hacker // Dance across the winter sky // \> http://www.acc.umu.se/~tao/ </ Full colour fire </
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]