LMbench2.0 results
Subsystems (2)
Hi all,
I've just ran lmbench2.0 on my laptop.
Here the results (again, 2.5.33 seems to be "slow", I don't know why...)
cd results && make summary percent 2>/dev/null | more make[1]: Entering directory `/usr/src/LMbench/results'
L M B E N C H 2 . 0 S U M M A R Y
------------------------------------
Basic system parameters
----------------------------------------------------
Host OS Description Mhz
--------- ------------- ----------------------- ----
frodo Linux 2.4.18 i686-pc-linux-gnu 797 frodo Linux 2.4.19 i686-pc-linux-gnu 797 frodo Linux 2.5.33 i686-pc-linux-gnu 797
Processor, Processes - times in microseconds - smaller is better
----------------------------------------------------------------
Host OS Mhz null null open selct sig sig fork exec sh
call I/O stat clos TCP inst hndl proc proc proc
--------- ------------- ---- ---- ---- ---- ---- ----- ---- ---- ---- ---- ----
frodo Linux 2.4.18 797 0.40 0.56 3.18 3.97 1.00 3.18 115. 1231 13.K frodo Linux 2.4.19 797 0.40 0.56 3.07 3.88 1.00 3.19 129. 1113 13.K frodo Linux 2.5.33 797 0.40 0.61 3.78 4.76 1.02 3.37 201. 1458 13.K
Context switching - times in microseconds - smaller is better
-------------------------------------------------------------
Host OS 2p/0K 2p/16K 2p/64K 8p/16K 8p/64K 16p/16K 16p/64K
ctxsw ctxsw ctxsw ctxsw ctxsw ctxsw ctxsw
--------- ------------- ----- ------ ------ ------ ------ ------- -------
frodo Linux 2.4.18 0.990 4.4200 13.8 6.2700 309.8 58.6 310.5 frodo Linux 2.4.19 0.900 4.2900 15.3 5.9100 309.6 57.7 309.9 frodo Linux 2.5.33 1.620 5.2800 15.3 9.3500 312.7 54.9 312.7
*Local* Communication latencies in microseconds - smaller is better
-------------------------------------------------------------------
Host OS 2p/0K Pipe AF UDP RPC/ TCP RPC/ TCP
ctxsw UNIX UDP TCP conn
--------- ------------- ----- ----- ---- ----- ----- ----- ----- ----
frodo Linux 2.4.18 0.990 4.437 8.66 frodo Linux 2.4.19 0.900 4.561 7.76 frodo Linux 2.5.33 1.620 6.497 9.11
File & VM system latencies in microseconds - smaller is better
--------------------------------------------------------------
Host OS 0K File 10K File Mmap Prot Page
Create Delete Create Delete Latency Fault Fault
--------- ------------- ------ ------ ------ ------ ------- ----- -----
frodo Linux 2.4.18 68.9 16.0 185.8 31.6 425.0 0.789 2.00000
frodo Linux 2.4.19 68.9 14.9 186.5 29.8 416.0 0.798 2.00000
frodo Linux 2.5.33 77.8 19.1 211.6 38.3 774.0 0.832 3.00000*Local* Communication bandwidths in MB/s - bigger is better
-----------------------------------------------------------
Host OS Pipe AF TCP File Mmap Bcopy Bcopy Mem Mem
UNIX reread reread (libc) (hand) read write
--------- ------------- ---- ---- ---- ------ ------ ------ ------ ---- -----
frodo Linux 2.4.18 810. 650. 181.7 203.7 101.5 101.4 203. 195.3 frodo Linux 2.4.19 808. 680. 187.2 203.8 101.5 101.4 203. 190.1 frodo Linux 2.5.33 571. 636. 185.6 202.5 100.5 100.4 202. 190.3
Memory latencies in nanoseconds - smaller is better
(WARNING - may not be correct, check graphs)
---------------------------------------------------
Host OS Mhz L1 $ L2 $ Main mem Guesses --------- ------------- ---- ----- ------ -------- ------- frodo Linux 2.4.18 797 3.767 8.7890 158.9 frodo Linux 2.4.19 797 3.767 8.7980 158.9 frodo Linux 2.5.33 797 3.798 8.8660 160.1
make[1]: Leaving directory `/usr/src/LMbench/results'
Comments?
Let me know if you need further information (.config, info about my hardware) or if you want I run other tests.
Ciao,
Paolo
Signature · 4 lines
Get your free email from www.linuxmail.org Powered by Outblaze
Quoted 1 line · Paolo Ciarrocchi
Comments?
Yeah: "ouch" because I don't see a single category that's faster.
Oh well, it still needs to be tuned....
Quoted 2 lines · Paolo Ciarrocchi
Let me know if you need further information (.config, info about my hardware) or if you want I run other tests.
Would you be able to run the tests for 2.5.31? I'm looking into a slowdown in 2.5.32/33 which may be related. Some hardware info might be useful too.
- James
Signature · 2 lines
James Morris <[email protected]>
Quoted 5 lines · Paolo Ciarrocchi
Hi all,
I've just ran lmbench2.0 on my laptop.
Here the results (again, 2.5.33 seems to be "slow", I don't know why...)
The fork/exec/mmap slowdown is the rmap overhead. I have some stuff which partialy improves it.
The many-small-file-create slowdown is known but its cause is not. I need to get oprofile onto it.
Quoted 4 lines · Jeff Garzik
Quoted 1 line · Paolo Ciarrocchi
Comments?
Yeah: "ouch" because I don't see a single category that's faster.
HZ went to 1000, which should help multimedia latencies a lot.
Quoted 1 line
Oh well, it still needs to be tuned....
For throughput or for latency ? ;)
Rik
Signature · 5 lines
Bravely reimplemented by the knights who say "NIH". http://www.surriel.com/ http://distro.conectiva.com/ Spamtraps of the month: [email protected] [email protected]
Quoted 3 lines · Paolo Ciarrocchi
Quoted 3 lines
Hi all,
I've just ran lmbench2.0 on my laptop.
Here the results (again, 2.5.33 seems to be "slow", I don't know why...)
Quoted 2 lines · Andrew Morton
The fork/exec/mmap slowdown is the rmap overhead. I have some stuff which partialy improves it.
Hmm, Where does it enter the mmap() path? PTE instantiation is only done for the VM_LOCKED case IIRC. Otherwise it should be invisible.
Perhaps testing with overcommit on would be useful.
Cheers,
Bill
Quoted 7 lines · Rik van Riel
Quoted 4 lines · Jeff Garzik
Quoted 1 line · Paolo Ciarrocchi
Comments?
Yeah: "ouch" because I don't see a single category that's faster.
HZ went to 1000, which should help multimedia latencies a lot.
It shouldn't materially damage performance unless we have other things extremely wrong. Its easy enough to verify by putting HZ back to 100 and rebenching
Quoted 6 lines · William Lee Irwin III
Quoted 3 lines · Paolo Ciarrocchi
Quoted 3 lines
Hi all,
I've just ran lmbench2.0 on my laptop.
Here the results (again, 2.5.33 seems to be "slow", I don't know why...)
Quoted 6 lines
Quoted 2 lines · Andrew Morton
The fork/exec/mmap slowdown is the rmap overhead. I have some stuff which partialy improves it.
Hmm, Where does it enter the
mmap()path? PTE instantiation is only done for theVM_LOCKEDcase IIRC. Otherwise it should be invisible.
Oh, is that just the mmap() call itself?
Quoted 1 line
Perhaps testing with overcommit on would be useful.
Well yes - the new overcommit code was a significant hit on the 16ways was it not? You have some numbers on that?
Quoted 2 lines · Andrew Morton
Quoted 2 lines
Quoted 2 lines
The fork/exec/mmap slowdown is the rmap overhead. I have some stuff which partialy improves it.
Quoted 2 lines · William Lee Irwin III
Quoted 2 lines
Hmm, Where does it enter the
mmap()path? PTE instantiation is only done for theVM_LOCKEDcase IIRC. Otherwise it should be invisible.
Quoted 1 line · Andrew Morton
Oh, is that just the
mmap()call itself?
I'm not actually sure what lmbench is doing.
Quoted 1 line · William Lee Irwin III
Quoted 1 line
Perhaps testing with overcommit on would be useful.
Quoted 2 lines · Andrew Morton
Well yes - the new overcommit code was a significant hit on the 16ways was it not? You have some numbers on that?
I don't remember the before/after numbers, but I can collect some.
Cheers,
Bill
Quoted 4 lines
Quoted 1 line
Perhaps testing with overcommit on would be useful.
Well yes - the new overcommit code was a significant hit on the 16ways was it not? You have some numbers on that?
About 20% hit on system time for kernel compiles.
M.
Quoted 6 lines · William Lee Irwin III
Quoted 3 lines · Paolo Ciarrocchi
Quoted 3 lines
Hi all,
I've just ran lmbench2.0 on my laptop.
Here the results (again, 2.5.33 seems to be "slow", I don't know why...)
Quoted 7 lines
Quoted 2 lines · Andrew Morton
The fork/exec/mmap slowdown is the rmap overhead. I have some stuff which partialy improves it.
Hmm, Where does it enter the
mmap()path? PTE instantiation is only done for theVM_LOCKEDcase IIRC. Otherwise it should be invisible.
lat_mmap seems to do a mmap, faults in ten pages and then a munmap(). Most of the CPU cost is in cache misses against the pagetables in munmap().
c012d54c 153 0.569493 do_mmap_pgoff c012db5c 158 0.588104 find_vma c01301ec 172 0.640214 filemap_nopage c0134e84 172 0.640214 release_pages c0114744 184 0.684881 smp_apic_timer_interrupt c012ce3c 248 0.9231 handle_mm_fault c012f738 282 1.04965 find_get_page c013e2b0 356 1.32509 __set_page_dirty_buffers c0116294 377 1.40326 do_page_fault c013e72c 383 1.42559 page_add_rmap c013e8bc 398 1.48143 page_remove_rmap c012cb10 425 1.58193 do_no_page c0109d70 629 2.34125 page_fault c012b2f4 1036 3.85618 zap_pte_range c0107048 20205 75.2066 poll_idle
(Multiply everything by four - it's a quad)
Instruction-level profile for -mm5:
c012b2f4 1036 3.85618 0 0 zap_pte_range /usr/src/25/mm/memory.c:325
c012b2f5 2 0.19305 0 0 /usr/src/25/mm/memory.c:325 c012b2fd 1 0.0965251 0 0 /usr/src/25/mm/memory.c:325 c012b300 2 0.19305 0 0 /usr/src/25/mm/memory.c:325 c012b306 1 0.0965251 0 0 /usr/src/25/mm/memory.c:329 c012b309 1 0.0965251 0 0 /usr/src/25/mm/memory.c:329 c012b30f 1 0.0965251 0 0 /usr/src/25/mm/memory.c:331 c012b319 1 0.0965251 0 0 /usr/src/25/mm/memory.c:331 c012b340 1 0.0965251 0 0 /usr/src/25/mm/memory.c:336 c012b348 1 0.0965251 0 0 /usr/src/25/include/asm/highmem.h:80 c012b350 1 0.0965251 0 0 /usr/src/25/include/asm/thread_info.h:75 c012b35a 2 0.19305 0 0 /usr/src/25/include/asm/highmem.h:85 c012b365 2 0.19305 0 0 /usr/src/25/include/asm/highmem.h:86 c012b3c3 2 0.19305 0 0 /usr/src/25/mm/memory.c:337 c012b3d6 1 0.0965251 0 0 /usr/src/25/mm/memory.c:338 c012b3e9 3 0.289575 0 0 /usr/src/25/mm/memory.c:341 c012b3f5 106 10.2317 0 0 /usr/src/25/mm/memory.c:342 c012b3f8 2 0.19305 0 0 /usr/src/25/mm/memory.c:342 c012b3fa 26 2.50965 0 0 /usr/src/25/mm/memory.c:343 c012b3fc 124 11.9691 0 0 /usr/src/25/mm/memory.c:343 c012b405 13 1.25483 0 0 /usr/src/25/mm/memory.c:345 c012b40b 1 0.0965251 0 0 /usr/src/25/mm/memory.c:346 c012b410 2 0.19305 0 0 /usr/src/25/mm/memory.c:348 c012b412 1 0.0965251 0 0 /usr/src/25/mm/memory.c:348 c012b414 62 5.98456 0 0 /usr/src/25/mm/memory.c:349 c012b41b 1 0.0965251 0 0 /usr/src/25/mm/memory.c:350 c012b421 21 2.02703 0 0 /usr/src/25/mm/memory.c:350 c012b427 2 0.19305 0 0 /usr/src/25/mm/memory.c:351 c012b432 2 0.19305 0 0 /usr/src/25/include/asm/bitops.h:244 c012b434 10 0.965251 0 0 /usr/src/25/mm/memory.c:352 c012b437 1 0.0965251 0 0 /usr/src/25/mm/memory.c:352 c012b43d 5 0.482625 0 0 /usr/src/25/mm/memory.c:353 c012b446 7 0.675676 0 0 /usr/src/25/include/linux/mm.h:389 c012b44b 1 0.0965251 0 0 /usr/src/25/include/linux/mm.h:392 c012b44e 1 0.0965251 0 0 /usr/src/25/include/linux/mm.h:392 c012b451 7 0.675676 0 0 /usr/src/25/include/linux/mm.h:393 c012b453 2 0.19305 0 0 /usr/src/25/include/linux/mm.h:393 c012b461 6 0.579151 0 0 /usr/src/25/include/linux/mm.h:396 c012b466 8 0.772201 0 0 /usr/src/25/include/linux/mm.h:396 c012b46f 6 0.579151 0 0 /usr/src/25/mm/memory.c:356 c012b476 15 1.44788 0 0 /usr/src/25/include/asm-generic/tlb.h:105 c012b481 3 0.289575 0 0 /usr/src/25/include/asm-generic/tlb.h:106 c012b490 5 0.482625 0 0 /usr/src/25/include/asm-generic/tlb.h:110 c012b493 7 0.675676 0 0 /usr/src/25/include/asm-generic/tlb.h:110 c012b49a 1 0.0965251 0 0 /usr/src/25/include/asm-generic/tlb.h:110 c012b49d 3 0.289575 0 0 /usr/src/25/include/asm-generic/tlb.h:110 c012b4a0 1 0.0965251 0 0 /usr/src/25/include/asm-generic/tlb.h:110 c012b4a3 8 0.772201 0 0 /usr/src/25/include/asm-generic/tlb.h:111 c012b4aa 13 1.25483 0 0 /usr/src/25/include/asm-generic/tlb.h:111 c012b500 128 12.3552 0 0 /usr/src/25/mm/memory.c:341 c012b504 108 10.4247 0 0 /usr/src/25/mm/memory.c:341 c012b50b 111 10.7143 0 0 /usr/src/25/mm/memory.c:341 c012b50e 99 9.55598 0 0 /usr/src/25/mm/memory.c:341 c012b511 86 8.30116 0 0 /usr/src/25/mm/memory.c:341 c012b51c 4 0.3861 0 0 /usr/src/25/include/asm/thread_info.h:75 c012b521 3 0.289575 0 0 /usr/src/25/mm/memory.c:366 c012b525 1 0.0965251 0 0 /usr/src/25/mm/memory.c:366 c012b526 1 0.0965251 0 0 /usr/src/25/mm/memory.c:366
So it's a bit of rmap in there. I'd have to compare with a 2.4 profile and fiddle a few kernel parameters. But I'm not sure that munmap of extremely sparsely populated pagtetables is very interesting?
From: Andrew Morton <[email protected]>
Date: Sun, 08 Sep 2002 00:51:19 -0700
So it's a bit of rmap in there. I'd have to compare with a 2.4 profile and fiddle a few kernel parameters. But I'm not sure that munmap of extremely sparsely populated pagtetables is very interesting?
Another issue is that x86 doesn't use a pagetable cache. I think it got killed from x86 when the pagetables in highmem went in.
This is all from memory.
From: Andrew Morton <[email protected]>
Quoted 4 lines
So it's a bit of rmap in there. I'd have to compare with a 2.4 profile and fiddle a few kernel parameters. But I'm not sure that munmap of extremely sparsely populated pagtetables is very interesting?
Quoted 3 lines · David S. Miller
Another issue is that x86 doesn't use a pagetable cache. I think it got killed from x86 when the pagetables in highmem went in. This is all from memory.
They seemed to have some other issues related to extreme memory pressure (routine for me). But if this were truly the issue, the allocation and deallocation overhead for pagetables should show up as additional pressure against zone->lock. I can't tell at the moment because zone->lock is hammered quite hard to begin with and no one's gone out and done a pagetable cacheing patch for the stuff since. It should be simple to chain with links in struct page instead of links embedded in the pagetables & smp_call_function() to reclaim. But this raises questions of generality.
Cheers,
Bill
From: William Lee Irwin III <[email protected]>
Date: Sun, 8 Sep 2002 01:28:21 -0700
But if this were truly the issue, the allocation and deallocation overhead for pagetables should show up as additional pressure against zone->lock.
The big gain is not only that allocation/free is cheap, also page table entries tend to hit in cpu cache for even freshly allocated page tables.
I think that is the bit that would show up in the mmap lmbench test.
From: William Lee Irwin III <[email protected]>
Date: Sun, 8 Sep 2002 01:28:21 -0700
Quoted 3 lines
But if this were truly the issue, the allocation and deallocation overhead for pagetables should show up as additional pressure against
zone->lock.
Quoted 5 lines · David S. Miller
The big gain is not only that allocation/free is cheap, also page table entries tend to hit in cpu cache for even freshly allocated page tables.
I think that is the bit that would show up in the mmap lmbench test.
I'd have to doublecheck to see how parallelized lat_mmap is. My machines are considerably more sensitive to locking uglies than cache warmth. (They're taking my machines out, not just slowing them down.) Cache warmth goodies are certainly nice optimizations, though.
Cheers,
Bill
Quoted 6 lines · Martin J. Bligh
Quoted 4 lines
Quoted 1 line
Perhaps testing with overcommit on would be useful.
Well yes - the new overcommit code was a significant hit on the 16ways was it not? You have some numbers on that?
About 20% hit on system time for kernel compiles.
That suprises me a lot. On a 2 way and 4 way the 2.4 memory overcommit check code didnt show up. That may be down to the 2 way being on a CPU that has no measurable cost for locked operations and the 4 way being an ancient ppro a friend has.
If it is the memory overcommit handling then there are plenty of ways to deal with it efficiently in the non-preempt case at least. I had wondered originally about booking chunks of pages off per CPU (take the remaining overcommit divide by four and only when a CPU finds its private block is empty take a lock and redistribute the remaining allocation). Since boxes almost never get that close to overcommit kicking in then it should mean we close to never touch a locked count.
Quoted 11 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Perhaps testing with overcommit on would be useful.
Well yes - the new overcommit code was a significant hit on the 16ways was it not? You have some numbers on that?
About 20% hit on system time for kernel compiles.
That suprises me a lot. On a 2 way and 4 way the 2.4 memory overcommit check code didnt show up. That may be down to the 2 way being on a CPU that has no measurable cost for locked operations and the 4 way being an ancient ppro a friend has.
Remember this is a NUMA machine - gathering global information is extremely expensive. On an SMP system, I wouldn't expect it to show up so much, though it still doesn't seem terribly efficient. The code is admits it's broken anyway, for the overcommit = 2 case (which was NOT what I was running - the 20% is for 1). Below is a simple patch that I've never got around to testing, that I think will improve that case (not that I'm that interested in setting overcommit to 2 ;-)).
Quoted 7 lines
If it is the memory overcommit handling then there are plenty of ways to deal with it efficiently in the non-preempt case at least. I had wondered originally about booking chunks of pages off per CPU (take the remaining overcommit divide by four and only when a CPU finds its private block is empty take a lock and redistribute the remaining allocation). Since boxes almost never get that close to overcommit kicking in then it should mean we close to never touch a locked count.
Can you use per-zone stats rather than global ones? That tends to fix things pretty efficently on these type of machines - per zone LRUs made a huge impact.
Here's a little patch (untested!). I'll go look at the other case and see if there's something easy to do, but I think it needs some significant rework to do anything.
mm/mmap.c +1 -7
@@ -74,7 +74,6 @@int vm_enough_memory (long pages) {unsigned long free, allowed; -struct sysinfo i;atomic_add (pages, &vm_committed_space);
@@ -115,12 +114,7 @@return 0 ; } -/* -* FIXME: need to add arch hooks to get the bits we need -* without this higher overhead crap -*/ -si_meminfo (&i); - allowed = i.totalram * sysctl_overcommit_ratio /100 ; + allowed = totalram_pages * sysctl_overcommit_ratio /100 ; allowed += total_swap_pages;if (atomic_read (&vm_committed_space) < allowed)
Quoted 14 lines · Alan Cox
Quoted 6 lines · Martin J. Bligh
Quoted 4 lines
Quoted 1 line
Perhaps testing with overcommit on would be useful.
Well yes - the new overcommit code was a significant hit on the 16ways was it not? You have some numbers on that?
About 20% hit on system time for kernel compiles.
That suprises me a lot. On a 2 way and 4 way the 2.4 memory overcommit check code didnt show up. That may be down to the 2 way being on a CPU that has no measurable cost for locked operations and the 4 way being an ancient ppro a friend has.
Quoted 7 lines
If it is the memory overcommit handling then there are plenty of ways to deal with it efficiently in the non-preempt case at least. I had wondered originally about booking chunks of pages off per CPU (take the remaining overcommit divide by four and only when a CPU finds its private block is empty take a lock and redistribute the remaining allocation). Since boxes almost never get that close to overcommit kicking in then it should mean we close to never touch a locked count.
Martin had this profile for a kernel build on 2.5.31-mm1:
c01299d0 6761 1.28814 vm_enough_memory c0114584 8085 1.5404 load_balance c01334c0 8292 1.57984 __free_pages_ok c011193c 11559 2.20228 smp_apic_timer_interrupt c0113040 12075 2.3006 do_page_fault c012bf08 12075 2.3006 find_get_page c0114954 12912 2.46007 scheduler_tick c012c430 13199 2.51475 file_read_actor c01727e8 20440 3.89434 __generic_copy_from_user c0133fb8 25792 4.91403 nr_free_pages c01337c0 27318 5.20478 rmqueue c0129588 36955 7.04087 handle_mm_fault c013a65c 38391 7.31447 page_remove_rmap c0134094 43755 8.33645 get_page_state c0105300 57699 10.9931 default_idle c0128e64 58735 11.1905 do_anonymous_page
We can make nr_free_pages go away by adding global free page accounting to struct page_states. So we're accounting it in two places, but it'll be simple.
The global page accounting is very much optimised for the fast path at the expense of get_page_state(). (And that kernel didn't have the rmap speedups).
We need to find some way of making vm_enough_memory not call get_page_state so often. One way of doing that might be to make get_page_state dump its latest result into a global copy, and make vm_enough_memory() only get_page_state once per N invokations. A speed/accuracy tradeoff there.
Quoted 5 lines · Andrew Morton
We need to find some way of making
vm_enough_memorynot callget_page_stateso often. One way of doing that might be to makeget_page_statedump its latest result into a global copy, and makevm_enough_memory()onlyget_page_stateonce per N invokations. A speed/accuracy tradeoff there.
Accuracy is not very important in that sysctl_overcommit_memory 0 case e.g. the swapper_space.nr_pages addition was brought in at a time when it was very necessary, but usually overestimates now (or last time I thought about it). The main thing to look out for is running the same memory grabber twice in quick succession: not nice if it succeeds the first time, but not the second, just because of some transient effect that its old pages are temporarily uncounted.
Hugh
Quoted 8 lines · Hugh Dickins
Quoted 5 lines · Andrew Morton
We need to find some way of making
vm_enough_memorynot callget_page_stateso often. One way of doing that might be to makeget_page_statedump its latest result into a global copy, and makevm_enough_memory()onlyget_page_stateonce per N invokations. A speed/accuracy tradeoff there.
Quoted 8 lines
Accuracy is not very important in that
sysctl_overcommit_memory0 case e.g. the swapper_space.nr_pages addition was brought in at a time when it was very necessary, but usually overestimates now (or last time I thought about it). The main thing to look out for is running the same memory grabber twice in quick succession: not nice if it succeeds the first time, but not the second, just because of some transient effect that its old pages are temporarily uncounted.
That's right - there can be sudden and huge changes in pages used/free.
So any rate limiting tweak in there would have to be in terms of number-of-pages rather than number-of-seconds.
Quoted 9 lines · Andrew Morton
Quoted 5 lines · Paolo Ciarrocchi
Hi all,
I've just ran lmbench2.0 on my laptop.
Here the results (again, 2.5.33 seems to be "slow", I don't know why...)The fork/exec/mmap slowdown is the rmap overhead. I have some stuff which partialy improves it.
It only seems like a big deal if you get out your microscope and focus on the fork times. On the other hand, look at the sh times: the rmap setup time gets lost in the noise. The latter looks more like reality to me.
I suspect the overall performance loss on the laptop has more to do with several months of focussing exclusively on the needs of 4-way and higher smp machines.
Signature · 1 line
Daniel
Quoted 3 lines · Daniel Phillips
I suspect the overall performance loss on the laptop has more to do with several months of focussing exclusively on the needs of 4-way and higher smp machines.
Probably true, we're pulling off an indecent number of tricks for 4-way and 8-way SMP performance. This overhead shouldn't be too bad on UP and 2-way machines, but might easily be a percent or so.
Rik
Signature · 5 lines
Bravely reimplemented by the knights who say "NIH". http://www.surriel.com/ http://distro.conectiva.com/ Spamtraps of the month: [email protected] [email protected]
Quoted 10 lines · Rik van Riel
Quoted 3 lines · Daniel Phillips
I suspect the overall performance loss on the laptop has more to do with several months of focussing exclusively on the needs of 4-way and higher smp machines.
Probably true, we're pulling off an indecent number of tricks for 4-way and 8-way SMP performance. This overhead shouldn't be too bad on UP and 2-way machines, but might easily be a percent or so.
Though to be fair, it's smart to concentrate on the high end with a view to achieving world domination sooner. And it's a stretch to call the low end performance 'slow'.
An idea that's looking more and more attractive as time goes by is to have a global config option that specifies that we want to choose the simple way of doing things wherever possible, over the enterprise way. We want this especially for embedded. On low end processors, it's even possible that the small way will be faster in some cases than the enterprise way, due to cache effects.
Signature · 1 line
Daniel
Quoted 8 lines
Quoted 4 lines
Probably true, we're pulling off an indecent number of tricks for 4-way and 8-way SMP performance. This overhead shouldn't be too bad on UP and 2-way machines, but might easily be a percent or so.
Though to be fair, it's smart to concentrate on the high end with a view to achieving world domination sooner. And it's a stretch to call the low end performance 'slow'.
I don't think there's that much overhead, it's just not where people have been focusing tuning efforts recently. If you run the numbers, and point out specific problems, I'm sure people will fix them ;-) In other words, I don't think the recent focus has caused a problem for low end machines, it just hasn't really looked at solving one.
Quoted 6 lines
An idea that's looking more and more attractive as time goes by is to have a global config option that specifies that we want to choose the simple way of doing things wherever possible, over the enterprise way. We want this especially for embedded. On low end processors, it's even possible that the small way will be faster in some cases than the enterprise way, due to cache effects.
Can't we just use the existing config options instead? CONFIG_SMP is a good start ;-) How many embedded systems with SMP do you have?
M.
Quoted 8 lines · Daniel Phillips
Quoted 6 lines · Rik van Riel
Quoted 3 lines · Daniel Phillips
I suspect the overall performance loss on the laptop has more to do with several months of focussing exclusively on the needs of 4-way and higher smp machines.
Quoted 8 lines
Quoted 4 lines
Probably true, we're pulling off an indecent number of tricks for 4-way and 8-way SMP performance. This overhead shouldn't be too bad on UP and 2-way machines, but might easily be a percent or so.
Though to be fair, it's smart to concentrate on the high end with a view to achieving world domination sooner. And it's a stretch to call the low end performance 'slow'.
It's on the larger machines where 2.4 has problems. Fixing them up makes the kernel broader, more general purpose. We're seeing 50-100% gains in some areas there. Giving away a few percent on smaller machines at this stage is OK. But yup, we need to go and get that back later.
Quoted 3 lines
An idea that's looking more and more attractive as time goes by is to have a global config option that specifies that we want to choose the simple way of doing things wherever possible, over the enterprise way.
Prefer not to. We've been able to cover all bases moderately well thus far without adding a big boolean switch.
Quoted 3 lines
We want this especially for embedded. On low end processors, it's even possible that the small way will be faster in some cases than the enterprise way, due to cache effects.
The main thing we can do for smaller systems is to not allocate as much memory at boot time. Some more careful scaling is needed there. I'll generate a list soon.
Quoted 9 lines · Martin J. Bligh
Quoted 6 lines
An idea that's looking more and more attractive as time goes by is to have a global config option that specifies that we want to choose the simple way of doing things wherever possible, over the enterprise way. We want this especially for embedded. On low end processors, it's even possible that the small way will be faster in some cases than the enterprise way, due to cache effects.
Can't we just use the existing config options instead?
CONFIG_SMPis a good start ;-) How many embedded systems with SMP do you have?
You need to look at it from the other direction: how do the needs of a uniprocessor Clawhammer box differ from a Linksys adsl router?
Signature · 1 line
Daniel
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
An idea that's looking more and more attractive as time goes by is to have a global config option that specifies that we want to choose the simple way of doing things wherever possible, over the enterprise way. We want this especially for embedded. On low end processors, it's even possible that the small way will be faster in some cases than the enterprise way, due to cache effects.
Quoted 5 lines
Quoted 2 lines
Can't we just use the existing config options instead?
CONFIG_SMPis a good start ;-) How many embedded systems with SMP do you have?You need to look at it from the other direction: how do the needs of a uniprocessor Clawhammer box differ from a Linksys adsl router?
I wouldn't call uniprocessor Clawhammer the "enterprise way" type machine. But other than that, I see your point. You're in a far better position to answer your own question than I am, so I'll leave that as rhetorical ;-)
M.
Quoted 2 lines · Daniel Phillips
You need to look at it from the other direction: how do the needs of a uniprocessor Clawhammer box differ from a Linksys adsl router?
I've advocated several times having a single config option for "fine tuning" that sane people say "N" to and which if set lets you force small hash sizes, disable block layer support and kill various other 'always needed' PC crap. Tell me - on a 4Mb embedded 386 running your toaster do you really care if the TCP hash lookup is a little slower than perfect scaling, and do you need a 64Kbyte mount hash ?
Quoted 4 lines · Andrew Morton
We need to find some way of making
vm_enough_memorynot callget_page_stateso often. One way of doing that might be to makeget_page_statedump its latest result into a global copy, and makevm_enough_memory()onlyget_page_stateonce per N invokations. A speed/accuracy tradeoff there.
Unless the error always falls on the same side the accuracy tradeoff is fatal to the entire scheme of things. Sorting out the use of get_page_state is worth doing if that is the bottleneck, and snapshooting such that we only look at it if we might be close to the limit would work, but we'd need to know when the limit had shifted too much
Quoted 7 lines · Alan Cox
Quoted 4 lines · Andrew Morton
We need to find some way of making
vm_enough_memorynot callget_page_stateso often. One way of doing that might be to makeget_page_statedump its latest result into a global copy, and makevm_enough_memory()onlyget_page_stateonce per N invokations. A speed/accuracy tradeoff there.
Quoted 6 lines
Unless the error always falls on the same side the accuracy tradeoff is fatal to the entire scheme of things. Sorting out the use of
get_page_stateis worth doing if that is the bottleneck, and snapshooting such that we only look at it if we might be close to the limit would work, but we'd need to know when the limit had shifted too much
It could be that the cost is only present on the IBM whackomatics, so they can twiddle the /proc setting and we can all be happy. Certainly I did not see any problems on the quad.
Does "heuristic" overcommit handling need so much accuracy? Perhaps we can push some of the cost over into mode 2 somehow.
Or we could turn it the other way up and, in __add_to_page_cache(), do:
if (overcommit_mode == anal) atomic_inc(&nr_pagecache_pages);
Quoted 2 lines · Andrew Morton
Does "heuristic" overcommit handling need so much accuracy? Perhaps we can push some of the cost over into mode 2 somehow.
Its only needed in mode 2, but its also only computed for mode 2,3 in 2,4 8)
Quoted 10 lines · Paolo Ciarrocchi
Quoted 3 lines
Let me know if you need further information (.config, info about my hardware) or if you want I run other tests.
Would you be able to run the tests for 2.5.31? I'm looking into a slowdown in 2.5.32/33 which may be related. Some hardware info might be useful too.
Certainly, we have those in the STP data base, and here's a quick summary:
(Of course you can search these yourself ) The full reports have the hardware summary also. see web links at the end. Full reports have each test run 5x.
Processor, Processes - times in microseconds - smaller is better
----------------------------------------------------------------
Host OS Mhz null null open selct sig sig fork exec sh
call I/O stat clos TCP inst hndl proc proc proc
--------- ------------- ---- ---- ---- ---- ---- ----- ---- ---- ---- ---- ----
stp1-000. Linux 2.5.33 1000 0.33 0.49 2.84 3.52 0.79 2.62 168. 1279 4475 stp1-002. Linux 2.5.32 1000 0.32 0.47 2.94 4.41 15.7 0.80 2.63 202. 1292 4603 stp1-003. Linux 2.5.31 1000 0.32 0.46 2.85 6.92 14.4 0.80 2.60 856. 2596 8122
Context switching - times in microseconds - smaller is better
-------------------------------------------------------------
Host OS 2p/0K 2p/16K 2p/64K 8p/16K 8p/64K 16p/16K 16p/64K
ctxsw ctxsw ctxsw ctxsw ctxsw ctxsw ctxsw
--------- ------------- ----- ------ ------ ------ ------ ------- -------
stp1-000. Linux 2.5.33 1.530 4.1100 12.2 6.4700 136.4 32.7 136.2 stp1-002. Linux 2.5.32 1.590 4.2200 12.4 5.4000 139.1 26.6 136.7 stp1-003. Linux 2.5.31 1.830 46.4 142.6 47.5 141.7 47.6 141.2
*Local* Communication latencies in microseconds - smaller is better
-------------------------------------------------------------------
Host OS 2p/0K Pipe AF UDP RPC/ TCP RPC/ TCP
ctxsw UNIX UDP TCP conn
--------- ------------- ----- ----- ---- ----- ----- ----- ----- ----
stp1-000. Linux 2.5.33 1.530 5.320 10.7 13.8 30.5 19.3 42.1 65.3 stp1-002. Linux 2.5.32 1.570 5.456 11.3 14.2 31.3 21.1 42.6 67.4 stp1-003. Linux 2.5.31 1.810 7.377 14.9 50.5 173.7 117.1 263.8 414.
File & VM system latencies in microseconds - smaller is better
--------------------------------------------------------------
Host OS 0K File 10K File Mmap Prot Page
Create Delete Create Delete Latency Fault Fault
--------- ------------- ------ ------ ------ ------ ------- ----- -----
stp1-000. Linux 2.5.33 32.9 5.4600 117.0 13.3 1261.0 0.575 3.00000
stp1-002. Linux 2.5.32 34.0 5.9460 118.6 14.0 1265.0 0.619 3.00000
stp1-003. Linux 2.5.31 72.5 15.3 225.5 38.2 2062.0 0.657 4.00000*Local* Communication bandwidths in MB/s - bigger is better
-----------------------------------------------------------
Host OS Pipe AF TCP File Mmap Bcopy Bcopy Mem Mem
UNIX reread reread (libc) (hand) read write
--------- ------------- ---- ---- ---- ------ ------ ------ ------ ---- -----
stp1-000. Linux 2.5.33 699. 855. 68.0 407.9 460.1 168.5 157.8 460. 233.8 stp1-002. Linux 2.5.32 690. 297. 93.7 397.5 459.2 162.1 150.0 458. 233.1 stp1-003. Linux 2.5.31 145. 74.8 58.5 118.6 456.9 169.7 156.8 456. 269.0
Full list:
2.5.33 http://khack.osdl.org/stp/4925 -1cpu
http://khack.osdl.org/stp/4915 -1cpu http://khack.osdl.org/stp/4932 -2cpu http://khack.osdl.org/stp/4926 -2cpu 2.5.32 http://khack.osdl.org/stp/4758 -2cpu http://khack.osdl.org/stp/4752 -2cpu http://khack.osdl.org/stp/4751 -1cpu http://khack.osdl.org/stp/4741 -1cpu 2.5.31 http://khack.osdl.org/stp/4302 -1cpu http://khack.osdl.org/stp/4312 -1cpu http://khack.osdl.org/stp/4313 -2cpu http://khack.osdl.org/stp/4319 -2cpu
cliffw
OSDL
Quoted 12 lines
- James
Signature · 9 lines
James Morris <[email protected]> - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Hi!
Quoted 9 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
Comments?
Yeah: "ouch" because I don't see a single category that's faster.
HZ went to 1000, which should help multimedia latencies a lot.
It shouldn't materially damage performance unless we have other things extremely wrong. Its easy enough to verify by putting HZ back to 100 and rebenching
1000 times per second, enter timer interrupt, acknowledge it, exit interrupt. Few i/o accessess, few tlb entries kicked out, some L1 cache consumed?
Is 10usec per timer interrupt reasonable on modern system? That's 10 msec per second spend in timer with HZ=1000, thats 1% overall. So it seems to me it is possible for HZ=1000 to have performance impact...
Pavel
Signature · 2 lines
Worst form of spam? Adding advertisment signatures ala sourceforge.net. What goes next? Inserting advertisment *into* email?