benh performance problem
hi all,
starting 2.6.0-test9-benh3 i noticed an increased system overhead on my ibook 500. Each process eats much more cpu time than in 2.4 or even vanilla 2.6.
gkrellm takes 1% instead of 0.1, xmms 5% instead of ~1% and so on. I benchmarked the lot with octave and it indeed runs ~10% slower on the benh kernels.
I would use vanilla 2.6.1 but it won't sleep/wake correctly (at all). All pre-2.6.0-test9-benh3 kernels worked fast but would't sleep/wake either. I tried to isolate the patch responsible but failed.
Has anybody come across similar behaviour?
greetings, M.
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Quoted 4 lines · Mikolaj Krzewicki
hi all,
starting 2.6.0-test9-benh3 i noticed an increased system overhead on my ibook 500. Each process eats much more cpu time than in 2.4 or even vanilla 2.6.
Quoted 8 lines
gkrellm takes 1% instead of 0.1, xmms 5% instead of ~1% and so on. I benchmarked the lot with octave and it indeed runs ~10% slower on the benh kernels.
I would use vanilla 2.6.1 but it won't sleep/wake correctly (at all). All pre-2.6.0-test9-benh3 kernels worked fast but would't sleep/wake either. I tried to isolate the patch responsible but failed.
Has anybody come across similar behaviour?
greetings, M
I switched to HZ=1000, the userland procps utilities (like ps and top) tend to not properly deal with that, at least earlier versions, afaik.
I suspect it's just display crap.
Alos, you can find a more recent kernel than that :) My bk tree is currently at 2.6.2-rc3-ben1
Ben.
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Quoted 20 lines · Mikolaj Krzewicki
Quoted 13 lines
Quoted 13 lines
hi all,
starting 2.6.0-test9-benh3 i noticed an increased system overhead on my ibook 500. Each process eats much more cpu time than in 2.4 or even vanilla 2.6.
gkrellm takes 1% instead of 0.1, xmms 5% instead of ~1% and so on. I benchmarked the lot with octave and it indeed runs ~10% slower on the benh kernels.
I would use vanilla 2.6.1 but it won't sleep/wake correctly (at all). All pre-2.6.0-test9-benh3 kernels worked fast but would't sleep/wake either. I tried to isolate the patch responsible but failed.
Has anybody come across similar behaviour?
greetings, MI switched to HZ=1000, the userland procps utilities (like ps and top) tend to not properly deal with that, at least earlier versions, afaik.
Quoted 6 lines
I suspect it's just display crap.
Alos, you can find a more recent kernel than that My bk tree is currently at 2.6.2-rc3-ben1
Ben.
i tested 2.6.1-benh1 now with HZ=100
the benchmark i used shows a speedup back to the values i'm used to. The benchmark itself is an octave script the execution time of which i tested with different kernels under similar circumstances, so here are the details:
2.6.1-benh1, HZ=1000: timing=36.5s
2.6.1-benh1, HZ=100 : timing=32s
this is with X running and a lot more processes(not running). the weird thing is it executes slightly faster (on average) with pbbuttonsd off.
The machine is g3 500 ibook, the octave script is:
tic;
a = abs(randn(1500, 1500)/10);
b = a';
c= a*b;
a = reshape(b, 750, 3000);
b = a';
timing=toc;so lots of system calls and cache flushing is in order.
Mikolaj.
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 21 lines · Mikolaj Krzewicki
the benchmark i used shows a speedup back to the values i'm used to. The benchmark itself is an octave script the execution time of which i tested with different kernels under similar circumstances, so here are the details:
2.6.1-benh1, HZ=1000: timing=36.5s
2.6.1-benh1, HZ=100 : timing=32sthis is with X running and a lot more processes(not running). the weird thing is it executes slightly faster (on average) with pbbuttonsd off.
The machine is g3 500 ibook, the octave script is:tic; a = abs(randn(1500, 1500)/10); b = a'; c= a*b; a = reshape(b, 750, 3000); b = a'; timing=toc;
Quoted 3 lines
so lots of system calls and cache flushing is in order.
Mikolaj.
Well, I don't knwo what the above means, I don't talk that language anyway :)
The fact that pbbuttons makes a difference makes me think the interrupt handling is taking way too much time on your setup, and pbbuttons is loading the machine with PMU interrupts...
Not sure if I can fix any of this at this point without doing a major rewrite of the exception handling code, I suspect those CPUs don't like running in real mode and our exception handling happens mostly in that mode in ppc32...
Ben.
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Quoted 8 lines · Benjamin Herrenschmidt
Quoted 3 lines · Mikolaj Krzewicki
2.6.1-benh1, HZ=1000: timing=36.5s
2.6.1-benh1, HZ=100 : timing=32sThe fact that pbbuttons makes a difference makes me think the interrupt handling is taking way too much time on your setup, and pbbuttons is loading the machine with PMU interrupts...
pbbuttons is loading the machine? What about all those extra decrementer ticks? I assume benchmarks showed no performance degradation on some class of ppc32; what was the slowest system tested? For that matter, couldn't this hurt 4xx, 8xx(x), as well?
Perhaps HZ should be user-configurable? Although that doesn't help distributions who'd like one kernel to work everywhere... How does Linux handle increased HZ on old i386 anyways?
Quoted 4 lines
Not sure if I can fix any of this at this point without doing a major rewrite of the exception handling code, I suspect those CPUs don't like running in real mode and our exception handling happens mostly in that mode in ppc32...
I didn't realize CPUs go slower in real mode..?
--
Hollis Blanchard
IBM Linux Technology Center
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Quoted 5 lines
pbbuttons is loading the machine? What about all those extra decrementer ticks? I assume benchmarks showed no performance degradation on some class of ppc32; what was the slowest system tested? For that matter, couldn't this hurt 4xx, 8xx(x), as well?
Heh, well, I did see any obvious degradation on my powerbook but I didn't benchmark much actually. Maybe I should, or make kernel HZ a config option.
Quoted 3 lines
Perhaps HZ should be user-configurable? Although that doesn't help distributions who'd like one kernel to work everywhere... How does Linux handle increased HZ on old i386 anyways?
I don't know :)
Quoted 6 lines
Quoted 4 lines
Not sure if I can fix any of this at this point without doing a major rewrite of the exception handling code, I suspect those CPUs don't like running in real mode and our exception handling happens mostly in that mode in ppc32...
I didn't realize CPUs go slower in real mode..?
It is the case with 970's and I think with IBM G3s, I'm not sure about Motorola G4s and other CPUs. In real mode, all memory is treated like it has the G bit set, preventing speculation (among others, I suppose prefetch gets killed too).
I's very visible on lmbench with a G5: The null syscall overhead of the ppc64 kernel is lower than the one of the ppc32 kernel despite actually running more code and saving more & bigger registers.
Ben.
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Quoted 15 lines · Benjamin Herrenschmidt
Quoted 4 lines · Mikolaj Krzewicki
hi all,
starting 2.6.0-test9-benh3 i noticed an increased system overhead on my ibook 500. Each process eats much more cpu time than in 2.4 or even vanilla 2.6.Quoted 8 lines
gkrellm takes 1% instead of 0.1, xmms 5% instead of ~1% and so on. I benchmarked the lot with octave and it indeed runs ~10% slower on the benh kernels.
I would use vanilla 2.6.1 but it won't sleep/wake correctly (at all). All pre-2.6.0-test9-benh3 kernels worked fast but would't sleep/wake either. I tried to isolate the patch responsible but failed.
Has anybody come across similar behaviour?
greetings, M
Quoted 4 lines
I switched to HZ=1000, the userland procps utilities (like ps and top) tend to not properly deal with that, at least earlier versions, afaik.
I suspect it's just display crap.
The other problem is that measuring CPU time just by sampling which process is running at timer interrupt is a crappy method bound to be inaccurate, espcially on very bursty workloads, or one which depend on other frequencies which beat with the timer.
Didn't somebody implemented CPU process accounting based on the timebase? Or was it for another architecture (maybe s390)?
Regards,
Gabriel
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/