Skip to contentSkip to conversation

scheduler vs hardware? (was Re: another i7 (linux) bug?)

DISCUSSION Alex Chiang 6 participants 10 messages Jan 21, 2010 – Jan 23, 2010 1 message not in the archive

lore
Share & tools
lore
#1 lore raw
In reply to a message not in the archive

Scheduler folks cc'ed.

* Luca Zini <[email protected]>:

Quoted 3 lines

Dear all,
I find your addresses in the thread
(http://bugzilla.kernel.org/show_bug.cgi?id=15064) that I followed.

In general, it's better to send mail to the mailing lists in case individual developers don't have time/expertise to help you.

Which kernel are you using?

This may be an issue for the scheduler folks since in your experiment, the only thing you're changing are the nice levels, not hardware.

Quoted 21 lines

I had the same problem that now seems to be solved. There is also another peculiarity in my system:

if I run a cpu intensive process with the lowest priority (19 from man nice) I obtain much better performance that with the highest priority available (-20 from man nice).

For example the same file is processed by lame in 8.7 seconds at the lowest priority, and in 12 seconds at the highest priority. Before posting a bug I wold like to understand if this is a problem related to the i7 mobile (my processor is a i7 Q720).

As far as I tested on the same laptop series (dell studio 15), with the same kernel this problem does not exists.

Can you test your i7 mobile system to help me to understand if it is related to the processor?

Regards, Luca Zini

#2 lore raw
Quoted 4 lines

if I run a cpu intensive process with the lowest priority (19 from man nice) I obtain much better performance that with the highest priority available (-20 from man nice).

Quoted 8 lines

For example the same file is processed by lame in 8.7 seconds at the lowest priority, and in 12 seconds at the highest priority. Before posting a bug I wold like to understand if this is a problem related to the i7 mobile (my processor is a i7 Q720).

As far as I tested on the same laptop series (dell studio 15), with the same kernel this problem does not exists.

So you only see this on the i7. That's odd. Can you try 33-rc5?

Posting a reliable reproducer would be nice. It'd also be nice to see what all is running when you see this, and where.

-Mike

#3 lore raw
Quoted 10 lines · Mike Galbraith
Quoted 10 lines

if I run a cpu intensive process with the lowest priority (19 from man nice) I obtain much better performance that with the highest priority available (-20 from man nice).

For example the same file is processed by lame in 8.7 seconds at the lowest priority, and in 12 seconds at the highest priority. Before posting a bug I wold like to understand if this is a problem related to the i7 mobile (my processor is a i7 Q720).

Quoted 7 lines
Quoted 2 lines

As far as I tested on the same laptop series (dell studio 15), with the same kernel this problem does not exists.

So you only see this on the i7. That's odd. Can you try 33-rc5?

Posting a reliable reproducer would be nice. It'd also be nice to see what all is running when you see this, and where.

Using a sample from: http://lame.sourceforge.net/quality.php

My laptop does:

# time nice -n 19 lame -b 256 -V0 -h youcantdothat.wav - > /dev/null
LAME 3.98.2 64bits (http://www.mp3dev.org/)
Using polyphase lowpass filter, transition band: 19383 Hz - 19916 Hz
Encoding youcantdothat.wav to <stdout>
Encoding as 44.1 kHz j-stereo MPEG-1 Layer III VBR(q=0)
    Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA
  1147/1147  (100%)|    0:03/    0:03|    0:03/    0:03|   9.3341x|    0:00
 32 [   1] *
256 [1009] %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
320 [ 137] %%%%%%%%%
----------------------------------------------------------------------------
   kbps        LR    MS  %     long switch short %
  263.4       99.1   0.9        90.5   5.5   4.0
real    0m3.273s
user    0m3.217s
sys     0m0.022s
# time nice -n 0 lame -b 256 -V0 -h youcantdothat.wav - > /dev/null 
LAME 3.98.2 64bits (http://www.mp3dev.org/)
Using polyphase lowpass filter, transition band: 19383 Hz - 19916 Hz
Encoding youcantdothat.wav to <stdout>
Encoding as 44.1 kHz j-stereo MPEG-1 Layer III VBR(q=0)
    Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA
  1147/1147  (100%)|    0:01/    0:01|    0:01/    0:01|   27.239x|    0:00
 32 [   1] *
256 [1009] %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
320 [ 137] %%%%%%%%%
----------------------------------------------------------------------------
   kbps        LR    MS  %     long switch short %
  263.4       99.1   0.9        90.5   5.5   4.0
real    0m1.121s
user    0m1.102s
sys     0m0.013s
# time nice -n -20 lame -b 256 -V0 -h youcantdothat.wav - > /dev/null
LAME 3.98.2 64bits (http://www.mp3dev.org/)
Using polyphase lowpass filter, transition band: 19383 Hz - 19916 Hz
Encoding youcantdothat.wav to <stdout>
Encoding as 44.1 kHz j-stereo MPEG-1 Layer III VBR(q=0)
    Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA
  1147/1147  (100%)|    0:01/    0:01|    0:01/    0:01|   27.239x|    0:00
 32 [   1] *
256 [1009] %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
320 [ 137] %%%%%%%%%
----------------------------------------------------------------------------
   kbps        LR    MS  %     long switch short %
  263.4       99.1   0.9        90.5   5.5   4.0
real    0m1.112s
user    0m1.093s
sys     0m0.018s

On a Nehalem class server machine it does:

# time nice -n 19 lame -b 256 -V0 -h youcantdothat.wav - > /dev/null
LAME 3.98.2 64bits (http://www.mp3dev.org/)
Using polyphase lowpass filter, transition band: 19383 Hz - 19916 Hz
Encoding youcantdothat.wav to <stdout>
Encoding as 44.1 kHz j-stereo MPEG-1 Layer III VBR(q=0)
    Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA
  1147/1147  (100%)|    0:00/    0:00|    0:00/    0:00|   32.926x|    0:00
 32 [   1] *
256 [1009] %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
320 [ 137] %%%%%%%%%
----------------------------------------------------------------------------
   kbps        LR    MS  %     long switch short %
  263.4       99.1   0.9        90.5   5.5   4.0
real    0m0.932s
user    0m0.917s
sys     0m0.005s
# time nice -n 0 lame -b 256 -V0 -h youcantdothat.wav - > /dev/null
LAME 3.98.2 64bits (http://www.mp3dev.org/)
Using polyphase lowpass filter, transition band: 19383 Hz - 19916 Hz
Encoding youcantdothat.wav to <stdout>
Encoding as 44.1 kHz j-stereo MPEG-1 Layer III VBR(q=0)
    Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA
  1147/1147  (100%)|    0:00/    0:00|    0:00/    0:00|   32.568x|    0:00
 32 [   1] *
256 [1009] %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
320 [ 137] %%%%%%%%%
----------------------------------------------------------------------------
   kbps        LR    MS  %     long switch short %
  263.4       99.1   0.9        90.5   5.5   4.0
real    0m0.927s
user    0m0.922s
sys     0m0.003s
# time nice -n -20 lame -b 256 -V0 -h youcantdothat.wav - > /dev/null
LAME 3.98.2 64bits (http://www.mp3dev.org/)
Using polyphase lowpass filter, transition band: 19383 Hz - 19916 Hz
Encoding youcantdothat.wav to <stdout>
Encoding as 44.1 kHz j-stereo MPEG-1 Layer III VBR(q=0)
    Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA
  1147/1147  (100%)|    0:00/    0:00|    0:00/    0:00|   32.926x|    0:00
 32 [   1] *
256 [1009] %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
320 [ 137] %%%%%%%%%
----------------------------------------------------------------------------
   kbps        LR    MS  %     long switch short %
  263.4       99.1   0.9        90.5   5.5   4.0
real    0m0.919s
user    0m0.914s
sys     0m0.005s
#4 lore raw
Quoted 63 lines · Peter Zijlstra
Quoted 10 lines · Mike Galbraith
Quoted 10 lines

if I run a cpu intensive process with the lowest priority (19 from man nice) I obtain much better performance that with the highest priority available (-20 from man nice).

For example the same file is processed by lame in 8.7 seconds at the lowest priority, and in 12 seconds at the highest priority. Before posting a bug I wold like to understand if this is a problem related to the i7 mobile (my processor is a i7 Q720).

Quoted 7 lines
Quoted 2 lines

As far as I tested on the same laptop series (dell studio 15), with the same kernel this problem does not exists.

So you only see this on the i7. That's odd. Can you try 33-rc5?

Posting a reliable reproducer would be nice. It'd also be nice to see what all is running when you see this, and where.

Using a sample from: http://lame.sourceforge.net/quality.php

My laptop does:

# time nice -n 19 lame -b 256 -V0 -h youcantdothat.wav - > /dev/null
real    0m3.273s
user    0m3.217s
sys     0m0.022s
# time nice -n 0 lame -b 256 -V0 -h youcantdothat.wav - > /dev/null 
real    0m1.121s
user    0m1.102s
sys     0m0.013s
# time nice -n -20 lame -b 256 -V0 -h youcantdothat.wav - > /dev/null
real    0m1.112s
user    0m1.093s
sys     0m0.018s

On a Nehalem class server machine it does:

# time nice -n 19 lame -b 256 -V0 -h youcantdothat.wav - > /dev/null
real    0m0.932s
user    0m0.917s
sys     0m0.005s
# time nice -n 0 lame -b 256 -V0 -h youcantdothat.wav - > /dev/null
real    0m0.927s
user    0m0.922s
sys     0m0.003s
Quoted 5 lines
# time nice -n -20 lame -b 256 -V0 -h youcantdothat.wav - > /dev/null
real    0m0.919s
user    0m0.914s
sys     0m0.005s

Weird. Here there is zip squat difference, as expected with 1 thread.

time nice -n 19 lame -b 256 -V0 -h youcantdothat.wav - > /dev/null

real    0m0.912s
user    0m0.908s
sys     0m0.000s

time nice -n 0 lame -b 256 -V0 -h youcantdothat.wav - > /dev/null

real    0m0.912s
user    0m0.904s
sys     0m0.004s

time nice -n -20 lame -b 256 -V0 -h youcantdothat.wav - > /dev/null

real    0m0.912s
user    0m0.904s
sys     0m0.004s

(bah, who needs a nehalem;)

-Mike

#5 lore raw
In reply to Peter Zijlstra
Quoted 10 lines · Peter Zijlstra

My laptop does:

# time nice -n 19 lame -b 256 -V0 -h youcantdothat.wav - > /dev/null
real    0m3.273s
user    0m3.217s
sys     0m0.022s
Quoted 5 lines
# time nice -n 0 lame -b 256 -V0 -h youcantdothat.wav - > /dev/null
real    0m1.121s
user    0m1.102s
sys     0m0.013s

I don't see any way that can be scheduler (knocks wood). Could it be cpufreq? IIRC there is or was an ignore nice gizmo in there.

-Mike

#6 lore raw
Quoted 12 lines · Mike Galbraith
Quoted 10 lines · Peter Zijlstra

My laptop does:

# time nice -n 19 lame -b 256 -V0 -h youcantdothat.wav - > /dev/null
real    0m3.273s
user    0m3.217s
sys     0m0.022s
Quoted 8 lines
Quoted 5 lines
# time nice -n 0 lame -b 256 -V0 -h youcantdothat.wav - > /dev/null
real    0m1.121s
user    0m1.102s
sys     0m0.013s

I don't see any way that can be scheduler (knocks wood). Could it be cpufreq? IIRC there is or was an ignore nice gizmo in there.

That's a laptop with full desktop bloat, I'm sure either firefox or evo farted, or both.

#7 lore raw

First of all sorry for misunderstanding with Alex Chiang, I was trying to collect some data from other i7 mobile users to try to isolate the problem before posting.

I tried different things: I disabled speedstep from the bios and the results where more sensible (higher priority lower execution time). The same thing happens if I disable throttling by software selecting "aggressive powersave" settings.

So I suppose that is something related directly or indirectly to frequency scaling or turbo boost (unfortunately I have no bios option to disable only turbo boost)
Here are the results of the same test of Peter Zijlstra

time sudo nice -n 19 lame -b 256 -V0 -h youcantdothat.wav 2&> /dev/null

real    0m1.105s
user    0m1.090s
sys     0m0.010s

time sudo nice -n 0 lame -b 256 -V0 -h youcantdothat.wav 2&> /dev/null

real    0m1.108s
user    0m1.100s
sys     0m0.010s

time sudo nice -n -20 lame -b 256 -V0 -h youcantdothat.wav 2&> /dev/null

real    0m1.354s
user    0m1.330s
sys     0m0.000s

They are almost the same results that I obtained before. Looking at top there are no other process that is using the cpu (main are Xorg and kopete).

I tested this on 2.6.31-17-server (ubuntu 9.10) 2.6.31-17-generic (ubuntu 9.10) and the default kernel of fedora 12 (live cd).
As soon as possible I'll try 33-rc5 as requested by Mike Galbraith (now I can't shut down the computer).

Ok, a last minute update!
I slightly modified i7z to save a log to disk. If i7z output are reliable I suppose that it can show is the problem:
with nice set to 19 a processor 3 reach the maximum speed, with nice set to -20 its maximum value is 300-400 mhz under the maximum value. (I attach the two logs named as the nice level used obtained running lame on a bigger file) Please note that logs may have some value read before and/or after the start/end of the running process (lame).

The next update will be the results with 33-rc5 kernel.

Luca

	Processor 0:  1062.13 (7.99x)	   1	99.7	   0	98.5
	Processor 1:  1246.75 (9.37x)	   1	99.8	   0	  99
	Processor 2:  1113.41 (8.37x)	   1	99.8	   0	98.4
	Processor 3:  1477.30 (11.11x)	4.67	95.7	   0	85.7
	Processor 0:  1109.70 (8.34x)	   1	99.7	2.93	95.8
	Processor 1:  2316.51 (17.42x)	 5.5	  92	   0	91.7
	Processor 2:  1114.96 (8.38x)	   1	99.8	14.9	83.7
	Processor 3:  1225.48 (9.21x)	9.95	92.4	1.33	67.6
	Processor 0:  1877.15 (14.11x)	   1	99.8	29.6	69.9
	Processor 1:  1914.75 (14.40x)	   1	99.2	   1	98.1
	Processor 2:  1955.22 (14.70x)	   1	99.8	   0	99.4
	Processor 3:  2402.49 (18.06x)	38.8	41.6	   0	   0
	Processor 0:  2114.28 (15.90x)	  12	84.1	   0	35.4
	Processor 1:  1528.62 (11.49x)	   1	99.8	25.4	73.2
	Processor 2:  1582.10 (11.90x)	   1	99.9	74.2	25.2
	Processor 3:  2359.66 (17.74x)	14.6	78.3	18.6	  56
	Processor 0:  2242.78 (16.86x)	16.3	77.1	   0	   0
	Processor 1:  1591.62 (11.97x)	   1	99.9	87.9	10.9
	Processor 2:  1885.84 (14.18x)	   1	 100	99.2	   1
	Processor 3:  2035.81 (15.31x)	1.77	97.7	  15	75.4
	Processor 0:  2280.59 (17.15x)	18.3	73.8	   1	70.4
	Processor 1:  1906.72 (14.34x)	   1	99.7	26.9	  72
	Processor 2:  1778.56 (13.37x)	   1	99.9	70.9	28.6
	Processor 3:  1077.58 (8.10x)	8.56	94.2	3.66	4.79
	Processor 0:  2009.19 (15.11x)	7.07	91.1	   1	88.3
	Processor 1:  1708.96 (12.85x)	   1	99.8	37.3	61.5
	Processor 2:  1673.01 (12.58x)	   1	99.9	13.4	85.9
	Processor 3:  1421.09 (10.68x)	12.1	89.2	   1	21.8
	Processor 0:  935.30 (7.03x)	   1	99.8	   0	73.4
	Processor 1:  938.46 (7.06x)	   1	99.9	   0	98.3
	Processor 2:  932.33 (7.01x)	   1	99.9	   0	97.7
	Processor 3:  1017.69 (7.65x)	6.09	96.1	   0	92.1
	Processor 0:  1040.73 (7.83x)	   1	99.8	   0	99.1
	Processor 1:  1062.37 (7.99x)	   1	99.7	   0	96.1
	Processor 2:  1047.94 (7.88x)	   1	99.8	   1	97.7
	Processor 3:  1941.75 (14.60x)	11.1	86.5	2.98	81.5
	Processor 0:  1940.64 (14.59x)	   1	99.5	   0	72.7
	Processor 1:  1600.11 (12.03x)	   1	99.8	11.5	  86
	Processor 2:  2141.58 (16.10x)	   1	 100	99.6	   1
	Processor 3:  2760.11 (20.75x)	83.2	   0	4.33	11.6
	Processor 0:  2331.59 (17.53x)	   1	99.4	5.98	  93
	Processor 1:  2287.80 (17.20x)	   1	99.8	   0	97.6
	Processor 2:  2122.39 (15.96x)	   1	 100	88.5	11.1
	Processor 3:  2774.36 (20.86x)	99.9	   0	   0	   0
	Processor 0:  2322.64 (17.46x)	   1	99.5	   1	98.6
	Processor 1:  2275.29 (17.11x)	   1	99.8	   1	97.9
	Processor 2:  2268.43 (17.06x)	   1	99.8	   0	99.5
	Processor 3:  2779.36 (20.90x)	99.9	   0	   0	   0
	Processor 0:  2277.58 (17.12x)	   1	99.8	   0	99.6
	Processor 1:  2372.66 (17.84x)	   1	99.5	   0	99.2
	Processor 2:  2346.90 (17.65x)	   1	99.8	   0	99.5
	Processor 3:  2783.34 (20.93x)	99.9	   0	   0	   0
#8 lore raw
Quoted 1 line · Luca Zini

with nice set to 19 a processor 3 reach the maximum speed, with nice

set to

Quoted 1 line

-20 its maximum value is 300-400 mhz under the maximum value.

I wonder if at lower nice levels the timing on something changes, forcing multiple cores to be active just often enough to keep the i7 from doing the turbo boost thing, whereas with higher nice levels everything is serialized allowing the boost to be enabled.

Chris

#10 lore raw
In reply to Mike Galbraith

vanilla 33-rc5 runs ok:
with nice set to -20

real    0m8.649s
user    0m8.600s
sys     0m0.030s

with nice set to 19:

real    0m8.658s
user    0m8.600s
sys     0m0.040s

thank you!
Luca

Quoted 20 lines · Mike Galbraith
Quoted 4 lines

if I run a cpu intensive process with the lowest priority (19 from man nice) I obtain much better performance that with the highest priority available (-20 from man nice).

Quoted 8 lines

For example the same file is processed by lame in 8.7 seconds at the lowest priority, and in 12 seconds at the highest priority. Before posting a bug I wold like to understand if this is a problem related to the i7 mobile (my processor is a i7 Q720).

As far as I tested on the same laptop series (dell studio 15), with the same kernel this problem does not exists.

So you only see this on the i7. That's odd. Can you try 33-rc5?

Posting a reliable reproducer would be nice. It'd also be nice to see what all is running when you see this, and where.

-Mike

#9 lore raw
In reply to Peter Zijlstra
Quoted 19 lines · Peter Zijlstra
Quoted 10 lines · Mike Galbraith
Quoted 10 lines

if I run a cpu intensive process with the lowest priority (19 from man nice) I obtain much better performance that with the highest priority available (-20 from man nice).

For example the same file is processed by lame in 8.7 seconds at the lowest priority, and in 12 seconds at the highest priority. Before posting a bug I wold like to understand if this is a problem related to the i7 mobile (my processor is a i7 Q720).

Quoted 7 lines
Quoted 2 lines

As far as I tested on the same laptop series (dell studio 15), with the same kernel this problem does not exists.

So you only see this on the i7. That's odd. Can you try 33-rc5?

Posting a reliable reproducer would be nice. It'd also be nice to see what all is running when you see this, and where.

Quoted 6 lines

Using a sample from: http://lame.sourceforge.net/quality.php

My laptop does:

# time nice -n 19 lame -b 256 -V0 -h youcantdothat.wav - > /dev/null

A 'perf stat --repeat 3' run would tell us more about the underlying reasons i think.

Ingo