2.6.22.2-cfs-v19.1 stops honouring nice 3 after some hours
Hi Ingo,
I played a bit with renicing my (mostly) CPU bound jobs and found somewhat strange behaviour.
I'm on 2.6.22.2-cfs-v19.1 -- I can't boot into 2.6.23-rc3 as part of my daily work because my NVIDIA driver doesn't yet work with it though I could try to do some X-less testing if need be.
I'm running 4 CPU bound jobs, 2 of which are more important to me. I therefor reniced the other 2 to level 3 and the CPU percentages were roughly 66/66/33/33 on my Dualcore system. So far all fine.
Some 6 hours later I rechecked with top -d 60 and got the output below:
19629 mgd 20 0 334m 263m 2700 R 49 8.1 69:16.45 perl 29668 mgd 20 0 396m 262m 2700 R 49 8.0 112:17.21 perl 6933 mgd 23 3 96576 21m 3672 R 47 0.7 46:38.83 perl 14162 mgd 23 3 165m 28m 3672 R 47 0.9 194:55.31 perl
I reniced to 0:
19629 mgd 20 0 334m 263m 2700 R 48 8.1 71:02.86 perl 6933 mgd 20 0 96652 21m 3672 R 48 0.7 48:23.41 perl 14162 mgd 20 0 165m 28m 3676 R 48 0.9 196:39.81 perl 29668 mgd 20 0 396m 262m 2700 R 48 8.0 114:03.56 perl
...and back to 3 with no noticeable change.
[at this point job 14162 finished and was replaced by an similar job 22558]
I then decided to do some statistics and reniced to 1:
29668 mgd 20 0 396m 262m 2700 R 48 8.0 115:24.12 perl 19629 mgd 20 0 334m 263m 2700 R 48 8.1 72:23.36 perl 22558 mgd 21 1 92184 17m 3672 R 48 0.5 0:56.81 perl 6933 mgd 21 1 96576 21m 3672 R 48 0.7 49:54.12 perl
Now I reniced to 2 and without knowing why, the ratios seem to be back to where I want them.
29668 mgd 20 0 396m 262m 2700 R 59 8.0 116:33.98 perl 19629 mgd 20 0 334m 264m 2700 R 58 8.1 73:31.49 perl 22558 mgd 22 2 92220 17m 3672 R 37 0.5 1:46.99 perl 6933 mgd 22 2 96688 21m 3672 R 36 0.7 50:44.77 perl
Renicing to 3 again gives me what I want:
29668 mgd 20 0 397m 262m 2700 R 65 8.0 117:25.93 perl 19629 mgd 20 0 334m 264m 2700 R 63 8.1 74:21.84 perl 22558 mgd 23 3 92220 17m 3672 R 32 0.5 2:13.83 perl 6933 mgd 23 3 96652 21m 3672 R 32 0.7 51:11.24 perl
I don't know why I had the very first output above though.
Best,
Michael
Signature · 5 lines
Technosis GmbH, Geschäftsführer: Heiko Gerdau, Tobias Dittmar Sitz Hamburg; HRB 89145 Amtsgericht Hamburg Vote against SPAM - see http://www.politik-digital.de/spam/ Michael Gerdau email: [email protected] GPG-keys available on request or at public keyserver
Quoted 8 lines · Michael Gerdau
Hi Ingo,
I played a bit with renicing my (mostly) CPU bound jobs and found somewhat strange behaviour.
I'm on 2.6.22.2-cfs-v19.1 -- I can't boot into 2.6.23-rc3 as part of my daily work because my NVIDIA driver doesn't yet work with it though I could try to do some X-less testing if need be.
Please do try, a lot has changed since v19.1.
FWIW the nv driver lets one do most desktop-ish things without problems, or so I've been told - just no 3d.
Quoted 8 lines
Quoted 3 lines
I'm on 2.6.22.2-cfs-v19.1 -- I can't boot into 2.6.23-rc3 as part of my daily work because my NVIDIA driver doesn't yet work with it though I could try to do some X-less testing if need be.
Please do try, a lot has changed since v19.1.
FWIW the nv driver lets one do most desktop-ish things without problems, or so I've been told - just no 3d.
Damien Wyart was kind enough to point me to a patch that solved the compile problem of the nv driver and I now have a ready-to-boot-into 2.6.23-rc3 lying around.
However due to other time constraints this will have to wait until tomorrow.
Anyway, observing the currently running CPU bound jobs and comparing their progress (as documented by their logfiles) with the CPU % one of the jobs receives (as documented by top -d 60) it seems as if this is not so much a scheduling problem but an accounting issue.
FWIW the jobs doing basically the same work are more or less head to head after 12+ hours of computation while according to top one of them is reported to receive roughly twice the CPU of the other.
I will retry with 2.6.23-rc3 tomorrow and over the weekend to see whether the problem persists.
Best wishes,
Michael
Signature · 4 lines
Vote against SPAM - see http://www.politik-digital.de/spam/ !tagline Michael Gerdau email: [email protected] GPG-keys available on request or at public keyserver
Quoted 8 lines
Quoted 6 lines
I played a bit with renicing my (mostly) CPU bound jobs and found somewhat strange behaviour.
I'm on 2.6.22.2-cfs-v19.1 -- I can't boot into 2.6.23-rc3 as part of my daily work because my NVIDIA driver doesn't yet work with it though I could try to do some X-less testing if need be.
Please do try, a lot has changed since v19.1.
Meanwhile I've done some testing on 2.6.23-rc3.
The good news is that the strange behaviour w/r to nice I had seen on 2.6.22.3-cfs-v19.1 is no longer there.
Long running CPU bound jobs at different nice levels do indeed progress (roughly) according to the %CPU figures as reported by top now.
However what I find somewhat unexpected is the IMO huge variation in %CPU as observed even with 'top -d 60'. I've attached a log showing 5 CPU bound tasks observed for an our with 'top -b -d 60 -n 60'.
Jobs 5897, 5928 and 5977 do perform essentially the same tasks. Jobs 17085 and 21492 do perform essentially the same task but differ from the other 3 task.
I would have expected Jobs 5897 and 5928 to differ by at most one percent point when measured for 60 seconds.
Best,
Michael
Signature · 5 lines
Technosis GmbH, Geschäftsführer: Heiko Gerdau, Tobias Dittmar Sitz Hamburg; HRB 89145 Amtsgericht Hamburg Vote against SPAM - see http://www.politik-digital.de/spam/ Michael Gerdau email: [email protected] GPG-keys available on request or at public keyserver