Skip to contentSkip to conversation

2.6.21.1 vs 2.6.21-sd046 vs 2.6.21-cfs-v6

ck Al Boldi 2 participants 7 messages May 2, 2007 – May 3, 2007

lore
Share & tools
lore

Al Boldi

author
#0 lore raw
Quoted 8 lines · Con Kolivas
Quoted 7 lines · Michael Gerdau

meanwhile I've redone my numbercrunching tests with the following
kernels: 2.6.21.1 (mainline)

    2.6.21-sd046
    2.6.21-cfs-v6

running on a dualcore x86_64.
[I will run the same test with 2.6.21.1-cfs-v7 over the next days, likely tonight]

:
:

Quoted 7 lines
Quoted 3 lines

However from these figures it seems as if sd does provide for the fairest (as in equal share for all) scheduling among the 3 schedulers tested.

Looks good, thanks. Ingo's been hard at work since then and has v8 out by now. SD has not changed so you wouldn't need to do the whole lot of tests on SD again unless you don't trust some of the results.

Well, I tried cfs-v8 and it still shows some nice regressions wrt mainline/sd. SD's nice-levels look rather solid, implying fairness.

Thanks!

--
Al

#1 lore raw
Quoted 3 lines · Con Kolivas
Quoted 3 lines

Looks good, thanks. Ingo's been hard at work since then and has v8 out by now. SD has not changed so you wouldn't need to do the whole lot of tests on SD again unless you don't trust some of the results.

Quoted 2 lines · Al Boldi

Well, I tried cfs-v8 and it still shows some nice regressions wrt mainline/sd. SD's nice-levels look rather solid, implying fairness.

That's odd. The ->load_weight changes should've improved that quite a bit. There may be something slightly off in how lag is computed, or maybe the O(n) lag issue Ying Tang spotted is biting you.

Also, I should say that the nice number affairs don't imply fairness per se. The way that works is that when tasks have "weights" (like nice levels in UNIX) the definition of fairness changes so that each task gets shares of CPU bandwidth proportional to its weight instead of one share for one task.

It takes a bit closer inspection than feel tests to see if weighted fairness is properly implemented. One thing to try is running a number of identical CPU hogs at the same time at different nice levels for a fixed period of time (e.g. 1 or 2 minutes) so they're in competition with each other and seeing what percent of the CPU each gets. From there you can figure out how many shares each is getting for its nice level. Trying different mixtures of nice levels and different numbers of tasks should give consistent results for the shares of CPU bandwidth the CPU hogs get for being at a particular nice level. A scheduler gets "bonus points" (i.e. is considered better at prioritizing) for the user being able to specify how the weightings come out. The finer-grained the control, the more bonus points.

Maybe con might want to take a stab at having users be able to specify the weights for each nice level individually.

CFS actually has a second set of weights for tasks, namely the timeslice for a given task. At the moment, they're all equal. It should be the case that the shorter the timeslice a given task has, the less latency it gets. So there is a fair amount of room for it to manuever with respect to feel tests. It really needs to be done numerically to get results we can be sure mean something.

The way this goes is task t_i gets a percent of the CPU p_i when the tasks t_1, t_2, ..., t_n are all competing, and task t_i has nice level n_i. The share corresponding to nice level n_i is then

          p_i

w_i = -------
sum p_j

One thing to check for is that if two tasks have the same nice level that their weights come out about equal. So for t_i and t_j, if n_i = n_j then you check that at least approximately, w_i = w_j, or even p_i = p_j, since we're not starting and stopping tasks in the midst of the test. Also, you can't simplify sum p_j to 1, since the set of tasks may not be the only things running.

The other thing to do is try a different number of tasks with a different mix of nice levels. The weight w_i for a given nice level n_i should be the same even in a different mix of tasks and nice levels if the nice levels are the same.

If this sounds too far out, there's nothing to worry about. You can just run the different numbers of tasks with different mixes of nice levels and post the %cpu numbers. Or if that's still a bit far out for you, a test that does all this is eventually going to get written.

-- wli

author
#2 lore raw
Quoted 5 lines · William Lee Irwin III
Quoted 3 lines · Con Kolivas
Quoted 3 lines

Looks good, thanks. Ingo's been hard at work since then and has v8 out by now. SD has not changed so you wouldn't need to do the whole lot of tests on SD again unless you don't trust some of the results.

Quoted 7 lines
Quoted 2 lines · Al Boldi

Well, I tried cfs-v8 and it still shows some nice regressions wrt mainline/sd. SD's nice-levels look rather solid, implying fairness.

That's odd. The ->load_weight changes should've improved that quite a bit. There may be something slightly off in how lag is computed, or maybe the O(n) lag issue Ying Tang spotted is biting you.

Is it not biting you too?

Quoted 5 lines

Also, I should say that the nice number affairs don't imply fairness per se. The way that works is that when tasks have "weights" (like nice levels in UNIX) the definition of fairness changes so that each task gets shares of CPU bandwidth proportional to its weight instead of one share for one task.

Ok, but you can easily expose scheduler unfairness by using nice levels as relative magnifiers; provided nice levels are implemented correctly.

Quoted 43 lines

It takes a bit closer inspection than feel tests to see if weighted fairness is properly implemented. One thing to try is running a number of identical CPU hogs at the same time at different nice levels for a fixed period of time (e.g. 1 or 2 minutes) so they're in competition with each other and seeing what percent of the CPU each gets. From there you can figure out how many shares each is getting for its nice level. Trying different mixtures of nice levels and different numbers of tasks should give consistent results for the shares of CPU bandwidth the CPU hogs get for being at a particular nice level. A scheduler gets "bonus points" (i.e. is considered better at prioritizing) for the user being able to specify how the weightings come out. The finer-grained the control, the more bonus points.

Maybe con might want to take a stab at having users be able to specify the weights for each nice level individually.

CFS actually has a second set of weights for tasks, namely the timeslice for a given task. At the moment, they're all equal. It should be the case that the shorter the timeslice a given task has, the less latency it gets. So there is a fair amount of room for it to manuever with respect to feel tests. It really needs to be done numerically to get results we can be sure mean something.

The way this goes is task t_i gets a percent of the CPU p_i when the tasks t_1, t_2, ..., t_n are all competing, and task t_i has nice level n_i. The share corresponding to nice level n_i is then

          p_i

w_i = -------
sum p_j

One thing to check for is that if two tasks have the same nice level that their weights come out about equal. So for t_i and t_j, if n_i = n_j then you check that at least approximately, w_i = w_j, or even p_i = p_j, since we're not starting and stopping tasks in the midst of the test. Also, you can't simplify sum p_j to 1, since the set of tasks may not be the only things running.

The other thing to do is try a different number of tasks with a different mix of nice levels. The weight w_i for a given nice level n_i should be the same even in a different mix of tasks and nice levels if the nice levels are the same.

Quoted 4 lines

If this sounds too far out, there's nothing to worry about. You can just run the different numbers of tasks with different mixes of nice levels and post the %cpu numbers. Or if that's still a bit far out for you, a test that does all this is eventually going to get written.

chew.c does exactly that, just make sure sched_granularity_ms >= 5,000,000.

Thanks!

--
Al

#3 lore raw
Quoted 3 lines · William Lee Irwin III
Quoted 3 lines

That's odd. The ->load_weight changes should've improved that quite a bit. There may be something slightly off in how lag is computed, or maybe the O(n) lag issue Ying Tang spotted is biting you.

Quoted 1 line · Al Boldi

Is it not biting you too?

I'm a kernel programmer. I'm not an objective tester.

It also happens to be the case that I personally have never encountered a performance problem with any of the schedulers, mainline included, on any system I use interactively. So my "user experience" is not valuable.

Quoted 5 lines · William Lee Irwin III
Quoted 5 lines

Also, I should say that the nice number affairs don't imply fairness per se. The way that works is that when tasks have "weights" (like nice levels in UNIX) the definition of fairness changes so that each task gets shares of CPU bandwidth proportional to its weight instead of one share for one task.

Quoted 2 lines · Al Boldi

Ok, but you can easily expose scheduler unfairness by using nice levels as relative magnifiers; provided nice levels are implemented correctly.

This doesn't really fit in with anything I'm aware of.

Quoted 8 lines · William Lee Irwin III
Quoted 8 lines

The other thing to do is try a different number of tasks with a different mix of nice levels. The weight w_i for a given nice level n_i should be the same even in a different mix of tasks and nice levels if the nice levels are the same.
If this sounds too far out, there's nothing to worry about. You can just run the different numbers of tasks with different mixes of nice levels and post the %cpu numbers. Or if that's still a bit far out for you, a test that does all this is eventually going to get written.

Quoted 1 line · Al Boldi

chew.c does exactly that, just make sure sched_granularity_ms >= 5,000,000.

Please post the source of chew.c

-- wli

author
#4 lore raw
Quoted 21 lines · William Lee Irwin III
Quoted 3 lines · William Lee Irwin III
Quoted 3 lines

That's odd. The ->load_weight changes should've improved that quite a bit. There may be something slightly off in how lag is computed, or maybe the O(n) lag issue Ying Tang spotted is biting you.

Quoted 1 line · Al Boldi

Is it not biting you too?

I'm a kernel programmer. I'm not an objective tester.

It also happens to be the case that I personally have never encountered a performance problem with any of the schedulers, mainline included, on any system I use interactively. So my "user experience" is not valuable.

Quoted 5 lines · William Lee Irwin III
Quoted 5 lines

Also, I should say that the nice number affairs don't imply fairness per se. The way that works is that when tasks have "weights" (like nice levels in UNIX) the definition of fairness changes so that each task gets shares of CPU bandwidth proportional to its weight instead of one share for one task.

Quoted 5 lines
Quoted 2 lines · Al Boldi

Ok, but you can easily expose scheduler unfairness by using nice levels as relative magnifiers; provided nice levels are implemented correctly.

This doesn't really fit in with anything I'm aware of.

You are not the first person that doesn't understand what I'm talking about. Don't worry about it.

Quoted 10 lines
Quoted 8 lines · William Lee Irwin III
Quoted 8 lines

The other thing to do is try a different number of tasks with a different mix of nice levels. The weight w_i for a given nice level n_i should be the same even in a different mix of tasks and nice levels if the nice levels are the same.
If this sounds too far out, there's nothing to worry about. You can just run the different numbers of tasks with different mixes of nice levels and post the %cpu numbers. Or if that's still a bit far out for you, a test that does all this is eventually going to get written.

Quoted 5 lines
Quoted 2 lines · Al Boldi

chew.c does exactly that, just make sure sched_granularity_ms >= 5,000,000.

Please post the source of chew.c

Attached.

Thanks!

--
Al

/*
* original idea by Chris Friesen. Thanks.
*/

#include <stdio.h>
#include <sys/time.h>
#include <sys/resource.h>

#define THRESHOLD_USEC 2000

unsigned long long stamp()

{
        struct timeval tv;
        gettimeofday(&tv, 0);
        return (unsigned long long) tv.tv_usec + ((unsigned long long) tv.tv_sec)*1000000;
}

int main()

{
        unsigned long long thresh_ticks = THRESHOLD_USEC;
        unsigned long long cur, last, start, act, delta;
        struct timespec ts;

        sched_rr_get_interval(0, &ts);
        printf("pid %d, prio %3d, interval of %d nsec\n", getpid(), getpriority(PRIO_PROCESS, 0), ts.tv_nsec);

        start = last = stamp();
        while(1) {
                cur = stamp();
                delta = cur-last;
                if (delta > thresh_ticks) {
			act = last - start;
                        printf("pid %d, prio %3d, out for %4llu ms, ran for %4llu ms, load %3llu%\n"
			, getpid(), getpriority(PRIO_PROCESS, 0), delta/1000, act/1000,(act*100)/(cur-start));
                        start = cur = stamp();
                }
                last = cur;
        }

        return 0;
}
#5 lore raw
Quoted 1 line

William Lee Irwin III wrote:

Quoted 2 lines · Al Boldi
        sched_rr_get_interval(0, &ts);
        printf("pid %d, prio %3d, interval of %d nsec\n", getpid(), getpriority(PRIO_PROCESS, 0), ts.tv_nsec);

Oh dear. What are you trying to figure out from the task's timeslice? That's not even meaningful in cfs.

Quoted 7 lines · Al Boldi
        start = last = stamp();
        while(1) {
                cur = stamp();
                delta = cur-last;
                if (delta > thresh_ticks) {
			act = last - start;
                        printf("pid %d, prio %3d, out for %4llu ms, ran for %4llu ms, load %3llu%\n"
Quoted 8 lines
			, getpid(), getpriority(PRIO_PROCESS, 0), delta/1000, act/1000,(act*100)/(cur-start));
                        start = cur = stamp();
                }
                last = cur;
        }

        return 0;
}

This is looking for scheduling latencies, which are necessarily O(tasks). This is not the way to do it.

-- wli

author
#6 lore raw
Quoted 6 lines · William Lee Irwin III
Quoted 3 lines · Al Boldi

sched_rr_get_interval(0, &ts);
printf("pid %d, prio %3d, interval of %d nsec\n", getpid(),
getpriority(PRIO_PROCESS, 0), ts.tv_nsec);

Oh dear. What are you trying to figure out from the task's timeslice?

chew.c is general purpose. It's not specific to cfs.

Quoted 16 lines

That's not even meaningful in cfs.

Quoted 13 lines · Al Boldi
        start = last = stamp();
        while(1) {
                cur = stamp();
                delta = cur-last;
                if (delta > thresh_ticks) {
			act = last - start;
                        printf("pid %d, prio %3d, out for %4llu ms, ran
for %4llu ms, load %3llu%\n" , getpid(), getpriority(PRIO_PROCESS, 0),
delta/1000, act/1000,(act*100)/(cur-start)); start = cur = stamp();
                }
                last = cur;
        }
Quoted 4 lines
Quoted 2 lines
        return 0;
}

This is looking for scheduling latencies, which are necessarily O(tasks).

This exposes the actual proc latency/waittime and runtime and cpu load irrespective of being O(tasks) or not.

Quoted 1 line

This is not the way to do it.

You'll need to boot into /bin/sh and run two or more chew.c's to see any reasonable results.

Then tell me whether this is the way to do it or not.

Thanks!

--
Al