Skip to contentSkip to conversation

Scheduling tests on IPC methods, fc6, sd0.48, cfs12

DISCUSSION Bill Davidsen 3 participants 5 messages May 17, 2007 – May 22, 2007

lore
Share & tools
lore
#1 lore raw
Quoted 5 lines · Bill Davidsen

I have posted the results of my initial testing, measuring IPC rates using various schedulers under no load, limited nice load, and heavy load at nice 0.

http://www.tmr.com/~davidsen/ctxbench_testing.html

nice! For this to become really representative though i'd like to ask for a real workload function to be used after the task gets the lock/message. The reason is that there is an inherent balancing conflict in this area: should the scheduler 'spread' tasks to other CPUs or not? In general, for all workloads that matter, the answer is almost always:
'yes, it should'.

But in your ctxbench results the work a task performs after doing IPC is not reflected (the benchmark goes about to do the next IPC - hence penalizing scheduling strategies that move tasks to other CPUs) - hence the bonus of a scheduler properly spreading out tasks is not measured fairly. A real-life IPC workload is rarely just about messaging around (a single task could do that itself) - some real workload function is used. You can see this effect yourself: do a "taskset -p 01 $$" before running ctxbench and you'll see the numbers improve significantly on all of the schedulers.

As a solution i'd suggest to add a workload function with a 100 or 200 usecs (or larger) cost (as a fixed-length loop or something like that) so that the 'spreading' effect/benefit gets measured fairly too.

Ingo

#3 lore raw
In reply to Bill Davidsen
Quoted 4 lines · Bill Davidsen

I have posted the results of my initial testing, measuring IPC rates using various schedulers under no load, limited nice load, and heavy load at nice 0.
http://www.tmr.com/~davidsen/ctxbench_testing.html

Kernel compiles are not how to stress these. The way to stress them is to have multiple simultaneous independent chains of communicators and deeper chains of communicators.

Kernel compiles are little but background cpu/memory load for these sorts of tests. Something expected to have some sort of mutual interference depending on quality of implementation would be a better sort of competing load, one vastly more reflective of real workloads. For instance, another set of processes communicating using the same primitive.

Perhaps best of all would be a macrobenchmark utilizing a variety of the primitives under consideration. Unsurprisingly, major commercial databases do so for major benchmarks.

-- wli

#2 lore raw
In reply to Ingo Molnar
Quoted 8 lines · Ingo Molnar
Quoted 3 lines · Bill Davidsen

I have posted the results of my initial testing, measuring IPC rates using various schedulers under no load, limited nice load, and heavy load at nice 0.

http://www.tmr.com/~davidsen/ctxbench_testing.html

nice! For this to become really representative though i'd like to ask for a real workload function to be used after the task gets the lock/message. The reason is that there is an inherent balancing conflict in this area: should the scheduler 'spread' tasks to other CPUs or not? In general, for all workloads that matter, the answer is almost always: 'yes, it should'.

Added to the short to-do list. Note that this was originally simply a check to see which IPC works best (or at all) in an o/s. It has been useful for some other things, and an option for work will be forthcoming.

Quoted 4 lines

But in your ctxbench results the work a task performs after doing IPC is not reflected (the benchmark goes about to do the next IPC - hence penalizing scheduling strategies that move tasks to other CPUs) - hence the bonus of a scheduler properly spreading out tasks is not measured fairly. A real-life IPC workload is rarely just about messaging around (a single task could do that itself) - some real workload function is used. You can see this effect yourself: do a "taskset -p 01 $$" before running ctxbench and you'll see the numbers improve significantly on all of the schedulers.

As a solution i'd suggest to add a workload function with a 100 or 200 usecs (or larger) cost (as a fixed-length loop or something like that) so that the 'spreading' effect/benefit gets measured fairly too.

Can do.

Signature · 3 lines
Bill Davidsen <[email protected]>
  "We have more to fear from the bungling of the incompetent than from
the machinations of the wicked."  - from Slashdot
#4 lore raw
In reply to William Lee Irwin III
Quoted 10 lines · William Lee Irwin III
Quoted 2 lines · Bill Davidsen

I have posted the results of my initial testing, measuring IPC rates using various schedulers under no load, limited nice load, and heavy load at nice 0.
http://www.tmr.com/~davidsen/ctxbench_testing.html

Kernel compiles are not how to stress these. The way to stress them is to have multiple simultaneous independent chains of communicators and deeper chains of communicators.

Kernel compiles are little but background cpu/memory load for these sorts of tests.

Just so. What is being quantified is the rate of slowdown due to external load. I would hope that each IPC method would slow by some similar factor.

Quoted 6 lines

... Something expected to have some sort of mutual
interference depending on quality of implementation would be a better sort of competing load, one vastly more reflective of real workloads. For instance, another set of processes communicating using the same primitive.

The original intent was purely to measure IPC speed under no load conditions, since fairness is in vogue I also attempted to look for surprising behavior. Corresponding values under equal load may be useful in relation to one another, but this isn't (and hopefully doesn't claim to be) a benchmark. It may or may not be useful viewed in that light, but that's not the target.

Quoted 4 lines

Perhaps best of all would be a macrobenchmark utilizing a variety of the primitives under consideration. Unsurprisingly, major commercial databases do so for major benchmarks.

And that's a very good point, either multiple copies or more forked processes might be useful, and I do intend to add threaded tests on the next upgrade, but perhaps a whole new code might be better for generating the load you suggest.

Signature · 3 lines
Bill Davidsen <[email protected]>
  "We have more to fear from the bungling of the incompetent than from
the machinations of the wicked."  - from Slashdot