Skip to contentSkip to conversation

big bw_pipe drop due to sched-domain patch?

DISCUSSION David Mosberger 2 participants 2 messages May 29, 2004

lore
Share & tools
lore
#0 lore raw

Today I noticed that going from 2.6.6 to 2.6.7-rc1 caused a big drop in LMbench2 bw_pipe throughput on a dual-CPU machine (2.6.7-rc1 shows 10 times lower bandwidth than 2.6.6). It appears that the drop is due to the two tasks being distributed across the two CPUs, rather than being kept on the same CPU. If I force the tasks to be pinned on a single CPU, e.g., like so:

	$ taskset 1 ./bw_pipe

then performance is "only" about 10% worse than with 2.6.6.

Is this kind of drop expected?

	--david
#1 lore raw
Quoted 7 lines · David Mosberger

Today I noticed that going from 2.6.6 to 2.6.7-rc1 caused a big drop in LMbench2 bw_pipe throughput on a dual-CPU machine (2.6.7-rc1 shows 10 times lower bandwidth than 2.6.6). It appears that the drop is due to the two tasks being distributed across the two CPUs, rather than being kept on the same CPU. If I force the tasks to be pinned on a single CPU, e.g., like so:

Quoted 6 lines
	$ taskset 1 ./bw_pipe

then performance is "only" about 10% worse than with 2.6.6.

Is this kind of drop expected?

(please CC Ingo for scheduler related stuff too)

Previously (ie. pre sched-domains), pipe based wakeups would unconditionally migrate the "wakee" onto the same CPU as the waker. Great for benchmarks, but possibly not really essential for real workloads, especially when tasks start getting pulled across NUMA nodes (I'd be happy to be shown otherwise though).

sched-domains introduces instead a "soft bias" that attempts to get wakers and wakees together. The concept is applied to _all_ types of wakeups, not just pipe based.

I think this is a slightly better scheme, but we could increase the aggressiveness of the bias if it is proven to be useful.

I don't know why your taskset performance is lower. taskset should completely take the scheduler out of the equation AFAIKS...