Skip to contentSkip to conversation

volano ~30% regression with 2.6.33-rc1 & -rc2

DISCUSSION Lin Ming 4 participants 10 messages Jan 4, 2010 – Jan 21, 2010

lore
Share & tools
lore

Lin Ming

author
#0 lore raw

Mike & Peter,

Compared with 2.6.32, volano has ~30% regression with 2.6.33-rc1 & -rc2. Testing machine: Tigerton Xeon, 16cpus(4P/4Core), 16G memory

Bisect to below commit,

commit a1f84a3ab8e002159498814eaa7e48c33752b04b

Author: Mike Galbraith <[email protected]>
Date:   Tue Oct 27 15:35:38 2009 +0100

sched: Check for an idle shared cache in select_task_rq_fair()

When waking affine, check for an idle shared cache, and if found, wake to that CPU/sibling instead of the waker's CPU.

This improves pgsql+oltp ramp up by roughly 8%. Possibly more for other loads, depending on overlap. The trade-off is a roughly 1% peak downturn if tasks are truly synchronous.

Cc Arjan van de Ven <[email protected]>
Cc Peter Zijlstra <[email protected]>

LKML-Reference: <[email protected]>

This commit can't be reverted due to conflict, so I reverted below 4 commits related to idle-shared-cache in 2.6.33-rc2, and then the performance was restored to 2.6.32.

fe3bcfe (sched: More generic WAKE_AFFINE vs select_idle_sibling()) a50bde5 (sched: Cleanup select_task_rq_fair())
fd21073 (sched: Fix affinity logic in select_task_rq_fair()) a1f84a3 (sched: Check for an idle shared cache in select_task_rq_fair())

This regression seems caused by cache misses of access to per cpu data. (see below perf top cache-misses data for detail)

select_idle_sibling(...)

{
        ....
        for_each_cpu_and(i, sched_domain_span(sd), &p->cpus_allowed) {
                if (!cpu_rq(i)->cfs.nr_running) {
                        target = i;
                        break;
                }
        }
	....
}

The performance can be restored to 2.6.32 as well if SD_PREFER_SIBLING is not set, so select_idle_sibling will not be called.

perf top data as follow,

2.6.33-rc1 cache-misses data (note 11.8% select_task_rq_fair)
------------------------------------------------------------------------------------
PerfTop: 12262 irqs/sec kernel:90.6% [1000Hz cache-misses], (all, 16 CPUs)
------------------------------------------------------------------------------------

             samples  pcnt function                      DSO
             _______ _____ _____________________________ ________________

            18272.00 11.8% select_task_rq_fair           [kernel.kallsyms]       
            15499.00 10.0% schedule                      [kernel.kallsyms]       
             9447.00  6.1% update_curr                   [kernel.kallsyms]       
             9255.00  6.0% _raw_spin_lock                [kernel.kallsyms]       
             5161.00  3.3% tcp_sendmsg                   [kernel.kallsyms] 

2.6.32 cache-misses data
--------------------------------------------------------------------------------------
PerfTop: 11749 irqs/sec kernel:88.2% [1000Hz cache-misses], (all, 16 CPUs)
--------------------------------------------------------------------------------------

samples pcnt function DSO
_______ _____ _____________________________ _________________
11974.00 11.5% schedule [kernel.kallsyms]
6656.00 6.4% _spin_lock [kernel.kallsyms]
5852.00 5.6% update_curr [kernel.kallsyms]
3140.00 3.0% enqueue_entity [kernel.kallsyms]
2846.00 2.7% tcp_sendmsg [kernel.kallsyms]

2.6.33-rc1 cycles data (note 6.5% select_task_rq_fair)
-------------------------------------------------------------------------------
PerfTop: 11106 irqs/sec kernel:99.7% [1000Hz cycles], (all, 16 CPUs)
-------------------------------------------------------------------------------

             samples  pcnt function                  DSO
             _______ _____ _________________________ _________________

            11658.00 10.0% schedule                  [kernel.kallsyms]
            10870.00  9.4% _raw_spin_lock            [kernel.kallsyms]
             7576.00  6.5% select_task_rq_fair       [kernel.kallsyms]
             3696.00  3.2% tcp_sendmsg               [kernel.kallsyms]
             3000.00  2.6% update_curr               [kernel.kallsyms]

2.6.32 cycles data
------------------------------------------------------------------------------------
PerfTop: 10462 irqs/sec kernel:99.8% [1000Hz cycles], (all, 16 CPUs)
------------------------------------------------------------------------------------

             samples  pcnt function                  DSO
             _______ _____ _________________________ _________________

            13364.00  9.9% schedule                  [kernel.kallsyms]
            13140.00  9.8% _spin_lock                [kernel.kallsyms]
             4903.00  3.6% tcp_sendmsg               [kernel.kallsyms]
             4017.00  3.0% update_curr               [kernel.kallsyms]
             3395.00  2.5% _spin_lock_bh             [kernel.kallsyms]

Lin Ming

#1 lore raw
Quoted 4 lines · On Mon, 04 Jan 2010 16:15:58 +0800 Lin Ming <[email protected]> wrote:

Mike & Peter,

Compared with 2.6.32, volano has ~30% regression with 2.6.33-rc1 & -rc2. Testing machine: Tigerton Xeon, 16cpus(4P/4Core), 16G memory

did this show up only on this cpu?
(since this is a multi-core-without-shared-cache cpu, it could be that we get the topology wrong and think cores share cache where they don't)

Signature · 3 lines
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org
#2 lore raw
Quoted 11 lines · Arjan van de Ven
Quoted 4 lines · On Mon, 04 Jan 2010 16:15:58 +0800 Lin Ming <[email protected]> wrote:

Mike & Peter,

Compared with 2.6.32, volano has ~30% regression with 2.6.33-rc1 & -rc2. Testing machine: Tigerton Xeon, 16cpus(4P/4Core), 16G memory

did this show up only on this cpu?
(since this is a multi-core-without-shared-cache cpu, it could be that we get the topology wrong and think cores share cache where they don't)

My fault for using PREFER_SIBLING I guess. However, I do wonder why in the heck we set that at the CPU domain level. Siblings lie northward.

-Mike

#3 lore raw
Quoted 9 lines · Mike Galbraith
Quoted 8 lines · Arjan van de Ven
Quoted 4 lines · On Mon, 04 Jan 2010 16:15:58 +0800 Lin Ming <[email protected]> wrote:

Mike & Peter,

Compared with 2.6.32, volano has ~30% regression with 2.6.33-rc1 & -rc2. Testing machine: Tigerton Xeon, 16cpus(4P/4Core), 16G memory

Quoted 6 lines
Quoted 3 lines

did this show up only on this cpu?
(since this is a multi-core-without-shared-cache cpu, it could be that we get the topology wrong and think cores share cache where they don't)

My fault for using PREFER_SIBLING I guess. However, I do wonder why in the heck we set that at the CPU domain level. Siblings lie northward.

Ah, PREFER_SIBLING means prefer sibling domain, not sibling thread. Its set at the CPU (really socket) level so make tasks spread over sockets first, so that there is no competition for the socket wide resources.

Your change is sane, but we really want a more extensive sched domain tree in the near future, reflecting the full machine topology.

#4 lore raw
Quoted 14 lines · Peter Zijlstra
Quoted 13 lines · Mike Galbraith
Quoted 11 lines · Arjan van de Ven
Quoted 4 lines · On Mon, 04 Jan 2010 16:15:58 +0800 Lin Ming <[email protected]> wrote:

Mike & Peter,

Compared with 2.6.32, volano has ~30% regression with 2.6.33-rc1 & -rc2. Testing machine: Tigerton Xeon, 16cpus(4P/4Core), 16G memory

did this show up only on this cpu?
(since this is a multi-core-without-shared-cache cpu, it could be that we get the topology wrong and think cores share cache where they don't)

Quoted 6 lines
Quoted 2 lines

My fault for using PREFER_SIBLING I guess. However, I do wonder why in the heck we set that at the CPU domain level. Siblings lie northward.

Ah, PREFER_SIBLING means prefer sibling domain, not sibling thread. Its set at the CPU (really socket) level so make tasks spread over sockets first, so that there is no competition for the socket wide resources.

WRT the regression, would you prefer only the sched_fair.c hunk, and maybe plunking the topology hunk in sched_devel, or both lines in one patch, since ramp-up gain remains unrealized half of the time on Nehalem and ilk.

Quoted 2 lines

Your change is sane, but we really want a more extensive sched domain tree in the near future, reflecting the full machine topology.

Yeah.

-Mike

#5 lore raw
Quoted 18 lines · Mike Galbraith
Quoted 17 lines · Peter Zijlstra
Quoted 15 lines · Mike Galbraith
Quoted 11 lines · Arjan van de Ven
Quoted 4 lines · On Mon, 04 Jan 2010 16:15:58 +0800 Lin Ming <[email protected]> wrote:

Mike & Peter,

Compared with 2.6.32, volano has ~30% regression with 2.6.33-rc1 & -rc2. Testing machine: Tigerton Xeon, 16cpus(4P/4Core), 16G memory

did this show up only on this cpu?
(since this is a multi-core-without-shared-cache cpu, it could be that we get the topology wrong and think cores share cache where they don't)

My fault for using PREFER_SIBLING I guess. However, I do wonder why in the heck we set that at the CPU domain level. Siblings lie northward.

Quoted 8 lines
Quoted 3 lines

Ah, PREFER_SIBLING means prefer sibling domain, not sibling thread. Its set at the CPU (really socket) level so make tasks spread over sockets first, so that there is no competition for the socket wide resources.

WRT the regression, would you prefer only the sched_fair.c hunk, and maybe plunking the topology hunk in sched_devel, or both lines in one patch, since ramp-up gain remains unrealized half of the time on Nehalem and ilk.

Both bits seem sane I guess, you change SD_SIBLING_INIT(), right? Threads really do share package resources so it makes sense to set it.

I guess its back to poking at nehalem to see what makes it tick..

#6 lore raw
Quoted 8 lines · Peter Zijlstra
Quoted 4 lines · Mike Galbraith

WRT the regression, would you prefer only the sched_fair.c hunk, and maybe plunking the topology hunk in sched_devel, or both lines in one patch, since ramp-up gain remains unrealized half of the time on Nehalem and ilk.

Both bits seem sane I guess, you change SD_SIBLING_INIT(), right?

Right.

Quoted 3 lines

Threads really do share package resources so it makes sense to set it.

I guess its back to poking at nehalem to see what makes it tick..

I asked Santa for a quad socket Nehalem and a portable nuclear reactor to power it, but the stingy old fart let me down ;-)

sched: fix vmark regression on big machines

SD_PREFER_SIBLING is set at the CPU domain level if power saving isn't enabled, leading to many cache misses on large machines as we traverse looking for an idle shared cache to wake to. Change the enabler of select_idle_sibling() to SD_SHARE_PKG_RESOURCES, and enable same at the sibling domain level.

LKML-Reference: <new-submission>

include/linux/topology.h +1 -1
index 57e6357..5b81156 100644
@@ -99,7 +99,7 @@ int arch_update_cpu_topology(void);
 				| 1*SD_WAKE_AFFINE			\
 				| 1*SD_SHARE_CPUPOWER			\
 				| 0*SD_POWERSAVINGS_BALANCE		\
-				| 0*SD_SHARE_PKG_RESOURCES		\
+				| 1*SD_SHARE_PKG_RESOURCES		\
 				| 0*SD_SERIALIZE			\
 				| 0*SD_PREFER_SIBLING			\
 				,					\
kernel/sched_fair.c +1 -1
index 42ac3c9..8fe7ee8 100644
@@ -1508,7 +1508,7 @@ static int select_task_rq_fair(struct task_struct *p, int sd_flag, int wake_flag
 			 * If there's an idle sibling in this domain, make that
 			 * the wake_affine target instead of the current cpu.
 			 */
-			if (tmp->flags & SD_PREFER_SIBLING)
+			if (tmp->flags & SD_SHARE_PKG_RESOURCES)
 				target = select_idle_sibling(p, tmp, target);
 
 			if (target >= 0) {
#7 lore raw

[tip:sched/urgent] sched: Fix vmark regression on big machines

Commit-ID:  50b926e439620c469565e8be0f28be78f5fca1ce
Gitweb:     http://git.kernel.org/tip/50b926e439620c469565e8be0f28be78f5fca1ce
Author:     Mike Galbraith <[email protected]>
AuthorDate: Mon, 4 Jan 2010 14:44:56 +0100
Committer:  Ingo Molnar <[email protected]>
CommitDate: Thu, 21 Jan 2010 13:39:03 +0100

sched: Fix vmark regression on big machines

SD_PREFER_SIBLING is set at the CPU domain level if power saving isn't enabled, leading to many cache misses on large machines as we traverse looking for an idle shared cache to wake to. Change the enabler of select_idle_sibling() to SD_SHARE_PKG_RESOURCES, and enable same at the sibling domain level.

LKML-Reference: <[email protected]>

---
 include/linux/topology.h |    2 +-
 kernel/sched_fair.c      |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
include/linux/topology.h +1 -1
index 57e6357..5b81156 100644
@@ -99,7 +99,7 @@ int arch_update_cpu_topology(void);
 				| 1*SD_WAKE_AFFINE			\
 				| 1*SD_SHARE_CPUPOWER			\
 				| 0*SD_POWERSAVINGS_BALANCE		\
-				| 0*SD_SHARE_PKG_RESOURCES		\
+				| 1*SD_SHARE_PKG_RESOURCES		\
 				| 0*SD_SERIALIZE			\
 				| 0*SD_PREFER_SIBLING			\
 				,					\
kernel/sched_fair.c +1 -1
index 42ac3c9..8fe7ee8 100644
@@ -1508,7 +1508,7 @@ static int select_task_rq_fair(struct task_struct *p, int sd_flag, int wake_flag
 			 * If there's an idle sibling in this domain, make that
 			 * the wake_affine target instead of the current cpu.
 			 */
-			if (tmp->flags & SD_PREFER_SIBLING)
+			if (tmp->flags & SD_SHARE_PKG_RESOURCES)
 				target = select_idle_sibling(p, tmp, target);
 
 			if (target >= 0) {
author
#8 lore raw
In reply to Arjan van de Ven
Quoted 11 lines · Arjan van de Ven
Quoted 4 lines · On Mon, 04 Jan 2010 16:15:58 +0800 Lin Ming <[email protected]> wrote:

Mike & Peter,

Compared with 2.6.32, volano has ~30% regression with 2.6.33-rc1 & -rc2. Testing machine: Tigerton Xeon, 16cpus(4P/4Core), 16G memory

did this show up only on this cpu?
(since this is a multi-core-without-shared-cache cpu, it could be that we get the topology wrong and think cores share cache where they don't)

Tulsa machine(16cpus, 4P/2Core/HT) also has ~8% regression and it's now fixed by Mike's patch.

Lin Ming

#9 lore raw
Quoted 9 lines · Lin Ming
Quoted 8 lines · Arjan van de Ven
Quoted 4 lines · On Mon, 04 Jan 2010 16:15:58 +0800 Lin Ming <[email protected]> wrote:

Mike & Peter,

Compared with 2.6.32, volano has ~30% regression with 2.6.33-rc1 & -rc2. Testing machine: Tigerton Xeon, 16cpus(4P/4Core), 16G memory

Quoted 6 lines
Quoted 3 lines

did this show up only on this cpu?
(since this is a multi-core-without-shared-cache cpu, it could be that we get the topology wrong and think cores share cache where they don't)

Tulsa machine(16cpus, 4P/2Core/HT) also has ~8% regression and it's now fixed by Mike's patch.

Excellent. Thanks for testing.

-Mike