Skip to contentSkip to conversation

sched: fix llc shared map unreleased during cpu hotplug

PATCH RESEND Superseded Wanpeng Li 3 participants 4 messages Aug 25, 2014 – Sep 9, 2014 x86

v4 of 7 Superseded Latest is v7 →

lore
Share & tools
lore

Wanpeng Li

author
#0 lore raw

PATCH RESEND v4 1/1 sched: fix llc shared map unreleased during cpu hotplug

[  220.262245] Call Trace:
[  220.262252]  [<ffffffff810e8396>] load_balance+0x156/0x980
[  220.262259]  [<ffffffff816eeffe>] ? _raw_spin_unlock_irqrestore+0x2e/0xa0
[  220.262266]  [<ffffffff810e9aa3>] idle_balance+0xe3/0x150
[  220.262270]  [<ffffffff816ec4e7>] __schedule+0x797/0x8d0
[  220.262277]  [<ffffffff816ec934>] schedule+0x24/0x70
[  220.262283]  [<ffffffff816e9cd9>] schedule_timeout+0x119/0x1f0
[  220.262294]  [<ffffffff810bb6e0>] ? lock_timer_base+0x70/0x70
[  220.262301]  [<ffffffff816e9dc9>] schedule_timeout_uninterruptible+0x19/0x20
[  220.262308]  [<ffffffff810bd3e8>] msleep+0x18/0x20
[  220.262317]  [<ffffffff813aa11a>] lock_device_hotplug_sysfs+0x2a/0x50
[  220.262323]  [<ffffffff813aa16e>] online_store+0x2e/0x80
[  220.262358]  [<ffffffff813a873b>] dev_attr_store+0x1b/0x20

Last level cache shared map is built during cpu up and build sched domain routine takes advantage of it to setup sched domain cpu topology, however, llc shared map is unreleased during cpu disable which lead to invalid sched domain cpu topology. This patch fix it by release llc shared map correctly during cpu disable.

---
v3 -> v4:
* simplify backtrace
v2 -> v3:
* simplify backtrace
v1 -> v2:
* fix subject line

 arch/x86/kernel/smpboot.c | 3 +++
 1 file changed, 3 insertions(+)
arch/x86/kernel/smpboot.c +3 -0
index 5492798..0134ec7 100644
@@ -1292,6 +1292,9 @@ static void remove_siblinginfo(int cpu)
 
 	for_each_cpu(sibling, cpu_sibling_mask(cpu))
 		cpumask_clear_cpu(cpu, cpu_sibling_mask(sibling));
+	for_each_cpu(sibling, cpu_llc_shared_mask(cpu))
+		cpumask_clear_cpu(cpu, cpu_llc_shared_mask(sibling));
+	cpumask_clear(cpu_llc_shared_mask(cpu));
 	cpumask_clear(cpu_sibling_mask(cpu));
 	cpumask_clear(cpu_core_mask(cpu));
 	c->phys_proc_id = 0;
Signature · 1 line
1.9.1
#1 lore raw

Hm, this patchlet seems to be having trouble taking wing.

(tosses it up in the air.. flap yer little wings patchlet)

I thought you had changed the title to..

x86,cpu-hotplug: clear llc_shared_mask at CPU hotplug

..due to Peter saying sched is only a consumer, but v4 somehow reverted back to being a sched patch.

Quoted 49 lines · Wanpeng Li
[  220.262245] Call Trace:
[  220.262252]  [<ffffffff810e8396>] load_balance+0x156/0x980
[  220.262259]  [<ffffffff816eeffe>] ? _raw_spin_unlock_irqrestore+0x2e/0xa0
[  220.262266]  [<ffffffff810e9aa3>] idle_balance+0xe3/0x150
[  220.262270]  [<ffffffff816ec4e7>] __schedule+0x797/0x8d0
[  220.262277]  [<ffffffff816ec934>] schedule+0x24/0x70
[  220.262283]  [<ffffffff816e9cd9>] schedule_timeout+0x119/0x1f0
[  220.262294]  [<ffffffff810bb6e0>] ? lock_timer_base+0x70/0x70
[  220.262301]  [<ffffffff816e9dc9>] schedule_timeout_uninterruptible+0x19/0x20
[  220.262308]  [<ffffffff810bd3e8>] msleep+0x18/0x20
[  220.262317]  [<ffffffff813aa11a>] lock_device_hotplug_sysfs+0x2a/0x50
[  220.262323]  [<ffffffff813aa16e>] online_store+0x2e/0x80
[  220.262358]  [<ffffffff813a873b>] dev_attr_store+0x1b/0x20

Last level cache shared map is built during cpu up and build sched domain routine takes advantage of it to setup sched domain cpu topology, however, llc shared map is unreleased during cpu disable which lead to invalid sched domain cpu topology. This patch fix it by release llc shared map correctly during cpu disable.

---
v3 -> v4:
* simplify backtrace
v2 -> v3:
* simplify backtrace
v1 -> v2:
* fix subject line

 arch/x86/kernel/smpboot.c | 3 +++
 1 file changed, 3 insertions(+)
arch/x86/kernel/smpboot.c +3 -0
index 5492798..0134ec7 100644
@@ -1292,6 +1292,9 @@ static void remove_siblinginfo(int cpu)
 
 	for_each_cpu(sibling, cpu_sibling_mask(cpu))
 		cpumask_clear_cpu(cpu, cpu_sibling_mask(sibling));
+	for_each_cpu(sibling, cpu_llc_shared_mask(cpu))
+		cpumask_clear_cpu(cpu, cpu_llc_shared_mask(sibling));
+	cpumask_clear(cpu_llc_shared_mask(cpu));
 	cpumask_clear(cpu_sibling_mask(cpu));
 	cpumask_clear(cpu_core_mask(cpu));
 	c->phys_proc_id = 0;
author
#2 lore raw

Hi Mike,

Quoted 11 lines · Mike Galbraith

Hm, this patchlet seems to be having trouble taking wing.

(tosses it up in the air.. flap yer little wings patchlet)

I thought you had changed the title to..

x86,cpu-hotplug: clear llc_shared_mask at CPU hotplug

..due to Peter saying sched is only a consumer, but v4 somehow reverted back to being a sched patch.

I send out the RESEND v4 by a mistake, please refer the below link for the patch.
https://lkml.org/lkml/2014/7/29/159

Regards,
Wanpeng Li

Quoted 51 lines · Wanpeng Li
Quoted 49 lines
[  220.262245] Call Trace:
[  220.262252]  [<ffffffff810e8396>] load_balance+0x156/0x980
[  220.262259]  [<ffffffff816eeffe>] ? _raw_spin_unlock_irqrestore+0x2e/0xa0
[  220.262266]  [<ffffffff810e9aa3>] idle_balance+0xe3/0x150
[  220.262270]  [<ffffffff816ec4e7>] __schedule+0x797/0x8d0
[  220.262277]  [<ffffffff816ec934>] schedule+0x24/0x70
[  220.262283]  [<ffffffff816e9cd9>] schedule_timeout+0x119/0x1f0
[  220.262294]  [<ffffffff810bb6e0>] ? lock_timer_base+0x70/0x70
[  220.262301]  [<ffffffff816e9dc9>] schedule_timeout_uninterruptible+0x19/0x20
[  220.262308]  [<ffffffff810bd3e8>] msleep+0x18/0x20
[  220.262317]  [<ffffffff813aa11a>] lock_device_hotplug_sysfs+0x2a/0x50
[  220.262323]  [<ffffffff813aa16e>] online_store+0x2e/0x80
[  220.262358]  [<ffffffff813a873b>] dev_attr_store+0x1b/0x20

Last level cache shared map is built during cpu up and build sched domain routine takes advantage of it to setup sched domain cpu topology, however, llc shared map is unreleased during cpu disable which lead to invalid sched domain cpu topology. This patch fix it by release llc shared map correctly during cpu disable.

---
v3 -> v4:
* simplify backtrace
v2 -> v3:
* simplify backtrace
v1 -> v2:
* fix subject line

 arch/x86/kernel/smpboot.c | 3 +++
 1 file changed, 3 insertions(+)
arch/x86/kernel/smpboot.c +3 -0
index 5492798..0134ec7 100644
@@ -1292,6 +1292,9 @@ static void remove_siblinginfo(int cpu)
 
 	for_each_cpu(sibling, cpu_sibling_mask(cpu))
 		cpumask_clear_cpu(cpu, cpu_sibling_mask(sibling));
+	for_each_cpu(sibling, cpu_llc_shared_mask(cpu))
+		cpumask_clear_cpu(cpu, cpu_llc_shared_mask(sibling));
+	cpumask_clear(cpu_llc_shared_mask(cpu));
 	cpumask_clear(cpu_sibling_mask(cpu));
 	cpumask_clear(cpu_core_mask(cpu));
 	c->phys_proc_id = 0;
#3 lore raw

I didn't look at it that hard since it claimed to be a sched patch. I prioritize things in my inbox which say x86 for obvious reasons. Since I have over 12,000 unread messages in my inbox alone...

Sent from my tablet, pardon any formatting problems.

Quoted 72 lines · Wanpeng Li

Hi Mike,

Quoted 11 lines · Mike Galbraith

Hm, this patchlet seems to be having trouble taking wing.

(tosses it up in the air.. flap yer little wings patchlet)

I thought you had changed the title to..

x86,cpu-hotplug: clear llc_shared_mask at CPU hotplug

..due to Peter saying sched is only a consumer, but v4 somehow reverted back to being a sched patch.

I send out the RESEND v4 by a mistake, please refer the below link for the patch.
https://lkml.org/lkml/2014/7/29/159

Regards,
Wanpeng Li

Quoted 51 lines
Quoted 50 lines · Wanpeng Li
[  220.262245] Call Trace:
[  220.262252]  [<ffffffff810e8396>] load_balance+0x156/0x980
[  220.262259]  [<ffffffff816eeffe>] ? _raw_spin_unlock_irqrestore+0x2e/0xa0
[  220.262266]  [<ffffffff810e9aa3>] idle_balance+0xe3/0x150
[  220.262270]  [<ffffffff816ec4e7>] __schedule+0x797/0x8d0
[  220.262277]  [<ffffffff816ec934>] schedule+0x24/0x70
[  220.262283]  [<ffffffff816e9cd9>] schedule_timeout+0x119/0x1f0
[  220.262294]  [<ffffffff810bb6e0>] ? lock_timer_base+0x70/0x70
[  220.262301]  [<ffffffff816e9dc9>] schedule_timeout_uninterruptible+0x19/0x20
[  220.262308]  [<ffffffff810bd3e8>] msleep+0x18/0x20
[  220.262317]  [<ffffffff813aa11a>] lock_device_hotplug_sysfs+0x2a/0x50
[  220.262323]  [<ffffffff813aa16e>] online_store+0x2e/0x80
[  220.262358]  [<ffffffff813a873b>] dev_attr_store+0x1b/0x20

Last level cache shared map is built during cpu up and build sched domain routine takes advantage of it to setup sched domain cpu topology, however, llc shared map is unreleased during cpu disable which lead to invalid sched domain cpu topology. This patch fix it by release llc shared map correctly during cpu disable.

---
v3 -> v4:
* simplify backtrace
v2 -> v3:
* simplify backtrace
v1 -> v2:
* fix subject line

arch/x86/kernel/smpboot.c | 3 +++

1 file changed, 3 insertions(+)
arch/x86/kernel/smpboot.c +3 -0
index 5492798..0134ec7 100644
@@ -1292,6 +1292,9 @@ static void remove_siblinginfo(int cpu)
 
   for_each_cpu(sibling, cpu_sibling_mask(cpu))
       cpumask_clear_cpu(cpu, cpu_sibling_mask(sibling));
+    for_each_cpu(sibling, cpu_llc_shared_mask(cpu))
+        cpumask_clear_cpu(cpu, cpu_llc_shared_mask(sibling));
+    cpumask_clear(cpu_llc_shared_mask(cpu));
   cpumask_clear(cpu_sibling_mask(cpu));
   cpumask_clear(cpu_core_mask(cpu));
   c->phys_proc_id = 0;