cpu hotplug: possible_cpus broken (again?) next-20130607
possible_cpus looks broken again. I'm booting with:
maxcpus=10 possible_cpus=160
But I only get 0-9 in sysfs:
Quoted 3 lines
# ls /sys/devices/system/cpu/ cpu0 cpu2 cpu4 cpu6 cpu8 cpufreq kernel_max offline possible probe uevent cpu1 cpu3 cpu5 cpu7 cpu9 cpuidle modalias online present release
Quoted 9 lines · Dave Hansen
possible_cpuslooks broken again. I'm booting with:maxcpus=10 possible_cpus=160
But I only get 0-9 in sysfs:
Quoted 3 lines
# ls /sys/devices/system/cpu/ cpu0 cpu2 cpu4 cpu6 cpu8 cpufreq kernel_max offline possible probe uevent cpu1 cpu3 cpu5 cpu7 cpu9 cpuidle modalias online present release
Can you please test the acpi-hotplug branch of the linux-pm.git tree?
Rafael
Signature · 2 lines
I speak only for myself. Rafael J. Wysocki, Intel Open Source Technology Center.
Quoted 5 lines · Rafael J. Wysocki
Quoted 4 lines · Dave Hansen
possible_cpuslooks broken again. I'm booting with:maxcpus=10 possible_cpus=160
Quoted 7 lines
Quoted 5 lines
But I only get 0-9 in sysfs:
Quoted 3 lines
# ls /sys/devices/system/cpu/ cpu0 cpu2 cpu4 cpu6 cpu8 cpufreq kernel_max offline possible probe uevent cpu1 cpu3 cpu5 cpu7 cpu9 cpuidle modalias online present releaseCan you please test the acpi-hotplug branch of the linux-pm.git tree?
That branch seems to work happily.
Quoted 8 lines · Dave Hansen
Quoted 7 lines · Rafael J. Wysocki
Quoted 6 lines · Dave Hansen
possible_cpuslooks broken again. I'm booting with:maxcpus=10 possible_cpus=160
But I only get 0-9 in sysfs:
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
# ls /sys/devices/system/cpu/ cpu0 cpu2 cpu4 cpu6 cpu8 cpufreq kernel_max offline possible probe uevent cpu1 cpu3 cpu5 cpu7 cpu9 cpuidle modalias online present releaseCan you please test the acpi-hotplug branch of the linux-pm.git tree?
That branch seems to work happily.
In that case the problem may have been reintroduced by a merge conflict fix in linux-next.
Thanks,
Rafael
Signature · 2 lines
I speak only for myself. Rafael J. Wysocki, Intel Open Source Technology Center.
Quoted 13 lines · Rafael J. Wysocki
Quoted 12 lines · Dave Hansen
Quoted 11 lines · Rafael J. Wysocki
Quoted 9 lines · Dave Hansen
possible_cpuslooks broken again. I'm booting with:maxcpus=10 possible_cpus=160
But I only get 0-9 in sysfs:
Quoted 3 lines
# ls /sys/devices/system/cpu/ cpu0 cpu2 cpu4 cpu6 cpu8 cpufreq kernel_max offline possible probe uevent cpu1 cpu3 cpu5 cpu7 cpu9 cpuidle modalias online present release
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Can you please test the acpi-hotplug branch of the linux-pm.git tree?
That branch seems to work happily.
In that case the problem may have been reintroduced by a merge conflict fix in linux-next.
I believe the problem was introduced by the following change. From the description, though, this is exactly what this patch was trying to change... Adding Youguan to the list.
commit 3e275a5ba367ab74b3a4e49114307baed989fcac
Author: Youquan Song <[email protected]> Date: Fri Jun 7 10:07:08 2013 +1000
drivers/base/cpu.c: fix maxcpus boot option
Thanks,
-Toshi
Quoted 25 lines · Toshi Kani
Quoted 13 lines · Rafael J. Wysocki
Quoted 12 lines · Dave Hansen
Quoted 11 lines · Rafael J. Wysocki
Quoted 9 lines · Dave Hansen
possible_cpuslooks broken again. I'm booting with:maxcpus=10 possible_cpus=160
But I only get 0-9 in sysfs:
Quoted 3 lines
# ls /sys/devices/system/cpu/ cpu0 cpu2 cpu4 cpu6 cpu8 cpufreq kernel_max offline possible probe uevent cpu1 cpu3 cpu5 cpu7 cpu9 cpuidle modalias online present releaseQuoted 6 lines
Quoted 3 lines
Quoted 1 line
Can you please test the acpi-hotplug branch of the linux-pm.git tree?
That branch seems to work happily.
In that case the problem may have been reintroduced by a merge conflict fix in linux-next.
I believe the problem was introduced by the following change. From the description, though, this is exactly what this patch was trying to change... Adding Youguan to the list.
Quoted 6 lines
commit
3e275a5ba367ab74b3a4e49114307baed989fcacAuthor: Youquan Song <[email protected]> Date: Fri Jun 7 10:07:08 2013 +1000
drivers/base/cpu.c: fix maxcpus boot option
Hi Toshi,
Thanks Thoshi for the information.
please try the below patch to fix the issue by moving the code to store_online.
drivers/base/cpu.c +2 -0
@@ -60,6 +60,13 @@ static ssize_t __ref store_online(struct device *dev,kobject_uevent (&dev->kobj,KOBJ_OFFLINE );break ;case '1' : +#ifdef CONFIG_SMP +/* return when cpu number greater than maximum number of
CPUs */
+ if (setup_max_cpus <= num_online_cpus() + 1) {
+ cpu_hotplug_driver_unlock();
+ return -EINVAL;
+ }
+#endif
from_nid = cpu_to_node(cpuid);
ret = cpu_up(cpuid);Thanks
-Youquan
Review
Quoted 10 lines · Youquan Song
drivers/base/cpu.c +2
-0+#ifdef CONFIG_SMP +/* return when cpu number greater than maximum number of CPUs */
(quoted diff) +5
-0+ if (setup_max_cpus <= num_online_cpus() + 1) { + cpu_hotplug_driver_unlock(); + return -EINVAL; + } +#endif from_nid = cpu_to_node(cpuid); ret = cpu_up(cpuid);
Your patch is line-wrapped.
Also, the #ifdef is unnecessary. If CONFIG_SMP is off:
static const unsigned int setup_max_cpus = NR_CPUS; #define num_online_cpus() 1U
The compiler will take care of optimizing out the the if() without the explicit #ifdef.
Also, the +1 looks goofy to me. Doesn't this do the same thing (and isn't it much easier to read)?
if (num_online_cpus() >= setup_max_cpus)
Quoted 19 lines · Youquan Song
Review
Quoted 10 lines
(quoted diff) +2
-0+#ifdef CONFIG_SMP + /* return when cpu number greater than maximum number ofCPUs */
(quoted diff) +5
-0+ if (setup_max_cpus <= num_online_cpus() + 1) { + cpu_hotplug_driver_unlock(); + return -EINVAL; + } +#endif from_nid = cpu_to_node(cpuid); ret = cpu_up(cpuid);Your patch is line-wrapped.
Also, the #ifdef is unnecessary. If
CONFIG_SMPis off:static const unsigned int setup_max_cpus = NR_CPUS; #define num_online_cpus() 1U
Quoted 8 lines
The compiler will take care of optimizing out the the
if()without the explicit #ifdef.Also, the +1 looks goofy to me. Doesn't this do the same thing (and isn't it much easier to read)?
if (
num_online_cpus()>=setup_max_cpus)
Thanks. Here is a formal patch for it. please review and try.
Subject: [PATCH] core: Fix maxcpus boot option broken
maxcpus boot option to limit maximum number of CPUs on system, but this option is broken at recent kernel. Though we use maxcpus to limit CPUs number, but current kernel will register all of present CPUs in sysfs.
udev will enumerate all registered cpu at sysfs, and it will bring up the CPU if the CPU is offline. So the maxcpus option is broken.
This patch will limit the online cpus number not over limitation of maxcpus option. So it will keep the maxcpus limitation when udev enumeration or other intention of bring up CPUs over the limitation by method like echo 1 > /sys/devices/system/cpu/online
---
drivers/base/cpu.c +7 -0
@@ -60,6 +60,13 @@ static ssize_t __ref store_online(struct device *dev,kobject_uevent (&dev->kobj,KOBJ_OFFLINE );break ;case '1' : +/* Return when online cpu number equal or greater than +* maximum number of CPUs */ +if (num_online_cpus () >= setup_max_cpus) { +cpu_hotplug_driver_unlock (); +return -EINVAL ; + } + from_nid =cpu_to_node (cpuid); ret =cpu_up (cpuid);
Quoted 38 lines · Youquan Song
Quoted 19 lines · Youquan Song
drivers/base/cpu.c static ssize_t __ref store_online(struct device *dev, · line 71
Quoted 10 lines
(quoted diff) +2
-0+#ifdef CONFIG_SMP + /* return when cpu number greater than maximum number ofCPUs */
drivers/base/cpu.c +5
-0+if (setup_max_cpus <=num_online_cpus () +1 ) { +cpu_hotplug_driver_unlock (); +return -EINVAL ; + } +#endif from_nid =cpu_to_node (cpuid); ret =cpu_up (cpuid);Your patch is line-wrapped.
Also, the #ifdef is unnecessary. If
CONFIG_SMPis off:static const unsigned int setup_max_cpus = NR_CPUS; #define num_online_cpus() 1UQuoted 8 lines
The compiler will take care of optimizing out the the
if()without the explicit #ifdef.Also, the +1 looks goofy to me. Doesn't this do the same thing (and isn't it much easier to read)?
if (
num_online_cpus()>=setup_max_cpus)Thanks. Here is a formal patch for it. please review and try.
Subject: [PATCH] core: Fix maxcpus boot option broken
maxcpus boot option to limit maximum number of CPUs on system, but this option is broken at recent kernel. Though we use maxcpus to limit CPUs number, but current kernel will register all of present CPUs in sysfs.
udev will enumerate all registered cpu at sysfs, and it will bring up the CPU if the CPU is offline. So the maxcpus option is broken.
Quoted 4 lines
This patch will limit the online cpus number not over limitation of maxcpus option. So it will keep the maxcpus limitation when udev enumeration or other intention of bring up CPUs over the limitation by method like echo 1 >
/sys/devices/system/cpu/online
Interesting, you are changing long standing meaning of maxcpus=
We always use maxcpus=1 to have one cpu up, and later in user space to online other cpus like
echo 1 > /sys/devices/system/cpuX/online.
aka maxcpus= is a soft limit or initial online nr.
we already have nr_cpus= for hard limit.
So need to drop
commit 3e275a5ba367ab74b3a4e49114307baed989fcac
Author: Youquan Song <[email protected]>
Date: Fri Jun 7 10:07:08 2013 +1000
drivers/base/cpu.c: fix maxcpus boot option
Greg,
Can you drop that 3e275a5ba36 from your drivers/core tree ?
Thanks
Yinghai
Quoted 53 lines · Yinghai Lu
Quoted 38 lines · Youquan Song
Quoted 19 lines · Youquan Song
drivers/base/cpu.c static ssize_t __ref store_online(struct device *dev, · line 71
Quoted 10 lines
(quoted diff) +2
-0+#ifdef CONFIG_SMP + /* return when cpu number greater than maximum number ofCPUs */
drivers/base/cpu.c +5
-0+if (setup_max_cpus <=num_online_cpus () +1 ) { +cpu_hotplug_driver_unlock (); +return -EINVAL ; + } +#endif from_nid =cpu_to_node (cpuid); ret =cpu_up (cpuid);Your patch is line-wrapped.
Also, the #ifdef is unnecessary. If
CONFIG_SMPis off:static const unsigned int setup_max_cpus = NR_CPUS; #define num_online_cpus() 1UQuoted 8 lines
The compiler will take care of optimizing out the the
if()without the explicit #ifdef.Also, the +1 looks goofy to me. Doesn't this do the same thing (and isn't it much easier to read)?
if (
num_online_cpus()>=setup_max_cpus)Thanks. Here is a formal patch for it. please review and try.
Subject: [PATCH] core: Fix maxcpus boot option broken
maxcpus boot option to limit maximum number of CPUs on system, but this option is broken at recent kernel. Though we use maxcpus to limit CPUs number, but current kernel will register all of present CPUs in sysfs.
udev will enumerate all registered cpu at sysfs, and it will bring up the CPU if the CPU is offline. So the maxcpus option is broken.Quoted 4 lines
This patch will limit the online cpus number not over limitation of maxcpus option. So it will keep the maxcpus limitation when udev enumeration or other intention of bring up CPUs over the limitation by method like echo 1 >
/sys/devices/system/cpu/onlineInteresting, you are changing long standing meaning of maxcpus=
We always use maxcpus=1 to have one cpu up, and later in user space to online other cpus like
echo 1 >/sys/devices/system/cpuX/online.aka maxcpus= is a soft limit or initial online nr.
Quoted 8 lines
we already have nr_cpus= for hard limit.
So need to drop
commit3e275a5ba367ab74b3a4e49114307baed989fcac
Author: Youquan Song <[email protected]>
Date: Fri Jun 7 10:07:08 2013 +1000
drivers/base/cpu.c: fix maxcpus boot option
Agreed.
Thanks,
Rafael
Quoted 6 lines
Greg,
Can you drop that3e275a5ba36from yourdrivers/coretree ?Thanks
Yinghai
Signature · 2 lines
I speak only for myself. Rafael J. Wysocki, Intel Open Source Technology Center.
Quoted 10 lines
Quoted 10 lines
Interesting, you are changing long standing meaning of maxcpus=
We always use maxcpus=1 to have one cpu up, and later in user space to online other cpus like
echo 1 >/sys/devices/system/cpuX/online.aka maxcpus= is a soft limit or initial online nr.
we already have nr_cpus= for hard limit.
Quoted 8 lines
Quoted 6 lines
So need to drop
commit3e275a5ba367ab74b3a4e49114307baed989fcac
Author: Youquan Song <[email protected]>
Date: Fri Jun 7 10:07:08 2013 +1000
drivers/base/cpu.c: fix maxcpus boot optionAgreed.
Yes. I also agree to drop it and the fix need more consideration. I try use maxcpus to limit cpu number to debug a well known applition because it fail to run when cpu number is larger to > 69. When I use maxcpus at to limit the boot CPUs number, but udev will enable all of the CPUs at 3.10 kernel automatically.
I also try maxcpus at 3.0 kernel, it does not show the maxcpus issue. I have digged out recently, it is the commit at 3.2 kernel 8a25a2fd126c621f44f3aeaef80d51f00fc11639 "cpu: convert 'cpu' and 'machinecheck' sysdev_class to a regular subsystem" result in udev automatically enable all of CPUs though maxcpus has been provided.
So the next, I need look at udev try to enable all of CPUs though maxcpus provided. Possibly, it can also fix it in udev daemon.
Secondly, I think that the maxcpus= option description is too confused in Documentation/kernel-parameters.txt. The maxcpus and nr_cpus option need switch their name.
Currently:
maxcpus= [SMP] Maximum number of processors that an SMP kernel
should make use of. maxcpus=n : n >= 0 limits the kernel to using 'n' processors. n=0 is a special case, it is equivalent to "nosmp", which also disables the IO APIC.
How about change to
maxcpus= [SMP] Maximum number of processors that an SMP kernel
bring up during booting. maxcpus=n : n >= 0 limits the kernel to using 'n' processors. n=0 is a special case, it is equivalent to "nosmp", which also disables the IO APIC.
Thanks
-Youquan
Quoted 15 lines · Youquan Song
Quoted 15 lines
Quoted 15 lines
Interesting, you are changing long standing meaning of maxcpus=
We always use maxcpus=1 to have one cpu up, and later in user space to online other cpus like
echo 1 >/sys/devices/system/cpuX/online.aka maxcpus= is a soft limit or initial online nr.
we already have nr_cpus= for hard limit.
So need to drop
commit3e275a5ba367ab74b3a4e49114307baed989fcac
Author: Youquan Song <[email protected]>
Date: Fri Jun 7 10:07:08 2013 +1000
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
drivers/base/cpu.c: fix maxcpus boot optionAgreed.
Yes. I also agree to drop it and the fix need more consideration.
Agreed.
Quoted 5 lines
I try use maxcpus to limit cpu number to debug a well known applition because it fail to run when cpu number is larger to > 69. When I use maxcpus at to limit the boot CPUs number, but udev will enable all of the CPUs at 3.10 kernel automatically.
I also try maxcpus at 3.0 kernel, it does not show the maxcpus issue.
I tested Linus's 3.10-rc5 kernel on Fedora 18, but I did not see such behavior. Could this be related with your environment?
Quoted 11 lines
I have digged out recently, it is the commit at 3.2 kernel
8a25a2fd126c621f44f3aeaef80d51f00fc11639"cpu: convert 'cpu' and 'machinecheck'sysdev_classto a regular subsystem" result in udev automatically enable all of CPUs though maxcpus has been provided.So the next, I need look at udev try to enable all of CPUs though maxcpus provided. Possibly, it can also fix it in udev daemon.
Secondly, I think that the maxcpus= option description is too confused in
Documentation/kernel-parameters.txt. The maxcpus andnr_cpusoption need switch their name.
I am fine with the clarification.
Quoted 8 lines
Currently:
maxcpus= [SMP] Maximum number of processors that an SMP kernel
should make use of. maxcpus=n : n >= 0 limits the kernel to using 'n' processors. n=0 is a special case, it is equivalent to "nosmp", which also disables the IO APIC.
Quoted 4 lines
How about change to
maxcpus= [SMP] Maximum number of processors that an SMP kernel
bring up during booting. maxcpus=n : n >= 0 limits the
bring -> brings
Quoted 3 lines
kernel to using 'n' processors. n=0 is a special case, it is equivalent to "nosmp", which also disables the IO APIC.
Thanks,
-Toshi