fix Kconfig, hotplug_cpu is needed for swsusp
Subsystems (2)
RFC 1/1 fix Kconfig, hotplug_cpu is needed for swsusp
HOTPLUG_CPU is needed on normal PCs, too -- it is neccessary for software suspend.
---
commit 2206dab43d50723d6b15caa8821e8d97c6b5ef28 tree cbfc324e15d216aa91ce6a51927668076de5b7db parent dd76aabd03933b80c61fa5b0c0c995950246c603 author <[email protected]> Thu, 30 Mar 2006 00:06:31 +0200 committer <[email protected]> Thu, 30 Mar 2006 00:06:31 +0200
arch/i386/Kconfig | 8 ++++---- kernel/power/process.c | 3 +-- kernel/signal.c | 1 + 3 files changed, 6 insertions(+), 6 deletions(-)
arch/i386/Kconfig +4 -4
@@ -741,12 +741,12 @@ config PHYSICAL_START config HOTPLUG_CPU bool "Support for hot-pluggable CPUs (EXPERIMENTAL)" - depends on SMP && HOTPLUG && EXPERIMENTAL && !X86_VOYAGER && !X86_PC + depends on SMP && HOTPLUG && EXPERIMENTAL && !X86_VOYAGER ---help--- - Say Y here to experiment with turning CPUs off and on. CPUs - can be controlled through /sys/devices/system/cpu. + Say Y here to experiment with turning CPUs off and on, and to + enable suspend on SMP systems. CPUs can be controlled through + /sys/devices/system/cpu. - Say N. config DOUBLEFAULT default y
Signature · 1 line
Picture of sleeping (Linux) penguin wanted...
Quoted 27 lines · Pavel Machek
HOTPLUG_CPUis needed on normal PCs, too -- it is neccessary for software suspend.Signed-off-by Pavel Machek <[email protected]>---
commit2206dab43d50723d6b15caa8821e8d97c6b5ef28treecbfc324e15d216aa91ce6a51927668076de5b7dbparentdd76aabd03933b80c61fa5b0c0c995950246c603author <[email protected]> Thu, 30 Mar 2006 00:06:31 +0200 committer <[email protected]> Thu, 30 Mar 2006 00:06:31 +0200arch/i386/Kconfig | 8 ++++---- kernel/power/process.c | 3 +-- kernel/signal.c | 1 + 3 files changed, 6 insertions(+), 6 deletions(-)arch/i386/Kconfig +0
-1@@ -741,12 +741,12 @@ config PHYSICAL_START config HOTPLUG_CPU bool "Support for hot-pluggable CPUs (EXPERIMENTAL)" - depends on SMP && HOTPLUG && EXPERIMENTAL && !X86_VOYAGER && !X86_PC
arch/i386/Kconfig config PHYSICAL_START · line 750
Quoted 10 lines
arch/i386/Kconfig +4
-3+ depends on SMP && HOTPLUG && EXPERIMENTAL && !X86_VOYAGER ---help--- - Say Y here to experiment with turning CPUs off and on. CPUs - can be controlled through /sys/devices/system/cpu. + Say Y here to experiment with turning CPUs off and on, and to + enable suspend on SMP systems. CPUs can be controlled through + /sys/devices/system/cpu. - Say N.
OK, this will get ugly. APICs are involved.
That patch simply undoes this one:
Quoted 17 lines
[PATCH] x86: make
CONFIG_HOTPLUG_CPUdepend on !X86_PCMake
CONFIG_HOTPLUG_CPUdepend on !X86_PC, so we need to turn on eitherCONFIG_GENERICARCH,CONFIG_BIGSMPor any other subarch exceptX86_PCwhenCONFIG_HOTPLUG_CPU=yWith 2.6.15+ kernels when
CONFIG_HOTPLUG_CPUis turned on we switch to bigsmp mode for sending IPI's and ioapic configurations that caused the following error message.Quoted 2 lines
Quoted 2 lines
More than 8 CPUs detected and
CONFIG_X86_PCcannot handle it. UseCONFIG_X86_GENERICARCHorCONFIG_X86_BIGSMP.Originally bigsmp was added just to handle >8 cpus, but now with hotplug cpu support we need to use bigsmp mode (why? see below), that cause the above error message even if there were less than 8 cpus in the system.
Quoted 7 lines
The message is bogus, but we are cannot use logical flat mode due to issues with broadcast IPI can confuse a CPU just comming up. We use flat physical mode just like
x86_64case. More details on why bigsmp now uses flat physical mode (vs. cluster mode) in following link.http://marc.theaimsgroup.com/?l=linux-kernel&m=113261865814107&w=2
Which I was distinctly uncomfortable with. (And glad I nixed it for 2.6.16 ;))
Ashok, we need to do something better, but what?
Quoted 7 lines · Andrew Morton
Quoted 4 lines · Pavel Machek
HOTPLUG_CPUis needed on normal PCs, too -- it is neccessary for software suspend.OK, this will get ugly. APICs are involved.
I guess you need only on systems that support >1 cpu right? I doubt you will need it on a system that cannot run with the config-generic-arch on. although we use bigsmp when hotplug is turned on, all we really end up is using flat physical mode instead of using logical mode.
I still havent understood why this wont work. Choosing CONFIG_X86_GENERICARCH shouldnt break anything AFAICT.
Pavel, you could use CONFIG_HOTPLUG_CPU, just need to enable X86_GENERICARCH now. Is there a reason you think that wont work? I wish we would revert it for a strong reason that we know will not make hotplug work on certain systems because of this choise not that we currently have X86_PC now, and are unwiling to change the config.
(PS: the word bigsmp although sounds like some large NR_CPUS, its just using a mode that permits the system to work from 1 .. >8 cpus. So there is really nothing determental to selecting this.)
Signature · 3 lines
Cheers, Ashok Raj - Open Source Technology Center
Hi.
Quoted 21 lines · Ashok Raj
Quoted 5 lines · Andrew Morton
Quoted 2 lines · Pavel Machek
HOTPLUG_CPUis needed on normal PCs, too -- it is neccessary for software suspend.OK, this will get ugly. APICs are involved.
I guess you need only on systems that support >1 cpu right? I doubt you will need it on a system that cannot run with the config-generic-arch on. although we use bigsmp when hotplug is turned on, all we really end up is using flat physical mode instead of using logical mode.
I still havent understood why this wont work. Choosing
CONFIG_X86_GENERICARCHshouldnt break anything AFAICT.Pavel, you could use
CONFIG_HOTPLUG_CPU, just need to enableX86_GENERICARCHnow. Is there a reason you think that wont work? I wish we would revert it for a strong reason that we know will not make hotplug work on certain systems because of this choise not that we currently haveX86_PCnow, and are unwiling to change the config.
Quoted 3 lines
(PS: the word bigsmp although sounds like some large
NR_CPUS, its just using a mode that permits the system to work from 1 .. >8 cpus. So there is really nothing determental to selecting this.)
So if you have a single core x86, you want X86_PC, and if you have HT or SMP, you want GENERICARCH? If so, could this be done via selects or depends or at least defaults in Kconfig?
Regards,
Nigel
Quoted 6 lines · Nigel Cunningham
Hi.
So if you have a single core x86, you want
X86_PC, and if you have HT or SMP, you want GENERICARCH? If so, could this be done via selects or depends or at least defaults inKconfig?
Yes, i think only SUSPEND_SMP is affect by this. I thought Rafael cced Pavel during that exchange, maybe i missed.
Quoted 4 lines
Regards,
Nigel
How about this patch.
Make SUSPEND_SMP depend on X86_GENERICARCH, since hotplug cpu requires !X86_PC due to some race in IPI handling. See more discussion here
http://marc.theaimsgroup.com/?l=linux-kernel&m=114303306032338&w=2
--------------------------------------------------------------
kernel/power/Kconfig +1 -1
@@ -96,5 +96,5 @@ config SWSUSP_ENCRYPT config SUSPEND_SMP bool - depends on HOTPLUG_CPU && X86 && PM + depends on HOTPLUG_CPU && X86 && PM && X86_GENERICARCH default y
Signature · 3 lines
Cheers, Ashok Raj - Open Source Technology Center
Hi.
Quoted 30 lines · Ashok Raj
Quoted 6 lines · Nigel Cunningham
Hi.
So if you have a single core x86, you want
X86_PC, and if you have HT or SMP, you want GENERICARCH? If so, could this be done via selects or depends or at least defaults inKconfig?Yes, i think only
SUSPEND_SMPis affect by this. I thought Rafael cced Pavel during that exchange, maybe i missed.Quoted 3 lines
Regards,
Nigel
How about this patch.
Make
SUSPEND_SMPdepend onX86_GENERICARCH, since hotplug cpu requires !X86_PC due to some race in IPI handling. See more discussion herehttp://marc.theaimsgroup.com/?l=linux-kernel&m=114303306032338&w=2
Signed-off-by Ashok Raj <[email protected]>--------------------------------------------------------------
Index: linux-2.6.16-git16/kernel/power/Kconfig
===================================================================
kernel/power/Kconfig config SWSUSP_ENCRYPT · line 100
Quoted 6 lines
kernel/power/Kconfig +1
-1@@ -96,5 +96,5 @@ config SWSUSP_ENCRYPT config SUSPEND_SMP bool - depends on HOTPLUG_CPU && X86 && PM + depends on HOTPLUG_CPU && X86 && PM && X86_GENERICARCH default y
Sounds like the right approach to me, but I think it's better to use selects. I reckon that if the user selects SMP and then selects suspend support, everything else required should be automatic. If we do too many 'depends on's, they have to mess about figuring out what they haven't selected yet and why they can't find the option to suspend. Most people don't seem to know about '/' in make menuconfig.
Regards,
Nigel
Quoted 4 lines · Nigel Cunningham
I think it's better to use selects.
I reckon that if the user selects SMP and then selects suspend support, everything else required should be automatic.
Yes. `select' is end-user-friendly but kernel-developer-hostile. Sometimes it's infuriating trying to work out why a symbol keeps on getting turned on.
<checks>
hm, menuconfig's "/" command does show "Selected by:". That helps.
Hi.
Quoted 5 lines · Andrew Morton
Quoted 3 lines · Nigel Cunningham
I think it's better to use selects.
I reckon that if the user selects SMP and then selects suspend support, everything else required should be automatic.
Quoted 7 lines
Yes. `select' is end-user-friendly but kernel-developer-hostile. Sometimes it's infuriating trying to work out why a symbol keeps on getting turned on.
<checks>
hm, menuconfig's "/" command does show "Selected by:". That helps.
This might give the Kconfig guys a headache, but maybe a middle road would be to make selects turn on options rather than force them on. That is, make it so that if SUSPEND depends on SWAP, and selects it, after enabling SUSPEND, SWAP is also enabled, but you can still go to SWAP and turn it off, thereby also disabling SUSPEND again?
Regards,
Nigel
Hi Sam.
A bunch of us were discussing an issue this morning, and came across the problem that selects don't seem to be enforced in choice menus. To give a concrete example, a couple of us tried to make CONFIG_SOFTWARE_SUSPEND select CONFIG_X86_GENERICARCH. After enabling SOFTWARE_SUSPEND, we want back to the subarchitecture menu, and could still select other subarches. Is this by design?
Regards,
Nigel
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hi,
Quoted 12 lines · Sam Ravnborg
Roman is the one to addrress this.
Roman?Quoted 8 lines · Nigel Cunningham
Hi Sam.
A bunch of us were discussing an issue this morning, and came across the problem that selects don't seem to be enforced in choice menus. To give a concrete example, a couple of us tried to make
CONFIG_SOFTWARE_SUSPENDselectCONFIG_X86_GENERICARCH. After enablingSOFTWARE_SUSPEND, we want back to the subarchitecture menu, and could still select other subarches. Is this by design?
Yes, kconfig is supposed to be deterministic. Imagine two options each select a choice option, which would create an inconsistent configuration, so it's not allowed in first place.
bye, Roman
Quoted 7 lines · Nigel Cunningham
kernel/power/Kconfig config SWSUSP_ENCRYPT · line 100
Quoted 6 lines
config
SUSPEND_SMP
boolkernel/power/Kconfig +1
-1- depends on HOTPLUG_CPU && X86 && PM + depends on HOTPLUG_CPU && X86 && PM && X86_GENERICARCH default y
Quoted 7 lines
Sounds like the right approach to me, but I think it's better to use selects. I reckon that if the user selects SMP and then selects suspend support, everything else required should be automatic. If we do too many 'depends on's, they have to mess about figuring out what they haven't selected yet and why they can't find the option to suspend. Most people don't seem to know about '/' in make menuconfig.
I tried the same with HOTPLUG_CPU as well, to just say
select X86_GENERICARCH
but problem was this didnt enforce the selection, i.e user still could go and revert the selection made automatic for him, i.e go ahead and select X86_PC, and it would still leave the HOTPLUG_CPU=y around. I thought "depends" sort of forces the selection.
Maybe i didnt try correctly, if you have alternatives please do, actually even for HOTPLUG_CPU if this could be made automatic select, and at the same time enforced strictly, thats great.
(for e.g i shoud;t be able to select X86_PC=y and leave CONFIG_HOTPLUG_CPU=y around)
Signature · 3 lines
Cheers, Ashok Raj - Open Source Technology Center
Hi.
Quoted 23 lines · Ashok Raj
Quoted 6 lines · Nigel Cunningham
kernel/power/Kconfig config SWSUSP_ENCRYPT · line 100
Quoted 5 lines
config
SUSPEND_SMP
boolkernel/power/Kconfig +1
-1- depends on HOTPLUG_CPU && X86 && PM + depends on HOTPLUG_CPU && X86 && PM && X86_GENERICARCH default yQuoted 6 lines
Sounds like the right approach to me, but I think it's better to use selects. I reckon that if the user selects SMP and then selects suspend support, everything else required should be automatic. If we do too many 'depends on's, they have to mess about figuring out what they haven't selected yet and why they can't find the option to suspend. Most people don't seem to know about '/' in make menuconfig.
I tried the same with
HOTPLUG_CPUas well, to just sayselect
X86_GENERICARCHbut problem was this didnt enforce the selection, i.e user still could go and revert the selection made automatic for him, i.e go ahead and select
X86_PC, and it would still leave the HOTPLUG_CPU=y around. I thought "depends" sort of forces the selection.
Quoted 6 lines
Maybe i didnt try correctly, if you have alternatives please do, actually even for
HOTPLUG_CPUif this could be made automatic select, and at the same time enforced strictly, thats great.(for e.g i shoud;t be able to select X86_PC=y and leave
CONFIG_HOTPLUG_CPU=yaround)
I tried too, with the same results. It seems to me that the problem is that Kconfig doesn't enforce select for choices in the same way that it does for bools. Should this be filed as a bug with the Kconfig guys?
Regards,
Nigel
Quoted 3 lines
Most people don't seem to know
about '/' in make menuconfig.
I find it very confusing to use, since it returns too verbose text to skim through. Probably the search function should be split into "only search titles[*]" and "search description too".
config UNIX
bool "Unix socket" <-- that's a "title", as in menuconfig
Jan Engelhardt
menuconfig search (Re: [rfc] fix Kconfig, hotplug_cpu is needed for swsusp)
Quoted 2 lines · On Thu, 30 Mar 2006 10:24:30 +0200 (MEST) Jan Engelhardt wrote:
Quoted 2 lines
Most people don't seem to know
about '/' in make menuconfig.
That's a shame.
Quoted 3 lines
I find it very confusing to use, since it returns too verbose text to skim through. Probably the search function should be split into "only search titles[*]" and "search description too".
Re verbosity: do you know that menuconfig search (/) takes regular expressions? That could help someone limit the amount of output from it.
Can you give an example of being too verbose?
Quoted 2 lines
config UNIX
bool "Unix socket" <-- that's a "title", as in menuconfig
The search function of menuconfig (currently) works in the namespace of CONFIG_ symbols. It will search for a regex in the symbol tables. Yes, someone could modify it to search thru the prompt strings of the symbols. I don't know if that would help or not.
I have just modified menuconfig search to make displaying the
Selects: and Selected by: output be an option (actually it's a different search command (\) to not see those lines. Would that help any regarding verbosity?
The patch is here if you would care to try it:
http://www.xenotime.net/linux/patches/menuconfig-search2.patch
---
~Randy
Re: menuconfig search (Re: [rfc] fix Kconfig, hotplug_cpu is needed for swsusp)
Quoted 3 lines · Randy.Dunlap
Re verbosity: do you know that menuconfig search (/) takes regular expressions? That could help someone limit the amount of output from it.
(Inability to jump from search screen directly to just found config option to set/unset it AND verbose output) is broken. Verbose output solely is probably not a big deal.
Re: menuconfig search (Re: [rfc] fix Kconfig, hotplug_cpu is needed for swsusp)
Quoted 8 lines · On Tue, 4 Apr 2006 10:01:56 +0400 Alexey Dobriyan wrote:
Quoted 3 lines · Randy.Dunlap
Re verbosity: do you know that menuconfig search (/) takes regular expressions? That could help someone limit the amount of output from it.
(Inability to jump from search screen directly to just found config option to set/unset it AND verbose output) is broken. Verbose output solely is probably not a big deal.
I don't recall seeing that ability advertised. Yes, it would be great to have it, but I wouldn't call it broken.
---
~Randy
Re: menuconfig search (Re: [rfc] fix Kconfig, hotplug_cpu is needed for swsusp)
In reply to Randy.DunlapQuoted 6 lines
Quoted 3 lines
I find it very confusing to use, since it returns too verbose text to skim through. Probably the search function should be split into "only search titles[*]" and "search description too".
Re verbosity: do you know that menuconfig search (/) takes regular expressions?
Regexes are not mentioned in the "Help" menu.
Quoted 3 lines
That could help someone limit the amount of output from it.
Can you give an example of being too verbose?
/ UNIX
Symbol: UNIX [=y]
Prompt: Unix domain sockets
Defined at net/unix/Kconfig:5
Depends on: NET
Location:
-> Networking
-> Networking support (NET [=y])
-> Networking options Symbol: UNIXWARE_DISKSLICES ...
...
What would be more useful:
UNIX
Unix domain sockets
Networking > Networking Support > Networking options
UNIXWARE_DISKSLICES
Unixware slices support
File systems > Advanced ...
I.e. strip the Defined and Depends lines and crunch the Location lines inasfar as that the full width of the window is used (break at col 70).
DEEP_FS
Just a symbol
File systems > Advanced filesystems > Very advanced filesystems >\n Extremely advanced filesystems > Deep filesystem
Quoted 4 lines
I have just modified menuconfig search to make displaying the
Selects: and Selected by: output be an option (actually it's a different search command (\) to not see those lines. Would that help any regarding verbosity?
At the moment, entering e.g. UNIX in both the / and \ menus return the very same output. Bug?
The \ should include everything what the original / had, plus the symbol description ("Say Y here if ....").
Jan Engelhardt
Re: menuconfig search (Re: [rfc] fix Kconfig, hotplug_cpu is needed for swsusp)
Quoted 8 lines · On Sat, 8 Apr 2006 10:42:46 +0200 (MEST) Jan Engelhardt wrote:
Quoted 6 lines
Quoted 3 lines
I find it very confusing to use, since it returns too verbose text to skim through. Probably the search function should be split into "only search titles[*]" and "search description too".
Re verbosity: do you know that menuconfig search (/) takes regular expressions?
Regexes are not mentioned in the "Help" menu.
It is mentioned in the Help text after one enters '/'.
│ Search for CONFIG_ symbols and display their relations. │ │ Regular expressions are allowed. │ │ Example: search for "^FOO" │
Quoted 23 lines
Quoted 3 lines
That could help someone limit the amount of output from it.
Can you give an example of being too verbose?/ UNIX
Symbol: UNIX [=y]
Prompt: Unix domain sockets
Defined atnet/unix/Kconfig:5
Depends on: NET
Location:-> Networking -> Networking support (NET [=y]) -> Networking optionsSymbol:
UNIXWARE_DISKSLICES...
...What would be more useful:
UNIX
Unix domain sockets
Networking > Networking Support > Networking options
Quoted 6 lines
UNIXWARE_DISKSLICES
Unixware slices support
File systems > Advanced ...I.e. strip the Defined and Depends lines and crunch the Location lines inasfar as that the full width of the window is used (break at col 70).
I don't see any need to limit it to 70 columns wide. It knows how to scroll left/right (using arrow keys).
Quoted 11 lines
DEEP_FS
Just a symbol
File systems > Advanced filesystems > Very advanced filesystems >\n Extremely advanced filesystems > Deep filesystemQuoted 4 lines
I have just modified menuconfig search to make displaying the
Selects: and Selected by: output be an option (actually it's a different search command (\) to not see those lines. Would that help any regarding verbosity?
Quoted 5 lines
At the moment, entering e.g. UNIX in both the / and \ menus return the very same output. Bug?
The \ should include everything what the original / had, plus the symbol description ("Say Y here if ....").
Nope, the only thing that the "reduced" search excludes atm is SELECT info.
Quoted 1 line
From my original email, as you quoted above, the less-verbose '\' search
only eliminates the Selects: and Selected by: output, and the UNIX config symbols don't have any SELECT info.
I don't know if we are converging any, but I made a new patch:
http://www.xenotime.net/linux/patches/menuconfig-search2b.patch
This one changes the default '/' search to NOT be verbose and the new, extended '\' search to be verbose. The non-verbose search omits Selects:, Selected by:, and Location: information.
---
~Randy
Jan Engelhardt
Acked-byRe: menuconfig search (Re: [rfc] fix Kconfig, hotplug_cpu is needed for swsusp)
Quoted 7 lines
Quoted 3 lines
I.e. strip the Defined and Depends lines and crunch the Location lines inasfar as that the full width of the window is used (break at col 70).
I don't see any need to limit it to 70 columns wide. It knows how to scroll left/right (using arrow keys).
Not because it does not know how to horizontally scroll, but because it takes quite some time for the user to scroll left/right. There's no Home or End key for the horizontal direction.
Quoted 2 lines
I don't know if we are converging any, but I made a new patch:
http://www.xenotime.net/linux/patches/menuconfig-search2b.patch
Yes, this one looks good and satisfies me.
Jan Engelhardt
Hi!
Quoted 10 lines
Quoted 3 lines
So if you have a single core x86, you want
X86_PC, and if you have HT or SMP, you want GENERICARCH? If so, could this be done via selects or depends or at least defaults inKconfig?Yes, i think only
SUSPEND_SMPis affect by this. I thought Rafael cced Pavel during that exchange, maybe i missed.Quoted 3 lines
Regards,
Quoted 8 lines
Quoted 1 line
Nigel
How about this patch.
Make
SUSPEND_SMPdepend onX86_GENERICARCH, since hotplug cpu requires !X86_PC due to some race in IPI handling. See more discussion herehttp://marc.theaimsgroup.com/?l=linux-kernel&m=114303306032338&w=2
I can't see useful discussion there.
kernel/power/Kconfig config SWSUSP_ENCRYPT · line 100
Quoted 11 lines
Index: linux-2.6.16-git16/kernel/power/Kconfig
===================================================================kernel/power/Kconfig +1
-1@@ -96,5 +96,5 @@ config SWSUSP_ENCRYPT config SUSPEND_SMP bool - depends on HOTPLUG_CPU && X86 && PM + depends on HOTPLUG_CPU && X86 && PM && X86_GENERICARCH default y
Heh, great, so one more magic option that is required.
Plus GENERICARCH does not sound like something normal users would enable:
config X86_GENERICARCH
bool "Generic architecture (Summit, bigsmp, ES7000, default)" depends on SMP
help
This option compiles in the Summit, bigsmp, ES7000, default subarchitectures. It is intended for a generic binary kernel.
(What does "default" mean there, anyway? X86_PC?)
Pavel
Signature · 1 line
Picture of sleeping (Linux) penguin wanted...
Hi,
Quoted 5 lines · Pavel Machek
Quoted 5 lines
Quoted 3 lines
So if you have a single core x86, you want
X86_PC, and if you have HT or SMP, you want GENERICARCH? If so, could this be done via selects or depends or at least defaults inKconfig?Yes, i think only
SUSPEND_SMPis affect by this.
That's correct.
Quoted 1 line
Quoted 1 line
I thought Rafael cced Pavel during that exchange, maybe i missed.
I did, but Pavel was travelling at that time (I think).
[Well, I had the feeling it would cause problems but unfortunately I couldn't show that.]
Quoted 15 lines
Quoted 6 lines
How about this patch.
Make
SUSPEND_SMPdepend onX86_GENERICARCH, since hotplug cpu requires !X86_PC due to some race in IPI handling. See more discussion herehttp://marc.theaimsgroup.com/?l=linux-kernel&m=114303306032338&w=2
I can't see useful discussion there.
Quoted 6 lines
Index: linux-2.6.16-git16/kernel/power/Kconfig
===================================================================kernel/power/Kconfig +0
-0@@ -96,5 +96,5 @@ config SWSUSP_ENCRYPT
Quoted 8 lines
kernel/power/Kconfig config SWSUSP_ENCRYPT · line 100
Quoted 5 lines
kernel/power/Kconfig +1
-1config SUSPEND_SMP bool - depends on HOTPLUG_CPU && X86 && PM + depends on HOTPLUG_CPU && X86 && PM && X86_GENERICARCH default yHeh, great, so one more magic option that is required.
BTW it's i386-centric, as X86_GENERICARCH is not defined on x86_64, for example.
FUrther, the whole problem, as far as I can understand it, is i386-specific, so it should be resolved in the i386 architecture config rather than anywhere else.
Greetings,
Rafael
Hi!
Quoted 3 lines
Quoted 1 line
Quoted 1 line
I thought Rafael cced Pavel during that exchange, maybe i missed.
I did, but Pavel was travelling at that time (I think).
Yes, I was cc-ed, but only watching discussion by single eye...
Quoted 3 lines
FUrther, the whole problem, as far as I can understand it, is i386-specific, so it should be resolved in the i386 architecture config rather than anywhere else.
Or maybe in i386 .c files :-). Could we just switch to BIGSMP mode by default? Intel claims it has no performance disadvatage, and distros want suspend, anyway...
Pavel
Signature · 1 line
Thanks, Sharp!
Quoted 4 lines · Pavel Machek
Or maybe in i386 .c files :-). Could we just switch to BIGSMP mode by default? Intel claims it has no performance disadvatage, and distros want suspend, anyway...
After some more thought and consulting with Pavel and folks, the path of least resistance for now seems like not to switch to bigsmp.
We will need to do a little bit more work to ensure that we can work the physical flat mode (bigsmp) instead of logical flat mode in i386. Primarily to ensure configs for SMP_SUSPEND, any memory hotplug prototype in i386 dont break by choosing bigsmp as default to ensure transition is smooth.
Andrew, please help queuing this patch. (Since pavel just reverted the HOTPLUG_CPU depends on X86_GENERICARCH recently in git9, people will now see the funky error message that they have more than 8 cpus until this patch is included.)
Signature · 63 lines
Cheers, Ashok Raj - Open Source Technology Center Switching to automatic bigsmp causes an misleading error message, that more then 8 cpus are detected, and user needs to select either X86_GENERICARCH or X86_BIGSMP to handle. Reason is we switched to bigsmp to avoid IP race when new cpu is comming up. [bigsmp is nothing but using physical flat mode that can work for 1 .. 255 cpus] [default is X86_PC, that uses logical flat mode up to 8 CPUs max] Current x86_64 code uses bigsmp as default when hotplug is enabled. It would be preferable to make bigsmp as default, and work the dependencies of other related code like SMP_SUSPEND, and some related to memory hotplug code for i386. Current logical flat mode doesnt use shortcuts that cause the race by using the send_IPI_mask() instead of shortcuts when HOTPLUG_CPU is enabled. In the meantime this patch is the path of lease resistance. We will switch to bigsmp default sometime soon, when we get to work it again. Signed-off-by: Ashok Raj <[email protected]> ----------------------------------------------------------- arch/i386/kernel/mpparse.c | 15 ++++++++------- 1 files changed, 8 insertions(+), 7 deletions(-) Index: linux-2.6.16-git19/arch/i386/kernel/mpparse.c =================================================================== --- linux-2.6.16-git19.orig/arch/i386/kernel/mpparse.c +++ linux-2.6.16-git19/arch/i386/kernel/mpparse.c @@ -38,12 +38,6 @@ int smp_found_config; unsigned int __initdata maxcpus = NR_CPUS; -#ifdef CONFIG_HOTPLUG_CPU -#define CPU_HOTPLUG_ENABLED (1) -#else -#define CPU_HOTPLUG_ENABLED (0) -#endif - /* * Various Linux-internal data structures created from the * MP-table. @@ -225,7 +219,14 @@ static void __devinit MP_processor_info cpu_set(num_processors, cpu_possible_map); num_processors++; - if (CPU_HOTPLUG_ENABLED || (num_processors > 8)) { + /* + * Would be preferable to switch to bigsmp when CONFIG_HOTPLUG_CPU=y + * but we need to work other dependencies like SMP_SUSPEND etc + * before this can be done without some confusion. + * if (CPU_HOTPLUG_ENABLED || num_processors > 8) + * - Ashok Raj <[email protected]> + */ + if (num_processors > 8) { switch (boot_cpu_data.x86_vendor) { case X86_VENDOR_INTEL: if (!APIC_XAPIC(ver)) {
Quoted 18 lines · Ashok Raj
Quoted 7 lines · Andrew Morton
Quoted 4 lines · Pavel Machek
HOTPLUG_CPUis needed on normal PCs, too -- it is neccessary for software suspend.OK, this will get ugly. APICs are involved.
I guess you need only on systems that support >1 cpu right? I doubt you will need it on a system that cannot run with the config-generic-arch on. although we use bigsmp when hotplug is turned on, all we really end up is using flat physical mode instead of using logical mode.
I still havent understood why this wont work. Choosing
CONFIG_X86_GENERICARCHshouldnt break anything AFAICT.
Quoted 8 lines
Pavel, you could use
CONFIG_HOTPLUG_CPU, just need to enableX86_GENERICARCHnow. Is there a reason you think that wont work? I wish we would revert it for a strong reason that we know will not make hotplug work on certain systems because of this choise not that we currently haveX86_PCnow, and are unwiling to change the config.(PS: the word bigsmp although sounds like some large
NR_CPUS, its just using a mode that permits the system to work from 1 .. >8 cpus. So there is really nothing determental to selecting this.)
Something which remains to be beaten into my head: *why* does HOTPLUG_CPU require flat pyhsical mode? What necessitated that change, and cannot we make it work OK in logical mode as well as flat mode?
On Wed, Mar 29, 2006 at 07:24:53PM -0800, Andrew Morton wrote:
Hi Andrew,
here is an attempt to explain why...
Quoted 4 lines
Something which remains to be beaten into my head: *why* does
HOTPLUG_CPUrequire flat pyhsical mode? What necessitated that change, and cannot we make it work OK in logical mode as well as flat mode?
Short answer
1. Will hotplug work today in logical flat mode without selecting bigsmp?
Yes, since we avoid broadcast on i386 IPI's with no_broadcast option.
2. Why use bigsmp then?
There is no reason to do the same thing in two different ways. i.e using logical flat with no_broadcast=1, or use the bigsmp by default when hotplug cpu is enabled. IOW we wanted the handling consistent with what we do for x86_64, hence the decision to use bigsmp automatically when hotplug is enabled or we notice >8 CPUS.
Long and detailed ...
1. broadcast IPI's are not hotplug friendly since hw has no control to *not* deliver the message to offline cpus. Nice explanation from Vatsa on what the corner cases are in link below.
http://marc.theaimsgroup.com/?l=linux-kernel&m=111695485610434&w=2
2. The discussion above was to support removing shortcuts, and using a send_IPI_mask() instead of send_IPI_allbutself() via a shortcut msg. This would not deliver an IPI to offline cpus. Andi Kleen had some doubts about tlb flush performance when we use send_IPI_mask() since it does one cpu at a time.
So we added a no_broadcast option, that permits us to continue to use the normal shortcut method for send_IPI_allbutself() when cpu-hotplug is disabled, and use the send_IPI_mask() version when cpu-hotplug is enabled.
[ set usage of no_broadcast in arch/i386/mach-default/setup.c and used in include/asm-i386/mach-default/mach_ipi.h]
In addition we also posted some perf testing patch for testing, and it proved we didnt have any perf degradation for using the mask or the shortcut. Based on those results, Andi removed the no_broadcast option in X86_64 and we used the physflat option since it does the unicast method anyway.
[Physflat mode is equivalent of bigsmp in x86_64]
3. So today using logical flat mode will work with hotlug without any races due to the no_broadcast=1 is automatically selected when hotplug cpu is enabled, even without bigsmp apic mode.
So Why we moved to automatically selecting bigsmp when hotlug was enabled?
we really didnt want 2 ways to do the same thing without clear benefit for one over other. (i.e using logical flat mode with no_broadcast v.s using bigsmp that already does send_IPI_sequence() which doesnt do the shortcut msg )
4. Why bigsmp was converted to physical flat mode instead of logical cluster mode?
We discovered a problem when platform/chipset is working in logical cluster mode, and a new CPU is brought up that causes bad interaction. More details below in the patch description.
http://marc.theaimsgroup.com/?l=linux-kernel&m=113261865814107&w=2
[In i386 bigsmp also means flat physical mode after the above patch was accepted]
5. Ideally the no_broadcast option will be removed soon once things settle down.
Hope this helps.
Signature · 3 lines
Cheers, Ashok Raj - Open Source Technology Center
Hi!
Quoted 11 lines
here is an attempt to explain why...
Quoted 4 lines
Something which remains to be beaten into my head: *why* does
HOTPLUG_CPUrequire flat pyhsical mode? What necessitated that change, and cannot we make it work OK in logical mode as well as flat mode?Short answer
1. Will hotplug work today in logical flat mode without selecting bigsmp?
Quoted 8 lines
Yes, since we avoid broadcast on i386 IPI's with
no_broadcastoption.2. Why use bigsmp then?
There is no reason to do the same thing in two different ways. i.e using logical flat with no_broadcast=1, or use the bigsmp by default when hotplug cpu is enabled. IOW we wanted the handling consistent with what we do forx86_64, hence the decision to use bigsmp automatically when hotplug is enabled or
we notice >8 CPUS.
Why not make it even more consistent and just use bigsmp mode by default, even for X86_PC? X86_BIGSMP would be reduntant, then...
Pavel
Signature · 1 line
Picture of sleeping (Linux) penguin wanted...
Quoted 3 lines · Andrew Morton
Something which remains to be beaten into my head: *why* does
HOTPLUG_CPUrequire flat pyhsical mode? What necessitated that change, and cannot we make it work OK in logical mode as well as flat mode?
Good question.
There is a window after a cpu starts and before we initialize it that we must be very careful and not deliver it any interrupts because we don't know what the cpu will do.
Which means in the presence of cpu hotplug we can never use broadcast interrupts.
I don't know how the that disallows logical mode though.
Eric
Quoted 9 lines · Ashok Raj
Quoted 7 lines · Andrew Morton
Quoted 4 lines · Pavel Machek
HOTPLUG_CPUis needed on normal PCs, too -- it is neccessary for software suspend.OK, this will get ugly. APICs are involved.
Quoted 7 lines
I guess you need only on systems that support >1 cpu right? I doubt you will need it on a system that cannot run with the config-generic-arch on. although we use bigsmp when hotplug is turned on, all we really end up is using flat physical mode instead of using logical mode.
I still havent understood why this wont work. Choosing
CONFIG_X86_GENERICARCHshouldnt break anything AFAICT.
If X86_GENERICARCH is not breaking anything, why does X86_PC exist? If you say X86_GENERICARCH is okay for everyone, at least help texts need to be fixed.
config X86_PC
bool "PC-compatible"
help
Choose this option if your computer is a standard PC or compatible.
config X86_BIGSMP
bool "Support for other sub-arch SMP systems with more than 8 CPUs" depends on SMP
help
This option is needed for the systems that have more than 8 CPUs and if the system is not of any sub-arch type above.
If you don't have such a system, you should say N here.
I certainly have PCtype machine here, and definitely not more than 8 CPUs.
Quoted 8 lines
Pavel, you could use
CONFIG_HOTPLUG_CPU, just need to enableX86_GENERICARCHnow. Is there a reason you think that wont work? I wish we would revert it for a strong reason that we know will not make hotplug work on certain systems because of this choise not that we currently haveX86_PCnow, and are unwiling to change the config.(PS: the word bigsmp although sounds like some large
NR_CPUS, its just using a mode that permits the system to work from 1 .. >8 cpus. So there is really nothing determental to selecting this.)
At least config texts need to be fixed if you want people to use X86_BIGSMP. And why keep X86_PC when X86_BIGSMP works as well?
Pavel
Signature · 1 line
Picture of sleeping (Linux) penguin wanted...