ARM: do not mark CPU 0 as hotpluggable
PATCH 1/1 ARM: do not mark CPU 0 as hotpluggable
A quick poll of the ARM platforms that implement CPU Hotplug support shows that every platform treats CPU 0 as a special case that cannot be hotplugged. In fact every platform has identical code for platform_cpu_die which returns -EPERM in the case of CPU 0.
The user-facing sysfs interfaces should reflect this by not populating an 'online' entry for CPU 0 at all. This better reflects reality by making it clear to users that CPU 0 cannot be hotplugged.
This patch prevents CPU 0 from being marked as hotpluggable on all ARM platforms during CPU registration. This in turn prevents the creation of an 'online' sysfs interface for that CPU.
--- arch/arm/kernel/setup.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)
arch/arm/kernel/setup.c +2 -1
@@ -940,7 +940,8 @@ static int __init topology_init(void)for_each_possible_cpu (cpu) {struct cpuinfo_arm *cpuinfo = &per_cpu (cpu_data, cpu); - cpuinfo->cpu.hotpluggable =1 ; +if (cpu) + cpuinfo->cpu.hotpluggable =1 ;register_cpu (&cpuinfo->cpu, cpu); }
Signature · 1 line
1.7.4.1
PATCH 1/1 ARM: do not mark CPU 0 as hotpluggable
Quoted 5 lines · Mike Turquette
A quick poll of the ARM platforms that implement CPU Hotplug support shows that every platform treats CPU 0 as a special case that cannot be hotplugged. In fact every platform has identical code for
platform_cpu_diewhich returns-EPERMin the case of CPU 0.
Quoted 8 lines
The user-facing sysfs interfaces should reflect this by not populating an 'online' entry for CPU 0 at all. This better reflects reality by making it clear to users that CPU 0 cannot be hotplugged.
This patch prevents CPU 0 from being marked as hotpluggable on all ARM platforms during CPU registration. This in turn prevents the creation of an 'online' sysfs interface for that CPU.
Unless there is a kernel limitation why CPU0 can't be hot unplugged, then this should remain a platform decision. This may be another case of everybody just copying other platforms' code, not a platform limitation.
Rob
Quoted 19 lines
Signed-off-by Mike Turquette <[email protected]>--- arch/arm/kernel/setup.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)arch/arm/kernel/setup.c +2
-1@@ -940,7 +940,8 @@ static int __init topology_init(void)for_each_possible_cpu (cpu) {struct cpuinfo_arm *cpuinfo = &per_cpu (cpu_data, cpu); - cpuinfo->cpu.hotpluggable =1 ; +if (cpu) + cpuinfo->cpu.hotpluggable =1 ;register_cpu (&cpuinfo->cpu, cpu); }
PATCH 1/1 ARM: do not mark CPU 0 as hotpluggable
Quoted 10 lines · Rob Herring
Quoted 9 lines · Mike Turquette
A quick poll of the ARM platforms that implement CPU Hotplug support shows that every platform treats CPU 0 as a special case that cannot be hotplugged. In fact every platform has identical code for
platform_cpu_diewhich returns-EPERMin the case of CPU 0.The user-facing sysfs interfaces should reflect this by not populating an 'online' entry for CPU 0 at all. This better reflects reality by making it clear to users that CPU 0 cannot be hotplugged.
Quoted 8 lines
Quoted 4 lines
This patch prevents CPU 0 from being marked as hotpluggable on all ARM platforms during CPU registration. This in turn prevents the creation of an 'online' sysfs interface for that CPU.
Unless there is a kernel limitation why CPU0 can't be hot unplugged, then this should remain a platform decision. This may be another case of everybody just copying other platforms' code, not a platform limitation.
Just talking on behalf of OMAP, we can't offline CPU0 and limitation would remain in future OMAPs too.
Regards
Santosh
PATCH 1/1 ARM: do not mark CPU 0 as hotpluggable
Quoted 2 lines · Santosh Shilimkar
Just talking on behalf of OMAP, we can't offline CPU0 and limitation would remain in future OMAPs too.
Apart from the broken IRQ migration, and the way CPU0 immediately reawakes if it is offlined on OMAP4 (even when IRQs are migrated off CPU0) because omap_read_auxcoreboot0() returns 0, is there any other reason?
With fixed IRQ migration and forcing CPU0 into an infinite WFI loop, I can offline CPU0 and still have a running system.
PATCH 1/1 ARM: do not mark CPU 0 as hotpluggable
Quoted 4 lines · Russell King - ARM Linux
Quoted 2 lines · Santosh Shilimkar
Just talking on behalf of OMAP, we can't offline CPU0 and limitation would remain in future OMAPs too.
Quoted 8 lines
Apart from the broken IRQ migration, and the way CPU0 immediately reawakes if it is offlined on OMAP4 (even when IRQs are migrated off CPU0) because
omap_read_auxcoreboot0()returns 0, is there any other reason?With fixed IRQ migration and forcing CPU0 into an infinite WFI loop, I can offline CPU0 and still have a running system.
The secure software runs only on CPU0 and that's the biggest limitation. Other issues like hand-shake as you pointed out, power sequencing etc can be worked around.
Regards
Santosh
PATCH 1/1 ARM: do not mark CPU 0 as hotpluggable
Quoted 10 lines
From: linux-arm-kernel-bounces at lists.infradead.org [mailto:linux-arm- kernel-bounces at lists.infradead.org] On Behalf Of Shilimkar, Santosh
Quoted 3 lines
With fixed IRQ migration and forcing CPU0 into an infinite WFI loop, I can offline CPU0 and still have a running system.
The secure software runs only on CPU0 and that's the biggest limitation. Other issues like hand-shake as you pointed out, power sequencing etc can be worked around.
As you know well some of the secure APIs work on CPU1 and others do not.
I notice in other thread Russell made assumption about CPU1 being able to use all because it could run some. This is not the case.
I'll re-add because of the many power states omap4 adds atop of standard 3 arm states there exists a lot of states and not all state transitions are valid. This will increase complexity of practical usage.
PATCH 1/1 ARM: do not mark CPU 0 as hotpluggable
Quoted 7 lines · Richard
Quoted 7 lines
From: linux-arm-kernel-bounces at lists.infradead.org [mailto:linux-arm- kernel-bounces at lists.infradead.org] On Behalf Of Shilimkar, Santosh
Quoted 3 lines
With fixed IRQ migration and forcing CPU0 into an infinite WFI loop, I can offline CPU0 and still have a running system.
Quoted 8 lines
Quoted 3 lines
The secure software runs only on CPU0 and that's the biggest limitation. Other issues like hand-shake as you pointed out, power sequencing etc can be worked around.
As you know well some of the secure APIs work on CPU1 and others do not.
I notice in other thread Russell made assumption about CPU1 being able to use all because it could run some. This is not the case.
I clarified that on the other thread.
Regards
Santosh
PATCH 1/1 ARM: do not mark CPU 0 as hotpluggable
Quoted 12 lines · Santosh Shilimkar
Quoted 11 lines · Richard
Quoted 10 lines
From: linux-arm-kernel-bounces at lists.infradead.org [mailto:linux-arm- kernel-bounces at lists.infradead.org] On Behalf Of Shilimkar, Santosh
Quoted 3 lines
With fixed IRQ migration and forcing CPU0 into an infinite WFI loop, I can offline CPU0 and still have a running system.
The secure software runs only on CPU0 and that's the biggest limitation. Other issues like hand-shake as you pointed out, power sequencing etc can be worked around.
Quoted 6 lines
Quoted 5 lines
As you know well some of the secure APIs work on CPU1 and others do not.
I notice in other thread Russell made assumption about CPU1 being able to use all because it could run some. This is not the case.
I clarified that on the other thread.
I've asked a few other ARM folks (out of band) to weigh in on this thread to determine if their platform has similar limitations as OMAP.
Unfortunately no one else has responded.
Still the limitation for OMAP remains. Earlier in this thread I provided an alternative to blacklisting CPU0 for all ARM platforms by instead using a config option, but it received no comments. What is the best way to move forward?
Thanks,
Mike
Quoted 3 lines
Regards
Santosh
PATCH 1/1 ARM: do not mark CPU 0 as hotpluggable
In reply to Mike TurquetteQuoted 4 lines · Mike Turquette
A quick poll of the ARM platforms that implement CPU Hotplug support shows that every platform treats CPU 0 as a special case that cannot be hotplugged. In fact every platform has identical code for
platform_cpu_diewhich returns-EPERMin the case of CPU 0.
Are you sure that's just not because everyone copied what Realview has been doing (highly likely)?
I suspect that there's no reason that CPU0 can't be taken down, especially on those platforms which don't take the CPU fully offline but just put it into a WFI loop.
Those which restart the CPUs through the boot loader probably detect CPU0 as the boot CPU, so they probably can't take CPU0 down.
PATCH 1/1 ARM: do not mark CPU 0 as hotpluggable
Quoted 13 lines
From: linux-arm-kernel-bounces at lists.infradead.org [mailto:linux-arm- kernel-bounces at lists.infradead.org] On Behalf Of Russell King - ARM Linux
Quoted 4 lines · Mike Turquette
A quick poll of the ARM platforms that implement CPU Hotplug support shows that every platform treats CPU 0 as a special case that cannot be hotplugged. In fact every platform has identical code for
platform_cpu_diewhich returns-EPERMin the case of CPU 0.Are you sure that's just not because everyone copied what Realview has been doing (highly likely)?
Quoted 6 lines
I suspect that there's no reason that CPU0 can't be taken down, especially on those platforms which don't take the CPU fully offline but just put it into a WFI loop.
Those which restart the CPUs through the boot loader probably detect CPU0 as the boot CPU, so they probably can't take CPU0 down.
This was a hot topic a couple years back for hw folks...
On 443x when you go to ARM-MPCore power states other than normal (dormant, powered-off) there are hardware and ti-trustzone-monitor constraints which dictate the CPU power state transition matrix (cpu1 can't make all calls cpu0 can and some hardware combinations were not validated).
For MPU-domain (mpcore-cluster + l2cache ++) and its sub-domains (cpu0, cpu1, ...) there exists ability to set multiple states which wfi is a gate into (on, inactive, retention, partial-off, off). A big matrix of possible states results. Things were coded and system tested for only a subset of these. The ARM code did line up with these constraints.
Maybe an offline to simple wfi loop is ok, but other lower states could not be entered from that state.
Some folks were talking about relaxing cpu0/cpu1 constraints moving forward. Doing this could result in many more states-combinations to be validated at hardware level. Do you think Linux code would be simplified/enhanced such that it is worth the extra hardware validation effort?
PATCH 1/1 ARM: do not mark CPU 0 as hotpluggable
In reply to Russell King - ARM LinuxQuoted 8 lines · Russell King - ARM Linux
Quoted 4 lines · Mike Turquette
A quick poll of the ARM platforms that implement CPU Hotplug support shows that every platform treats CPU 0 as a special case that cannot be hotplugged. ?In fact every platform has identical code for
platform_cpu_diewhich returns-EPERMin the case of CPU 0.Are you sure that's just not because everyone copied what Realview has been doing (highly likely)?
Copy/paste is always likely. Would be nice for other platform folks to weigh in on this.
Quoted 6 lines
I suspect that there's no reason that CPU0 can't be taken down, especially on those platforms which don't take the CPU fully offline but just put it into a WFI loop.
Those which restart the CPUs through the boot loader probably detect CPU0 as the boot CPU, so they probably can't take CPU0 down.
OMAP does seem to have this limitation and Santosh/Richard can provide much better details than me in the parallel thread. Again, would be nice to hear if other platforms have similar limitations from their stakeholders.
The idea here is to not mark a CPU hotpluggable if it cannot be hotplugged. If the limitations turn out to be legitimate for some platforms but not others, what's the best way to handle it? Either move the topology initialization to platform code or allow platforms to set some config option. Patches for the latter are below, but I think the current discussion on whether or not other platforms can actually hotplug CPU0 should run its course before considering below patches too seriously.