multi-cluster power management
Subsystems (6)
v4 cover multi-cluster power management
This is version 4 of the patch series required to safely power up and down CPUs in a cluster as can be found in b.L systems. If nothing major is reported, I'll send a pull request to Russell for this set very soon.
Also included are the needed patches to allow CPU hotplug on RTSM configured as a big.LITTLE system. I'll hopefully send a separate pull request for them later if the remaining issue with the CCI code can be sorted out in time.
Please refer to http://article.gmane.org/gmane.linux.ports.arm.kernel/208625
for the initial series and particularly the cover page blurb for this work.
Thanks to those who provided review comments.
Changes from v3:
- DT binding documentation for the DCSCB.
- Patch to probe DT for DCSCB presence folded into the patch that introduced it.
- Patch to select SMP ops at boot time now split in two patches.
- Use all 8 bits from the MPIDR affinity fields in mcpm_head.S.
- Replaced gic_raise_softirq() with arch_send_wakeup_ipi_mask().
- Clear CTRL.C bit rather than calling cpu_proc_fin() to keep the I-cache enabled.
- Added definitions in the CCI code to clean up some magic values.
Changes from v2:
- The bL_ prefix has been changed into mcpm_ and surroundings adjusted accordingly.
- Documentation moved up one level in Documentation/arm/.
- Clarifications in commit log for patch #1 about future work.
- The debug macro in mcpm_head.S now displays CPU and cluster numbers.
- Patch improving mcpm_cpu_die() folded into the original patch that created it.
- Return -EADDRNOTAVAIL on ioremap failure.
- The auxcr patch moved down in the series to better identify dependencies.
Changes from v1:
- Pulled in Rob Herring's auxcr accessor patch and converted this series to it.
- VMajor rework of various barriers (some DSBs demoted to DMBs, etc.)
- The sync_mem() macro is now split and enhanced to properly process the cache for writers and readers in the cluster critical region helpers.
- BL_NR_CLUSTERS and BL_CPUS_PER_CLUSTER renamed to BL_MAX_CLUSTERS and BL_MAX_CPUS_PER_CLUSTER.
- Removed unused C definitions and prototypes for vlocks.
- Simplified the vlock memory allocation.
- The vlock code is GPL v2.
- Replaced MPIDR inline asm by read_cpuid_mpidr().
- Use of MPIDR_AFFINITY_LEVEL() to replace explicit shifts and masks.
- Dropped gic_cpu_if_down().
- Added a DSB before SEV and WFI.
- Fixed power_up_setup helper prototype.
- Nuked smp_wmb() in bL_set_entry_vector().
- Moved the CCI driver to drivers/bus/.
- Dependency on CONFIG_EXPERIMENTAL removed.
- Leftover garbage in Makefile removed.
- Added/clarified various comments in the assembly code.
- Some documentation typos fixed.
- Copyright notices updated to 2013
Still not addressed yet in this series:
- The CCI device tree binding descriptions.
Diffstat:
Documentation/arm/cluster-pm-race-avoidance.txt | 498 ++++++++++++++++++ Documentation/arm/vlocks.txt | 211 ++++++++ .../devicetree/bindings/arm/rtsm-dcscb.txt | 19 + arch/arm/Kconfig | 8 + arch/arm/common/Makefile | 1 + arch/arm/common/mcpm_entry.c | 314 +++++++++++ arch/arm/common/mcpm_head.S | 219 ++++++++ arch/arm/common/mcpm_platsmp.c | 86 +++ arch/arm/common/vlock.S | 108 ++++ arch/arm/common/vlock.h | 29 + arch/arm/include/asm/cp15.h | 14 + arch/arm/include/asm/mach/arch.h | 3 + arch/arm/include/asm/mcpm_entry.h | 190 +++++++ arch/arm/kernel/setup.c | 5 +- arch/arm/mach-vexpress/Kconfig | 9 + arch/arm/mach-vexpress/Makefile | 1 + arch/arm/mach-vexpress/core.h | 2 + arch/arm/mach-vexpress/dcscb.c | 249 +++++++++ arch/arm/mach-vexpress/dcscb_setup.S | 80 +++ arch/arm/mach-vexpress/platsmp.c | 12 + arch/arm/mach-vexpress/v2m.c | 2 +- drivers/bus/Kconfig | 5 + drivers/bus/Makefile | 2 + drivers/bus/arm-cci.c | 125 +++++ drivers/cpuidle/cpuidle-calxeda.c | 14 - include/linux/arm-cci.h | 30 ++ 26 files changed, 2220 insertions(+), 16 deletions(-)
PATCH v4 1/15 ARM: multi-cluster PM: secondary kernel entry code
CPUs in cluster based systems, such as big.LITTLE, have special needs when entering the kernel due to a hotplug event, or when resuming from a deep sleep mode.
This is vectorized so multiple CPUs can enter the kernel in parallel without serialization.
The mcpm prefix stands for "multi cluster power management", however this is usable on single cluster systems as well. Only the basic structure is introduced here. This will be extended with later patches.
In order not to complexify things more than they currently have to, the planned work to make runtime adjusted MPIDR based indexing and dynamic memory allocation for cluster states is postponed to a later cycle. The MAX_NR_CLUSTERS and MAX_CPUS_PER_CLUSTER static definitions should be sufficient for those systems expected to be available in the near future.
--- arch/arm/Kconfig | 8 ++++ arch/arm/common/Makefile | 1 + arch/arm/common/mcpm_entry.c | 29 +++++++++++++ arch/arm/common/mcpm_head.S | 86 +++++++++++++++++++++++++++++++++++++++ arch/arm/include/asm/mcpm_entry.h | 35 ++++++++++++++++ 5 files changed, 159 insertions(+) create mode 100644 arch/arm/common/mcpm_entry.c create mode 100644 arch/arm/common/mcpm_head.S create mode 100644 arch/arm/include/asm/mcpm_entry.h
arch/arm/Kconfig +8 -0
@@ -1584,6 +1584,14 @@ config HAVE_ARM_TWD help This options enables support for the ARM timer and watchdog unit +config CLUSTER_PM + bool "Cluster Power Management Infrastructure" + depends on CPU_V7 && SMP + help + This option provides the common power management infrastructure + for (multi-)cluster based systems, such as big.LITTLE based + systems. + choice prompt "Memory split" default VMSPLIT_3G
arch/arm/common/Makefile +1 -0
@@ -13,3 +13,4 @@ obj-$(CONFIG_SHARP_PARAM) += sharpsl_param.o obj-$(CONFIG_SHARP_SCOOP) += scoop.o obj-$(CONFIG_PCI_HOST_ITE8152) += it8152.o obj-$(CONFIG_ARM_TIMER_SP804) += timer-sp.o +obj-$(CONFIG_CLUSTER_PM) += mcpm_head.o mcpm_entry.o
arch/arm/common/mcpm_entry.c new file +29 -0
@@ -0,0 +1,29 @@ +/* +* arch/arm/common/mcpm_entry.c -- entry point for multi-cluster PM +* +* Created by: Nicolas Pitre, March 2012 +* Copyright: (C) 2012-2013 Linaro Limited +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License version 2 as +* published by the Free Software Foundation. +*/ + +#include <linux/kernel.h> +#include <linux/init.h> + +#include <asm/mcpm_entry.h> +#include <asm/barrier.h> +#include <asm/proc-fns.h> +#include <asm/cacheflush.h> + +extern volatile unsigned long mcpm_entry_vectors[MAX_NR_CLUSTERS ][MAX_CPUS_PER_CLUSTER ]; + +void mcpm_set_entry_vector (unsigned cpu,unsigned cluster,void *ptr) +{ +unsigned long val = ptr ?virt_to_phys (ptr) :0 ; + mcpm_entry_vectors[cluster][cpu] = val; +__cpuc_flush_dcache_area ((void *)&mcpm_entry_vectors[cluster][cpu],4 ); +outer_clean_range (__pa (&mcpm_entry_vectors[cluster][cpu]), +__pa (&mcpm_entry_vectors[cluster][cpu +1 ])); +}
arch/arm/common/mcpm_head.S new file +86 -0
86 lines not on this page · raw message
arch/arm/include/asm/mcpm_entry.h new file +35 -0
@@ -0,0 +1,35 @@ +/* +* arch/arm/include/asm/mcpm_entry.h +* +* Created by: Nicolas Pitre, April 2012 +* Copyright: (C) 2012-2013 Linaro Limited +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License version 2 as +* published by the Free Software Foundation. +*/ + +#ifndef MCPM_ENTRY_H +#define MCPM_ENTRY_H + +#define MAX_CPUS_PER_CLUSTER 4 +#define MAX_NR_CLUSTERS 2 + +#ifndef __ASSEMBLY__ + +/* +* Platform specific code should use this symbol to set up secondary +* entry location for processors to use when released from reset. +*/ +extern void mcpm_entry_point (void ); + +/* +* This is used to indicate where the given CPU from given cluster should +* branch once it is ready to re-enter the kernel using ptr, or NULL if it +* should be gated. A gated CPU is held in a WFE loop until its vector +* becomes non NULL. +*/ +void mcpm_set_entry_vector (unsigned cpu,unsigned cluster,void *ptr); + +#endif /* ! __ASSEMBLY__ */ +#endif
Signature · 1 line
1.8.1.2
PATCH v4 1/15 ARM: multi-cluster PM: secondary kernel entry code
arch/arm/common/mcpm_entry.c line 20
Quoted 6 lines · Nicolas Pitre
arch/arm/common/mcpm_entry.c new file +6
-0+#include <asm/mcpm_entry.h> +#include <asm/barrier.h> +#include <asm/proc-fns.h> +#include <asm/cacheflush.h> + +extern volatile unsigned long mcpm_entry_vectors[MAX_NR_CLUSTERS ][MAX_CPUS_PER_CLUSTER ];
This should not be volatile. You should know by now the stance in the Linux community against using volatile on data declarations. See Documentation/volatile-considered-harmful.txt to remind yourself of the reasoning.
arch/arm/common/mcpm_entry.c line 28
Quoted 8 lines
arch/arm/common/mcpm_entry.c new file +8
-0+ +void mcpm_set_entry_vector (unsigned cpu,unsigned cluster,void *ptr) +{ +unsigned long val = ptr ?virt_to_phys (ptr) :0 ; + mcpm_entry_vectors[cluster][cpu] = val; +__cpuc_flush_dcache_area ((void *)&mcpm_entry_vectors[cluster][cpu],4 ); +outer_clean_range (__pa (&mcpm_entry_vectors[cluster][cpu]), +__pa (&mcpm_entry_vectors[cluster][cpu +1 ]));
And really, if the write hasn't been done by the compiler prior to calling __cpuc_flush_dcache_area() then we're into really bad problems.
PATCH v4 1/15 ARM: multi-cluster PM: secondary kernel entry code
Quoted 12 lines · Russell King - ARM Linux
arch/arm/common/mcpm_entry.c line 20
Quoted 6 lines · Nicolas Pitre
arch/arm/common/mcpm_entry.c new file +6
-0+#include <asm/mcpm_entry.h> +#include <asm/barrier.h> +#include <asm/proc-fns.h> +#include <asm/cacheflush.h> + +extern volatile unsigned long mcpm_entry_vectors[MAX_NR_CLUSTERS ][MAX_CPUS_PER_CLUSTER ];This should not be volatile. You should know by now the stance in the Linux community against using volatile on data declarations. See
Documentation/volatile-considered-harmful.txtto remind yourself of the reasoning.
That document says:
|The key point to understand with regard to volatile is that its purpose |is to suppress optimization, which is almost never what one really |wants to do.
Turns out that this is exactly what we want here: suppress optimization.
However ...
Quoted 11 lines
arch/arm/common/mcpm_entry.c line 28
Quoted 8 lines
arch/arm/common/mcpm_entry.c new file +8
-0+ +void mcpm_set_entry_vector (unsigned cpu,unsigned cluster,void *ptr) +{ +unsigned long val = ptr ?virt_to_phys (ptr) :0 ; + mcpm_entry_vectors[cluster][cpu] = val; +__cpuc_flush_dcache_area ((void *)&mcpm_entry_vectors[cluster][cpu],4 ); +outer_clean_range (__pa (&mcpm_entry_vectors[cluster][cpu]), +__pa (&mcpm_entry_vectors[cluster][cpu +1 ]));And really, if the write hasn't been done by the compiler prior to calling
__cpuc_flush_dcache_area()then we're into really bad problems.
That is indeed true. The memory might have been uncacheable at some point and then the volatile was necessary in that case.
I removed it in my tree.
Nicolas
PATCH v4 1/15 ARM: multi-cluster PM: secondary kernel entry code
Quoted 15 lines · Nicolas Pitre
Quoted 12 lines · Russell King - ARM Linux
arch/arm/common/mcpm_entry.c line 20
Quoted 6 lines · Nicolas Pitre
arch/arm/common/mcpm_entry.c new file +6
-0+#include <asm/mcpm_entry.h> +#include <asm/barrier.h> +#include <asm/proc-fns.h> +#include <asm/cacheflush.h> + +extern volatile unsigned long mcpm_entry_vectors[MAX_NR_CLUSTERS ][MAX_CPUS_PER_CLUSTER ];This should not be volatile. You should know by now the stance in the Linux community against using volatile on data declarations. See
Documentation/volatile-considered-harmful.txtto remind yourself of the reasoning.
Quoted 7 lines
That document says:
|The key point to understand with regard to volatile is that its purpose |is to suppress optimization, which is almost never what one really |wants to do.Turns out that this is exactly what we want here: suppress optimization.
What optimization are you trying to suppress in the function below?
Quoted 14 lines
However ...
Quoted 11 lines
arch/arm/common/mcpm_entry.c line 28
Quoted 8 lines
arch/arm/common/mcpm_entry.c new file +8
-0+ +void mcpm_set_entry_vector (unsigned cpu,unsigned cluster,void *ptr) +{ +unsigned long val = ptr ?virt_to_phys (ptr) :0 ; + mcpm_entry_vectors[cluster][cpu] = val; +__cpuc_flush_dcache_area ((void *)&mcpm_entry_vectors[cluster][cpu],4 ); +outer_clean_range (__pa (&mcpm_entry_vectors[cluster][cpu]), +__pa (&mcpm_entry_vectors[cluster][cpu +1 ]));And really, if the write hasn't been done by the compiler prior to calling
__cpuc_flush_dcache_area()then we're into really bad problems.
Quoted 2 lines
That is indeed true. The memory might have been uncacheable at some point and then the volatile was necessary in that case.
I don't buy the argument about it being uncachable - the compiler doesn't know that, and the cacheability of the memory really doesn't change the code that the compiler produces.
Moreover, the compiler can't really reorder the store to mcpm_entry_vectors[cluster][cpu] with the calls to the cache flushing anyway - and as the compiler has no clue what those calls are doing so it has to ensure that the results of the preceding code is visible to some "unknown code" which it can't see. Therefore, it practically has no option but to issue the store before calling those cache flush functions.
Also... consider using sizeof() rather than constant 4.
PATCH v4 1/15 ARM: multi-cluster PM: secondary kernel entry code
Quoted 18 lines · Russell King - ARM Linux
Quoted 17 lines · Nicolas Pitre
Quoted 12 lines · Russell King - ARM Linux
arch/arm/common/mcpm_entry.c line 20
Quoted 6 lines · Nicolas Pitre
arch/arm/common/mcpm_entry.c new file +6
-0+#include <asm/mcpm_entry.h> +#include <asm/barrier.h> +#include <asm/proc-fns.h> +#include <asm/cacheflush.h> + +extern volatile unsigned long mcpm_entry_vectors[MAX_NR_CLUSTERS ][MAX_CPUS_PER_CLUSTER ];This should not be volatile. You should know by now the stance in the Linux community against using volatile on data declarations. See
Documentation/volatile-considered-harmful.txtto remind yourself of the reasoning.That document says:
Quoted 7 lines
Quoted 5 lines
|The key point to understand with regard to volatile is that its purpose |is to suppress optimization, which is almost never what one really |wants to do.Turns out that this is exactly what we want here: suppress optimization.
What optimization are you trying to suppress in the function below?
Ordering of writes to this memory wrt other code in case the above function was inlined. But as I said this is a moot point now.
Quoted 14 lines
Quoted 14 lines
However ...
Quoted 11 lines
arch/arm/common/mcpm_entry.c line 28
Quoted 8 lines
arch/arm/common/mcpm_entry.c new file +8
-0+ +void mcpm_set_entry_vector (unsigned cpu,unsigned cluster,void *ptr) +{ +unsigned long val = ptr ?virt_to_phys (ptr) :0 ; + mcpm_entry_vectors[cluster][cpu] = val; +__cpuc_flush_dcache_area ((void *)&mcpm_entry_vectors[cluster][cpu],4 ); +outer_clean_range (__pa (&mcpm_entry_vectors[cluster][cpu]), +__pa (&mcpm_entry_vectors[cluster][cpu +1 ]));And really, if the write hasn't been done by the compiler prior to calling
__cpuc_flush_dcache_area()then we're into really bad problems.
Quoted 6 lines
Quoted 2 lines
That is indeed true. The memory might have been uncacheable at some point and then the volatile was necessary in that case.
I don't buy the argument about it being uncachable - the compiler doesn't know that, and the cacheability of the memory really doesn't change the code that the compiler produces.
My point is that the memory is now cacheable these days, which requires explicit cache maintenance for the writes to hit main memory. That cache maintenance acts as a barrier the volatile used to be before that cache maintenance call was there.
Quoted 1 line
Also... consider using
sizeof()rather than constant 4.
Sure.
Nicolas
PATCH v4 2/15 ARM: mcpm: introduce the CPU/cluster power API
In reply to Nicolas PitreThis is the basic API used to handle the powering up/down of individual CPUs in a (multi-)cluster system. The platform specific backend implementation has the responsibility to also handle the cluster level power as well when the first/last CPU in a cluster is brought up/down.
--- arch/arm/common/mcpm_entry.c | 88 +++++++++++++++++++++++++++++++++++++ arch/arm/include/asm/mcpm_entry.h | 92 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 180 insertions(+)
arch/arm/common/mcpm_entry.c +88 -0
@@ -11,11 +11,13 @@#include <linux/kernel.h> #include <linux/init.h> +#include <linux/irqflags.h> #include <asm/mcpm_entry.h> #include <asm/barrier.h> #include <asm/proc-fns.h> #include <asm/cacheflush.h> +#include <asm/idmap.h> extern volatile unsigned long mcpm_entry_vectors[MAX_NR_CLUSTERS ][MAX_CPUS_PER_CLUSTER ];
@@ -27,3 +29,89 @@ void mcpm_set_entry_vector(unsigned cpu, unsigned cluster, void *ptr)outer_clean_range (__pa (&mcpm_entry_vectors[cluster][cpu]),__pa (&mcpm_entry_vectors[cluster][cpu +1 ])); } + +static const struct mcpm_platform_ops *platform_ops; + +int __init mcpm_platform_register (const struct mcpm_platform_ops *ops) +{ +if (platform_ops) +return -EBUSY ; + platform_ops = ops; +return 0 ; +} + +int mcpm_cpu_power_up (unsigned int cpu,unsigned int cluster) +{ +if (!platform_ops) +return -EUNATCH ;/* try not to shadow power_up errors */ +might_sleep (); +return platform_ops->power_up (cpu, cluster); +} + +typedef void (*phys_reset_t )(unsigned long ); + +void mcpm_cpu_power_down (void ) +{ +phys_reset_t phys_reset; + +BUG_ON (!platform_ops); +BUG_ON (!irqs_disabled ()); + +/* +* Do this before calling into the power_down method, +* as it might not always be safe to do afterwards. +*/ +setup_mm_for_reboot (); + + platform_ops->power_down (); + +/* +* It is possible for a power_up request to happen concurrently +* with a power_down request for the same CPU. In this case the +* power_down method might not be able to actually enter a +* powered down state with the WFI instruction if the power_up +* method has removed the required reset condition. The +* power_down method is then allowed to return. We must perform +* a re-entry in the kernel as if the power_up method just had +* deasserted reset on the CPU. +* +* To simplify race issues, the platform specific implementation +* must accommodate for the possibility of unordered calls to +* power_down and power_up with a usage count. Therefore, if a +* call to power_up is issued for a CPU that is not down, then +* the next call to power_down must not attempt a full shutdown +* but only do the minimum (normally disabling L1 cache and CPU +* coherency) and return just as if a concurrent power_up request +* had happened as described above. +*/ + + phys_reset = (phys_reset_t )(unsigned long )virt_to_phys (cpu_reset); +phys_reset (virt_to_phys (mcpm_entry_point)); + +/* should never get here */ +BUG (); +} + +void mcpm_cpu_suspend (u64 expected_residency) +{ +phys_reset_t phys_reset; + +BUG_ON (!platform_ops); +BUG_ON (!irqs_disabled ()); + +/* Very similar to mcpm_cpu_power_down() */ +setup_mm_for_reboot (); + platform_ops->suspend (expected_residency); + phys_reset = (phys_reset_t )(unsigned long )virt_to_phys (cpu_reset); +phys_reset (virt_to_phys (mcpm_entry_point)); +BUG (); +} + +int mcpm_cpu_powered_up (void ) +{ +if (!platform_ops) +return -EUNATCH ; +if (platform_ops->powered_up) + platform_ops->powered_up (); +return 0 ; +}
arch/arm/include/asm/mcpm_entry.h +92 -0
97 lines not on this page · raw message
Signature · 1 line
1.8.1.2
PATCH v4 3/15 ARM: mcpm: introduce helpers for platform coherency exit/setup
In reply to Nicolas PitreFrom: Dave Martin <[email protected]>
This provides helper methods to coordinate between CPUs coming down and CPUs going up, as well as documentation on the used algorithms, so that cluster teardown and setup
operations are not done for a cluster simultaneously.
For use in the power_down() implementation:
* __mcpm_cpu_going_down(unsigned int cluster, unsigned int cpu)
* __mcpm_outbound_enter_critical(unsigned int cluster)
* __mcpm_outbound_leave_critical(unsigned int cluster)
* __mcpm_cpu_down(unsigned int cluster, unsigned int cpu)
The power_up_setup() helper should do platform-specific setup in preparation for turning the CPU on, such as invalidating local caches or entering coherency. It must be assembler for now, since it must run before the MMU can be switched on. It is passed the affinity level which should be initialized.
Because the mcpm_sync_struct content is looked-up and modified with the cache enabled or disabled depending on the code path, it is crucial to always ensure proper cache maintenance to update main memory right away. Therefore, any cached write must be followed by a cache clean operation and any cached read must be preceded by a cache invalidate operation (actually a cache flush i.e. clean+invalidate to avoid discarding possible concurrent writes) on the accessed memory.
Also, in order to prevent a cached writer from interfering with an adjacent non-cached writer, we ensure each state variable is located to a separate cache line.
Thanks to Nicolas Pitre and Achin Gupta for the help with this patch.
--- Documentation/arm/cluster-pm-race-avoidance.txt | 498 ++++++++++++++++++++++++ arch/arm/common/mcpm_entry.c | 197 ++++++++++ arch/arm/common/mcpm_head.S | 106 ++++- arch/arm/include/asm/mcpm_entry.h | 63 +++ 4 files changed, 862 insertions(+), 2 deletions(-) create mode 100644 Documentation/arm/cluster-pm-race-avoidance.txt
Documentation/arm/cluster-pm-race-avoidance.txt new file +498 -0
@@ -0,0 +1,498 @@ +Cluster-wide Power-up/power-down race avoidance algorithm +========================================================= + +This file documents the algorithm which is used to coordinate CPU and +cluster setup and teardown operations and to manage hardware coherency +controls safely. + +The section "Rationale" explains what the algorithm is for and why it is +needed. "Basic model" explains general concepts using a simplified view +of the system. The other sections explain the actual details of the +algorithm in use. + + +Rationale +--------- + +In a system containing multiple CPUs, it is desirable to have the +ability to turn off individual CPUs when the system is idle, reducing +power consumption and thermal dissipation. + +In a system containing multiple clusters of CPUs, it is also desirable +to have the ability to turn off entire clusters. + +Turning entire clusters off and on is a risky business, because it +involves performing potentially destructive operations affecting a group +of independently running CPUs, while the OS continues to run. This +means that we need some coordination in order to ensure that critical +cluster-level operations are only performed when it is truly safe to do +so. + +Simple locking may not be sufficient to solve this problem, because +mechanisms like Linux spinlocks may rely on coherency mechanisms which +are not immediately enabled when a cluster powers up. Since enabling or +disabling those mechanisms may itself be a non-atomic operation (such as +writing some hardware registers and invalidating large caches), other +methods of coordination are required in order to guarantee safe +power-down and power-up at the cluster level. + +The mechanism presented in this document describes a coherent memory +based protocol for performing the needed coordination. It aims to be as +lightweight as possible, while providing the required safety properties. + + +Basic model +----------- + +Each cluster and CPU is assigned a state, as follows: + + DOWN + COMING_UP + UP + GOING_DOWN + + +---------> UP ----------+ + | v + + COMING_UP GOING_DOWN + + ^ | + +--------- DOWN <--------+ + + +DOWN: The CPU or cluster is not coherent, and is either powered off or + suspended, or is ready to be powered off or suspended. + +COMING_UP: The CPU or cluster has committed to moving to the UP state. + It may be part way through the process of initialisation and + enabling coherency. + +UP: The CPU or cluster is active and coherent at the hardware + level. A CPU in this state is not necessarily being used + actively by the kernel. + +GOING_DOWN: The CPU or cluster has committed to moving to the DOWN + state. It may be part way through the process of teardown and + coherency exit. + + +Each CPU has one of these states assigned to it at any point in time. +The CPU states are described in the "CPU state" section, below. + +Each cluster is also assigned a state, but it is necessary to split the +state value into two parts (the "cluster" state and "inbound" state) and +to introduce additional states in order to avoid races between different +CPUs in the cluster simultaneously modifying the state. The cluster- +level states are described in the "Cluster state" section. + +To help distinguish the CPU states from cluster states in this +discussion, the state names are given a CPU_ prefix for the CPU states, +and a CLUSTER_ or INBOUND_ prefix for the cluster states. + + +CPU state +--------- + +In this algorithm, each individual core in a multi-core processor is +referred to as a "CPU". CPUs are assumed to be single-threaded: +therefore, a CPU can only be doing one thing@a single point in time. + +This means that CPUs fit the basic model closely. + +The algorithm defines the following states for each CPU in the system: + + CPU_DOWN + CPU_COMING_UP + CPU_UP + CPU_GOING_DOWN + + cluster setup and + CPU setup complete policy decision + +-----------> CPU_UP ------------+ + | v + + CPU_COMING_UP CPU_GOING_DOWN + + ^ | + +----------- CPU_DOWN <----------+ + policy decision CPU teardown complete + or hardware event + + +The definitions of the four states correspond closely to the states of +the basic model. + +Transitions between states occur as follows. + +A trigger event (spontaneous) means that the CPU can transition to the +next state as a result of making local progress only, with no +requirement for any external event to happen. + + +CPU_DOWN: + + A CPU reaches the CPU_DOWN state when it is ready for + power-down. On reaching this state, the CPU will typically + power itself down or suspend itself, via a WFI instruction or a + firmware call. + + Next state: CPU_COMING_UP + Conditions: none + + Trigger events: + + a) an explicit hardware power-up operation, resulting + from a policy decision on another CPU; + + b) a hardware event, such as an interrupt. + + +CPU_COMING_UP: + + A CPU cannot start participating in hardware coherency until the + cluster is set up and coherent. If the cluster is not ready, + then the CPU will wait in the CPU_COMING_UP state until the + cluster has been set up. + + Next state: CPU_UP + Conditions: The CPU's parent cluster must be in CLUSTER_UP. + Trigger events: Transition of the parent cluster to CLUSTER_UP. + + Refer to the "Cluster state" section for a description of the + CLUSTER_UP state. + + +CPU_UP: + When a CPU reaches the CPU_UP state, it is safe for the CPU to + start participating in local coherency. + + This is done by jumping to the kernel's CPU resume code. + + Note that the definition of this state is slightly different + from the basic model definition: CPU_UP does not mean that the + CPU is coherent yet, but it does mean that it is safe to resume + the kernel. The kernel handles the rest of the resume + procedure, so the remaining steps are not visible as part of the + race avoidance algorithm. + + The CPU remains in this state until an explicit policy decision + is made to shut down or suspend the CPU. + + Next state: CPU_GOING_DOWN + Conditions: none + Trigger events: explicit policy decision + + +CPU_GOING_DOWN: + + While in this state, the CPU exits coherency, including any + operations required to achieve this (such as cleaning data + caches). + + Next state: CPU_DOWN + Conditions: local CPU teardown complete + Trigger events: (spontaneous) + + +Cluster state +------------- + +A cluster is a group of connected CPUs with some common resources. +Because a cluster contains multiple CPUs, it can be doing multiple +things@the same time. This has some implications. In particular, a +CPU can start up while another CPU is tearing the cluster down. + +In this discussion, the "outbound side" is the view of the cluster state +as seen by a CPU tearing the cluster down. The "inbound side" is the +view of the cluster state as seen by a CPU setting the CPU up. + +In order to enable safe coordination in such situations, it is important +that a CPU which is setting up the cluster can advertise its state +independently of the CPU which is tearing down the cluster. For this +reason, the cluster state is split into two parts: + + "cluster" state: The global state of the cluster; or the state + on the outbound side: + + CLUSTER_DOWN + CLUSTER_UP + CLUSTER_GOING_DOWN + + "inbound" state: The state of the cluster on the inbound side. + + INBOUND_NOT_COMING_UP + INBOUND_COMING_UP + + + The different pairings of these states results in six possible + states for the cluster as a whole: + + CLUSTER_UP + +==========> INBOUND_NOT_COMING_UP -------------+ + # | + | + CLUSTER_UP <----+ | + INBOUND_COMING_UP | v + + ^ CLUSTER_GOING_DOWN CLUSTER_GOING_DOWN + # INBOUND_COMING_UP <=== INBOUND_NOT_COMING_UP + + CLUSTER_DOWN | | + INBOUND_COMING_UP <----+ | + | + ^ | + +=========== CLUSTER_DOWN <------------+ + INBOUND_NOT_COMING_UP + + Transitions -----> can only be made by the outbound CPU, and + only involve changes to the "cluster" state. + + Transitions ===##> can only be made by the inbound CPU, and only + involve changes to the "inbound" state, except where there is no + further transition possible on the outbound side (i.e., the + outbound CPU has put the cluster into the CLUSTER_DOWN state). + + The race avoidance algorithm does not provide a way to determine + which exact CPUs within the cluster play these roles. This must + be decided in advance by some other means. Refer to the section + "Last man and first man selection" for more explanation. + + + CLUSTER_DOWN/INBOUND_NOT_COMING_UP is the only state where the + cluster can actually be powered down. + + The parallelism of the inbound and outbound CPUs is observed by + the existence of two different paths from CLUSTER_GOING_DOWN/ + INBOUND_NOT_COMING_UP (corresponding to GOING_DOWN in the basic + model) to CLUSTER_DOWN/INBOUND_COMING_UP (corresponding to + COMING_UP in the basic model). The second path avoids cluster + teardown completely. + + CLUSTER_UP/INBOUND_COMING_UP is equivalent to UP in the basic + model. The final transition to CLUSTER_UP/INBOUND_NOT_COMING_UP + is trivial and merely resets the state machine ready for the + next cycle. + + Details of the allowable transitions follow. + + The next state in each case is notated + + <cluster state>/<inbound state> (<transitioner>) + + where the <transitioner> is the side on which the transition + can occur; either the inbound or the outbound side. + + +CLUSTER_DOWN/INBOUND_NOT_COMING_UP: + + Next state: CLUSTER_DOWN/INBOUND_COMING_UP (inbound) + Conditions: none + Trigger events: + + a) an explicit hardware power-up operation, resulting + from a policy decision on another CPU; + + b) a hardware event, such as an interrupt. + + +CLUSTER_DOWN/INBOUND_COMING_UP: + + In this state, an inbound CPU sets up the cluster, including + enabling of hardware coherency at the cluster level and any + other operations (such as cache invalidation) which are required + in order to achieve this. + + The purpose of this state is to do sufficient cluster-level + setup to enable other CPUs in the cluster to enter coherency + safely. + + Next state: CLUSTER_UP/INBOUND_COMING_UP (inbound) + Conditions: cluster-level setup and hardware coherency complete + Trigger events: (spontaneous) + + +CLUSTER_UP/INBOUND_COMING_UP: + + Cluster-level setup is complete and hardware coherency is + enabled for the cluster. Other CPUs in the cluster can safely + enter coherency. + + This is a transient state, leading immediately to + CLUSTER_UP/INBOUND_NOT_COMING_UP. All other CPUs on the cluster + should consider treat these two states as equivalent. + + Next state: CLUSTER_UP/INBOUND_NOT_COMING_UP (inbound) + Conditions: none + Trigger events: (spontaneous) + + +CLUSTER_UP/INBOUND_NOT_COMING_UP: + + Cluster-level setup is complete and hardware coherency is + enabled for the cluster. Other CPUs in the cluster can safely + enter coherency. + + The cluster will remain in this state until a policy decision is + made to power the cluster down. + + Next state: CLUSTER_GOING_DOWN/INBOUND_NOT_COMING_UP (outbound) + Conditions: none + Trigger events: policy decision to power down the cluster + + +CLUSTER_GOING_DOWN/INBOUND_NOT_COMING_UP: + + An outbound CPU is tearing the cluster down. The selected CPU + must wait in this state until all CPUs in the cluster are in the + CPU_DOWN state. + + When all CPUs are in the CPU_DOWN state, the cluster can be torn + down, for example by cleaning data caches and exiting + cluster-level coherency. + + To avoid wasteful unnecessary teardown operations, the outbound + should check the inbound cluster state for asynchronous + transitions to INBOUND_COMING_UP. Alternatively, individual + CPUs can be checked for entry into CPU_COMING_UP or CPU_UP. + + + Next states: + + CLUSTER_DOWN/INBOUND_NOT_COMING_UP (outbound) + Conditions: cluster torn down and ready to power off + Trigger events: (spontaneous) + + CLUSTER_GOING_DOWN/INBOUND_COMING_UP (inbound) + Conditions: none + Trigger events: + + a) an explicit hardware power-up operation, + resulting from a policy decision on another + CPU; + + b) a hardware event, such as an interrupt. + + +CLUSTER_GOING_DOWN/INBOUND_COMING_UP: + + The cluster is (or was) being torn down, but another CPU has + come online in the meantime and is trying to set up the cluster + again. + + If the outbound CPU observes this state, it has two choices: + + a) back out of teardown, restoring the cluster to the + CLUSTER_UP state; + + b) finish tearing the cluster down and put the cluster + in the CLUSTER_DOWN state; the inbound CPU will + set up the cluster again from there. + + Choice (a) permits the removal of some latency by avoiding + unnecessary teardown and setup operations in situations where + the cluster is not really going to be powered down. + + + Next states: + + CLUSTER_UP/INBOUND_COMING_UP (outbound) + Conditions: cluster-level setup and hardware + coherency complete + Trigger events: (spontaneous) + + CLUSTER_DOWN/INBOUND_COMING_UP (outbound) + Conditions: cluster torn down and ready to power off + Trigger events: (spontaneous) + + +Last man and First man selection +-------------------------------- + +The CPU which performs cluster tear-down operations on the outbound side +is commonly referred to as the "last man". + +The CPU which performs cluster setup on the inbound side is commonly +referred to as the "first man". + +The race avoidance algorithm documented above does not provide a +mechanism to choose which CPUs should play these roles. + + +Last man: + +When shutting down the cluster, all the CPUs involved are initially +executing Linux and hence coherent. Therefore, ordinary spinlocks can +be used to select a last man safely, before the CPUs become +non-coherent. + + +First man: + +Because CPUs may power up asynchronously in response to external wake-up +events, a dynamic mechanism is needed to make sure that only one CPU +attempts to play the first man role and do the cluster-level +initialisation: any other CPUs must wait for this to complete before +proceeding. + +Cluster-level initialisation may involve actions such as configuring +coherency controls in the bus fabric. + +The current implementation in mcpm_head.S uses a separate mutual exclusion +mechanism to do this arbitration. This mechanism is documented in +detail in vlocks.txt. + + +Features and Limitations +------------------------ + +Implementation: + + The current ARM-based implementation is split between + arch/arm/common/mcpm_head.S (low-level inbound CPU operations) and + arch/arm/common/mcpm_entry.c (everything else): + + __mcpm_cpu_going_down() signals the transition of a CPU to the + CPU_GOING_DOWN state. + + __mcpm_cpu_down() signals the transition of a CPU to the CPU_DOWN + state. + + A CPU transitions to CPU_COMING_UP and then to CPU_UP via the + low-level power-up code in mcpm_head.S. This could + involve CPU-specific setup code, but in the current + implementation it does not. + + __mcpm_outbound_enter_critical() and __mcpm_outbound_leave_critical() + handle transitions from CLUSTER_UP to CLUSTER_GOING_DOWN + and from there to CLUSTER_DOWN or back to CLUSTER_UP (in + the case of an aborted cluster power-down). + + These functions are more complex than the __mcpm_cpu_*() + functions due to the extra inter-CPU coordination which + is needed for safe transitions at the cluster level. + + A cluster transitions from CLUSTER_DOWN back to CLUSTER_UP via + the low-level power-up code in mcpm_head.S. This + typically involves platform-specific setup code, + provided by the platform-specific power_up_setup + function registered via mcpm_sync_init. + +Deep topologies: + + As currently described and implemented, the algorithm does not + support CPU topologies involving more than two levels (i.e., + clusters of clusters are not supported). The algorithm could be + extended by replicating the cluster-level states for the + additional topological levels, and modifying the transition + rules for the intermediate (non-outermost) cluster levels. + + +Colophon +-------- + +Originally created and documented by Dave Martin for Linaro Limited, in +collaboration with Nicolas Pitre and Achin Gupta. + +Copyright (C) 2012-2013 Linaro Limited +Distributed under the terms of Version 2 of the GNU General Public +License, as defined in linux/COPYING.
arch/arm/common/mcpm_entry.c +197 -0
206 lines not on this page · raw message
arch/arm/common/mcpm_head.S +104 -2
142 lines not on this page · raw message
arch/arm/include/asm/mcpm_entry.h +63 -0
76 lines not on this page · raw message
Signature · 1 line
1.8.1.2
PATCH v4 3/15 ARM: mcpm: introduce helpers for platform coherency exit/setup
Hi,
Just two nits below. One could be fixed incrementally, the other is a longer-term potential cleanup.
Quoted 14 lines · Nicolas Pitre
arch/arm/include/asm/mcpm_entry.h +6
-0@@ -15,8 +15,37 @@#define MAX_CPUS_PER_CLUSTER 4#define MAX_NR_CLUSTERS 2 +/* Definitions for mcpm_sync_struct */ +#define CPU_DOWN 0x11 +#define CPU_COMING_UP 0x12 +#define CPU_UP 0x13 +#define CPU_GOING_DOWN 0x14 +
arch/arm/include/asm/mcpm_entry.h line 33
Quoted 10 lines
arch/arm/include/asm/mcpm_entry.h +10
-0+#define CLUSTER_DOWN 0x21 +#define CLUSTER_UP 0x22 +#define CLUSTER_GOING_DOWN 0x23 + +#define INBOUND_NOT_COMING_UP 0x31 +#define INBOUND_COMING_UP 0x32 + +/* This is a complete guess. */ +#define __CACHE_WRITEBACK_ORDER 6 +#define __CACHE_WRITEBACK_GRANULE (1 << __CACHE_WRITEBACK_ORDER)
Something a little more educational could be useful here. It needs to be the max writeback/line size of the supported platforms of the binary kernel, i.e. if someone builds an SoC with 128-byte L2 cache lines this guess will break.
Quoted 18 lines
arch/arm/include/asm/mcpm_entry.h +12
-0+/* Offsets for the mcpm_sync_struct members, for use in asm: */ +#define MCPM_SYNC_CLUSTER_CPUS 0 +#define MCPM_SYNC_CPU_SIZE __CACHE_WRITEBACK_GRANULE +#define MCPM_SYNC_CLUSTER_CLUSTER \ + (MCPM_SYNC_CLUSTER_CPUS + MCPM_SYNC_CPU_SIZE * MAX_CPUS_PER_CLUSTER) +#define MCPM_SYNC_CLUSTER_INBOUND \ + (MCPM_SYNC_CLUSTER_CLUSTER + __CACHE_WRITEBACK_GRANULE) +#define MCPM_SYNC_CLUSTER_SIZE \ + (MCPM_SYNC_CLUSTER_INBOUND + __CACHE_WRITEBACK_GRANULE) +#ifndef __ASSEMBLY__ +#include <linux/types.h> +/* * Platform specific code should use this symbol to set up secondary * entry location for processors to use when released from reset.
arch/arm/include/asm/mcpm_entry.h struct mcpm_platform_ops { · line 160
Quoted 9 lines
arch/arm/include/asm/mcpm_entry.h +6
-0@@ -123,5 +152,39 @@ struct mcpm_platform_ops {*/ int __init mcpm_platform_register (const struct mcpm_platform_ops *ops); +/* Synchronisation structures for coordinating safe cluster setup/teardown: */ + +/* +* When modifying this structure, make sure you update the MCPM_SYNC_ defines +* to match. +*/
It would be nice to introduce something like arch/powerpc/kernel/asm-offsets.c on ARM to generate the ASM-side automatically from the C struct instead for this and other cases, but that's unrelated to this addition.
arch/arm/include/asm/mcpm_entry.h struct mcpm_platform_ops { · line 172
Quoted 12 lines
arch/arm/include/asm/mcpm_entry.h +12
-0+struct mcpm_sync_struct { +/* individual CPU states */ +struct { +volatile s8 cpu__aligned(__CACHE_WRITEBACK_GRANULE) ; + } cpus[MAX_CPUS_PER_CLUSTER ]; + +/* cluster state */ +volatile s8 cluster__aligned(__CACHE_WRITEBACK_GRANULE) ; + +/* inbound-side state */ +volatile s8 inbound__aligned(__CACHE_WRITEBACK_GRANULE) ; +};
-Olof
PATCH v4 3/15 ARM: mcpm: introduce helpers for platform coherency exit/setup
Quoted 4 lines · Olof Johansson
Hi,
Just two nits below. One could be fixed incrementally, the other is a longer-term potential cleanup.
I've updated my branch directly. Especially the comment addition is best presented in the original patch to avoid confusion.
Quoted 24 lines · Nicolas Pitre
Quoted 22 lines
arch/arm/include/asm/mcpm_entry.h +13
-0@@ -15,8 +15,37 @@#define MAX_CPUS_PER_CLUSTER 4#define MAX_NR_CLUSTERS 2 +/* Definitions for mcpm_sync_struct */ +#define CPU_DOWN 0x11 +#define CPU_COMING_UP 0x12 +#define CPU_UP 0x13 +#define CPU_GOING_DOWN 0x14 + +#define CLUSTER_DOWN 0x21 +#define CLUSTER_UP 0x22 +#define CLUSTER_GOING_DOWN 0x23 + +#define INBOUND_NOT_COMING_UP 0x31 +#define INBOUND_COMING_UP 0x32 +
Quoted 8 lines
arch/arm/include/asm/mcpm_entry.h line 33
Quoted 3 lines
arch/arm/include/asm/mcpm_entry.h +3
-0+/* This is a complete guess. */ +#define __CACHE_WRITEBACK_ORDER 6 +#define __CACHE_WRITEBACK_GRANULE (1 << __CACHE_WRITEBACK_ORDER)Something a little more educational could be useful here. It needs to be the max writeback/line size of the supported platforms of the binary kernel, i.e. if someone builds an SoC with 128-byte L2 cache lines this guess will break.
The commit log says:
|Also, in order to prevent a cached writer from interfering with an |adjacent non-cached writer, we ensure each state variable is located to |a separate cache line.
Although a more explicit comment here would be helpful indeed.
Fixed.
Yet, that part will be reworked once we move to dynamic allocation anyway. But for the time being it is best to keep the code simple while people get familiar with it.
Quoted 21 lines
Quoted 21 lines
arch/arm/include/asm/mcpm_entry.h +12
-0+/* Offsets for the mcpm_sync_struct members, for use in asm: */ +#define MCPM_SYNC_CLUSTER_CPUS 0 +#define MCPM_SYNC_CPU_SIZE __CACHE_WRITEBACK_GRANULE +#define MCPM_SYNC_CLUSTER_CLUSTER \ + (MCPM_SYNC_CLUSTER_CPUS + MCPM_SYNC_CPU_SIZE * MAX_CPUS_PER_CLUSTER) +#define MCPM_SYNC_CLUSTER_INBOUND \ + (MCPM_SYNC_CLUSTER_CLUSTER + __CACHE_WRITEBACK_GRANULE) +#define MCPM_SYNC_CLUSTER_SIZE \ + (MCPM_SYNC_CLUSTER_INBOUND + __CACHE_WRITEBACK_GRANULE) +#ifndef __ASSEMBLY__ +#include <linux/types.h> +/* * Platform specific code should use this symbol to set up secondary * entry location for processors to use when released from reset. @@ -123,5 +152,39 @@ struct mcpm_platform_ops {*/ int __init mcpm_platform_register (const struct mcpm_platform_ops *ops);
Quoted 10 lines
arch/arm/include/asm/mcpm_entry.h struct mcpm_platform_ops { · line 160
Quoted 6 lines
arch/arm/include/asm/mcpm_entry.h +6
-0+/* Synchronisation structures for coordinating safe cluster setup/teardown: */ + +/* +* When modifying this structure, make sure you update the MCPM_SYNC_ defines +* to match. +*/ It would be nice to introduce something like
arch/powerpc/kernel/asm-offsets.con ARM to generate the ASM-side automatically from the C struct instead for this and other cases, but that's unrelated to this addition.
There is a arch/arm/kernel/asm-offsets.c already.
Here's the answer I provided before:
http://article.gmane.org/gmane.linux.ports.arm.kernel/209009
|That's how that was done initially. But that ended up cluttering |asm-offsets.h for stuff that actually is really a local implementation |detail which doesn't need kernel wide scope. In other words, the end |result looked worse. | |One could argue that they are still exposed too much as the only files |that need to know about those defines are bL_head.S and bL_entry.c.
Nicolas
PATCH v4 3/15 ARM: mcpm: introduce helpers for platform coherency exit/setup
In reply to Olof JohanssonQuoted 28 lines · Olof Johansson
Hi,
Just two nits below. One could be fixed incrementally, the other is a longer-term potential cleanup.
Quoted 21 lines · Nicolas Pitre
arch/arm/include/asm/mcpm_entry.h +13
-0@@ -15,8 +15,37 @@#define MAX_CPUS_PER_CLUSTER 4#define MAX_NR_CLUSTERS 2 +/* Definitions for mcpm_sync_struct */ +#define CPU_DOWN 0x11 +#define CPU_COMING_UP 0x12 +#define CPU_UP 0x13 +#define CPU_GOING_DOWN 0x14 + +#define CLUSTER_DOWN 0x21 +#define CLUSTER_UP 0x22 +#define CLUSTER_GOING_DOWN 0x23 + +#define INBOUND_NOT_COMING_UP 0x31 +#define INBOUND_COMING_UP 0x32 +
Quoted 8 lines
arch/arm/include/asm/mcpm_entry.h line 33
Quoted 3 lines
arch/arm/include/asm/mcpm_entry.h +3
-0+/* This is a complete guess. */ +#define __CACHE_WRITEBACK_ORDER 6 +#define __CACHE_WRITEBACK_GRANULE (1 << __CACHE_WRITEBACK_ORDER)Something a little more educational could be useful here. It needs to be the max writeback/line size of the supported platforms of the binary kernel, i.e. if someone builds an SoC with 128-byte L2 cache lines this guess will break.
Will commented on the same thing. Ideally, a suitable definition should get selected through Kconfig.
The guess specified here is adequate for known b.L platforms, but a better framework for configuring this is desirable.
My original comment here is certainly not very informative though -- I'll post a clarification.
Quoted 21 lines
Quoted 21 lines
arch/arm/include/asm/mcpm_entry.h +12
-0+/* Offsets for the mcpm_sync_struct members, for use in asm: */ +#define MCPM_SYNC_CLUSTER_CPUS 0 +#define MCPM_SYNC_CPU_SIZE __CACHE_WRITEBACK_GRANULE +#define MCPM_SYNC_CLUSTER_CLUSTER \ + (MCPM_SYNC_CLUSTER_CPUS + MCPM_SYNC_CPU_SIZE * MAX_CPUS_PER_CLUSTER) +#define MCPM_SYNC_CLUSTER_INBOUND \ + (MCPM_SYNC_CLUSTER_CLUSTER + __CACHE_WRITEBACK_GRANULE) +#define MCPM_SYNC_CLUSTER_SIZE \ + (MCPM_SYNC_CLUSTER_INBOUND + __CACHE_WRITEBACK_GRANULE) +#ifndef __ASSEMBLY__ +#include <linux/types.h> +/* * Platform specific code should use this symbol to set up secondary * entry location for processors to use when released from reset. @@ -123,5 +152,39 @@ struct mcpm_platform_ops {*/ int __init mcpm_platform_register (const struct mcpm_platform_ops *ops);
Quoted 10 lines
arch/arm/include/asm/mcpm_entry.h struct mcpm_platform_ops { · line 160
Quoted 6 lines
arch/arm/include/asm/mcpm_entry.h +6
-0+/* Synchronisation structures for coordinating safe cluster setup/teardown: */ + +/* +* When modifying this structure, make sure you update the MCPM_SYNC_ defines +* to match. +*/ It would be nice to introduce something like
arch/powerpc/kernel/asm-offsets.con ARM to generate the ASM-side automatically from the C struct instead for this and other cases, but that's unrelated to this addition.
ARM already has its own asm-offsets, and we actually used this in an earlier version of the patches. However, this felt like pollution of that file for not a lot of benefit. Does powerpc have a problem with asm-offsets becoming a dumping ground for random junk, or is there a better way to organise it?
Cheers
---Dave
Quoted 15 lines
arch/arm/include/asm/mcpm_entry.h struct mcpm_platform_ops { · line 172
Quoted 12 lines
arch/arm/include/asm/mcpm_entry.h +12
-0+struct mcpm_sync_struct { +/* individual CPU states */ +struct { +volatile s8 cpu__aligned(__CACHE_WRITEBACK_GRANULE) ; + } cpus[MAX_CPUS_PER_CLUSTER ]; + +/* cluster state */ +volatile s8 cluster__aligned(__CACHE_WRITEBACK_GRANULE) ; + +/* inbound-side state */ +volatile s8 inbound__aligned(__CACHE_WRITEBACK_GRANULE) ; +};-Olof
PATCH v4 4/15 ARM: mcpm: Add baremetal voting mutexes
In reply to Nicolas PitreFrom: Dave Martin <[email protected]>
This patch adds a simple low-level voting mutex implementation to be used to arbitrate during first man selection when no load/store exclusive instructions are usable.
For want of a better name, these are called "vlocks". (I was tempted to call them ballot locks, but "block" is way too confusing an abbreviation...)
There is no function to wait for the lock to be released, and no vlock_lock() function since we don't need these at the moment. These could straightforwardly be added if vlocks get used for other purposes.
For architectural correctness even Strongly-Ordered memory accesses require barriers in order to guarantee that multiple CPUs have a coherent view of the ordering of memory accesses. Whether or not this matters depends on hardware implementation details of the memory system. Since the purpose of this code is to provide a clean, generic locking mechanism with no platform-specific dependencies the barriers should be present to avoid unpleasant surprises on future platforms.
Note:
* When taking the lock, we don't care about implicit background memory operations and other signalling which may be pending, because those are not part of the critical section anyway.
A DMB is sufficient to ensure correctly observed ordering if the explicit memory accesses in vlock_trylock.
* No barrier is required after checking the election result, because the result is determined by the store to VLOCK_OWNER_OFFSET and is already globally observed due to the barriers in voting_end. This means that global agreement on the winner is guaranteed, even before the winner is known locally.
--- Documentation/arm/vlocks.txt | 211 +++++++++++++++++++++++++++++++++++++++++++ arch/arm/common/vlock.S | 108 ++++++++++++++++++++++ arch/arm/common/vlock.h | 29 ++++++ 3 files changed, 348 insertions(+) create mode 100644 Documentation/arm/vlocks.txt create mode 100644 arch/arm/common/vlock.S create mode 100644 arch/arm/common/vlock.h
Documentation/arm/vlocks.txt new file +211 -0
@@ -0,0 +1,211 @@ +vlocks for Bare-Metal Mutual Exclusion +====================================== + +Voting Locks, or "vlocks" provide a simple low-level mutual exclusion +mechanism, with reasonable but minimal requirements on the memory +system. + +These are intended to be used to coordinate critical activity among CPUs +which are otherwise non-coherent, in situations where the hardware +provides no other mechanism to support this and ordinary spinlocks +cannot be used. + + +vlocks make use of the atomicity provided by the memory system for +writes to a single memory location. To arbitrate, every CPU "votes for +itself", by storing a unique number to a common memory location. The +final value seen in that memory location when all the votes have been +cast identifies the winner. + +In order to make sure that the election produces an unambiguous result +in finite time, a CPU will only enter the election in the first place if +no winner has been chosen and the election does not appear to have +started yet. + + +Algorithm +--------- + +The easiest way to explain the vlocks algorithm is with some pseudo-code: + + + int currently_voting[NR_CPUS] = { 0, }; + int last_vote = -1; /* no votes yet */ + + bool vlock_trylock(int this_cpu) + { + /* signal our desire to vote */ + currently_voting[this_cpu] = 1; + if (last_vote != -1) { + /* someone already volunteered himself */ + currently_voting[this_cpu] = 0; + return false; /* not ourself */ + } + + /* let's suggest ourself */ + last_vote = this_cpu; + currently_voting[this_cpu] = 0; + + /* then wait until everyone else is done voting */ + for_each_cpu(i) { + while (currently_voting[i] != 0) + /* wait */; + } + + /* result */ + if (last_vote == this_cpu) + return true; /* we won */ + return false; + } + + bool vlock_unlock(void) + { + last_vote = -1; + } + + +The currently_voting[] array provides a way for the CPUs to determine +whether an election is in progress, and plays a role analogous to the +"entering" array in Lamport's bakery algorithm [1]. + +However, once the election has started, the underlying memory system +atomicity is used to pick the winner. This avoids the need for a static +priority rule to act as a tie-breaker, or any counters which could +overflow. + +As long as the last_vote variable is globally visible to all CPUs, it +will contain only one value that won't change once every CPU has cleared +its currently_voting flag. + + +Features and limitations +------------------------ + + * vlocks are not intended to be fair. In the contended case, it is the + _last_ CPU which attempts to get the lock which will be most likely + to win. + + vlocks are therefore best suited to situations where it is necessary + to pick a unique winner, but it does not matter which CPU actually + wins. + + * Like other similar mechanisms, vlocks will not scale well to a large + number of CPUs. + + vlocks can be cascaded in a voting hierarchy to permit better scaling + if necessary, as in the following hypothetical example for 4096 CPUs: + + /* first level: local election */ + my_town = towns[(this_cpu >> 4) & 0xf]; + I_won = vlock_trylock(my_town, this_cpu & 0xf); + if (I_won) { + /* we won the town election, let's go for the state */ + my_state = states[(this_cpu >> 8) & 0xf]; + I_won = vlock_lock(my_state, this_cpu & 0xf)); + if (I_won) { + /* and so on */ + I_won = vlock_lock(the_whole_country, this_cpu & 0xf]; + if (I_won) { + /* ... */ + } + vlock_unlock(the_whole_country); + } + vlock_unlock(my_state); + } + vlock_unlock(my_town); + + +ARM implementation +------------------ + +The current ARM implementation [2] contains some optimisations beyond +the basic algorithm: + + * By packing the members of the currently_voting array close together, + we can read the whole array in one transaction (providing the number + of CPUs potentially contending the lock is small enough). This + reduces the number of round-trips required to external memory. + + In the ARM implementation, this means that we can use a single load + and comparison: + + LDR Rt, [Rn] + CMP Rt, #0 + + ...in place of code equivalent to: + + LDRB Rt, [Rn] + CMP Rt, #0 + LDRBEQ Rt, [Rn, #1] + CMPEQ Rt, #0 + LDRBEQ Rt, [Rn, #2] + CMPEQ Rt, #0 + LDRBEQ Rt, [Rn, #3] + CMPEQ Rt, #0 + + This cuts down on the fast-path latency, as well as potentially + reducing bus contention in contended cases. + + The optimisation relies on the fact that the ARM memory system + guarantees coherency between overlapping memory accesses of + different sizes, similarly to many other architectures. Note that + we do not care which element of currently_voting appears in which + bits of Rt, so there is no need to worry about endianness in this + optimisation. + + If there are too many CPUs to read the currently_voting array in + one transaction then multiple transations are still required. The + implementation uses a simple loop of word-sized loads for this + case. The number of transactions is still fewer than would be + required if bytes were loaded individually. + + + In principle, we could aggregate further by using LDRD or LDM, but + to keep the code simple this was not attempted in the initial + implementation. + + + * vlocks are currently only used to coordinate between CPUs which are + unable to enable their caches yet. This means that the + implementation removes many of the barriers which would be required + when executing the algorithm in cached memory. + + packing of the currently_voting array does not work with cached + memory unless all CPUs contending the lock are cache-coherent, due + to cache writebacks from one CPU clobbering values written by other + CPUs. (Though if all the CPUs are cache-coherent, you should be + probably be using proper spinlocks instead anyway). + + + * The "no votes yet" value used for the last_vote variable is 0 (not + -1 as in the pseudocode). This allows statically-allocated vlocks + to be implicitly initialised to an unlocked state simply by putting + them in .bss. + + An offset is added to each CPU's ID for the purpose of setting this + variable, so that no CPU uses the value 0 for its ID. + + +Colophon +-------- + +Originally created and documented by Dave Martin for Linaro Limited, for +use in ARM-based big.LITTLE platforms, with review and input gratefully +received from Nicolas Pitre and Achin Gupta. Thanks to Nicolas for +grabbing most of this text out of the relevant mail thread and writing +up the pseudocode. + +Copyright (C) 2012-2013 Linaro Limited +Distributed under the terms of Version 2 of the GNU General Public +License, as defined in linux/COPYING. + + +References +---------- + +[1] Lamport, L. "A New Solution of Dijkstra's Concurrent Programming + Problem", Communications of the ACM 17, 8 (August 1974), 453-455. + + http://en.wikipedia.org/wiki/Lamport%27s_bakery_algorithm + +[2] linux/arch/arm/common/vlock.S, www.kernel.org.
arch/arm/common/vlock.S new file +108 -0
108 lines not on this page · raw message
arch/arm/common/vlock.h new file +29 -0
@@ -0,0 +1,29 @@ +/* +* vlock.h - simple voting lock implementation +* +* Created by: Dave Martin, 2012-08-16 +* Copyright: (C) 2012-2013 Linaro Limited +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License version 2 as +* published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +*/ + +#ifndef __VLOCK_H +#define __VLOCK_H + +#include <asm/mcpm_entry.h> + +/* Offsets and sizes are rounded to a word (4 bytes) */ +#define VLOCK_OWNER_OFFSET 0 +#define VLOCK_VOTING_OFFSET 4 +#define VLOCK_VOTING_SIZE ((MAX_CPUS_PER_CLUSTER + 3) / 4 * 4) +#define VLOCK_SIZE (VLOCK_VOTING_OFFSET + VLOCK_VOTING_SIZE) +#define VLOCK_OWNER_NONE 0 + +#endif /* ! __VLOCK_H */
Signature · 1 line
1.8.1.2
PATCH v4 5/15 ARM: mcpm_head.S: vlock-based first man election
In reply to Nicolas PitreFrom: Dave Martin <[email protected]>
Instead of requiring the first man to be elected in advance (which can be suboptimal in some situations), this patch uses a per- cluster mutex to co-ordinate selection of the first man.
This should also make it more feasible to reuse this code path for asynchronous cluster resume (as in CPUidle scenarios).
We must ensure that the vlock data doesn't share a cacheline with anything else, or dirty cache eviction could corrupt it.
--- arch/arm/common/Makefile | 2 +- arch/arm/common/mcpm_head.S | 41 ++++++++++++++++++++++++++++++++++++----- 2 files changed, 37 insertions(+), 6 deletions(-)
arch/arm/common/Makefile +1 -1
@@ -13,4 +13,4 @@ obj-$(CONFIG_SHARP_PARAM) += sharpsl_param.o obj-$(CONFIG_SHARP_SCOOP) += scoop.o obj-$(CONFIG_PCI_HOST_ITE8152) += it8152.o obj-$(CONFIG_ARM_TIMER_SP804) += timer-sp.o -obj-$(CONFIG_CLUSTER_PM) += mcpm_head.o mcpm_entry.o +obj-$(CONFIG_CLUSTER_PM) += mcpm_head.o mcpm_entry.o vlock.o
arch/arm/common/mcpm_head.S +36 -5
82 lines not on this page · raw message
Signature · 1 line
1.8.1.2
PATCH v4 6/15 ARM: mcpm: generic SMP secondary bringup and hotplug support
In reply to Nicolas PitreNow that the cluster power API is in place, we can use it for SMP secondary bringup and CPU hotplug in a generic fashion.
--- arch/arm/common/Makefile | 2 +- arch/arm/common/mcpm_platsmp.c | 86 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 87 insertions(+), 1 deletion(-) create mode 100644 arch/arm/common/mcpm_platsmp.c
arch/arm/common/Makefile +1 -1
@@ -13,4 +13,4 @@ obj-$(CONFIG_SHARP_PARAM) += sharpsl_param.o obj-$(CONFIG_SHARP_SCOOP) += scoop.o obj-$(CONFIG_PCI_HOST_ITE8152) += it8152.o obj-$(CONFIG_ARM_TIMER_SP804) += timer-sp.o -obj-$(CONFIG_CLUSTER_PM) += mcpm_head.o mcpm_entry.o vlock.o +obj-$(CONFIG_CLUSTER_PM) += mcpm_head.o mcpm_entry.o mcpm_platsmp.o vlock.o
arch/arm/common/mcpm_platsmp.c new file +86 -0
86 lines not on this page · raw message
Signature · 1 line
1.8.1.2
PATCH v4 6/15 ARM: mcpm: generic SMP secondary bringup and hotplug support
arch/arm/common/mcpm_platsmp.c line 73
Quoted 8 lines · Nicolas Pitre
arch/arm/common/mcpm_platsmp.c new file +8
-0+static void mcpm_cpu_die (unsigned int cpu) +{ +unsigned int mpidr, pcpu, pcluster; + mpidr =read_cpuid_mpidr (); + pcpu =MPIDR_AFFINITY_LEVEL (mpidr,0 ); + pcluster =MPIDR_AFFINITY_LEVEL (mpidr,1 ); +mcpm_set_entry_vector (pcpu, pcluster,NULL ); +mcpm_cpu_power_down ();
How are dirty caches handled here for the dying CPU? Bear in mind at the moment, all that is left to the smp platform implementation, but I don't see anything in your patch set so far which handles this.
This is going to be especially important as we're moving that into the generic CPU hotplug code out of the platform code.
PATCH v4 6/15 ARM: mcpm: generic SMP secondary bringup and hotplug support
Quoted 14 lines · Russell King - ARM Linux
arch/arm/common/mcpm_platsmp.c line 73
Quoted 8 lines · Nicolas Pitre
arch/arm/common/mcpm_platsmp.c new file +8
-0+static void mcpm_cpu_die (unsigned int cpu) +{ +unsigned int mpidr, pcpu, pcluster; + mpidr =read_cpuid_mpidr (); + pcpu =MPIDR_AFFINITY_LEVEL (mpidr,0 ); + pcluster =MPIDR_AFFINITY_LEVEL (mpidr,1 ); +mcpm_set_entry_vector (pcpu, pcluster,NULL ); +mcpm_cpu_power_down ();How are dirty caches handled here for the dying CPU? Bear in mind at the moment, all that is left to the smp platform implementation, but I don't see anything in your patch set so far which handles this.
Quoted 2 lines
This is going to be especially important as we're moving that into the generic CPU hotplug code out of the platform code.
The cache is handled in the machine specific backend accessed through mcpm_cpu_power_down(). See the DCSCB related patches later in the series.
Nicolas
PATCH v4 7/15 ARM: introduce common set_auxcr/get_auxcr functions
In reply to Nicolas PitreFrom: Rob Herring <[email protected]>
Move the private set_auxcr/get_auxcr functions from drivers/cpuidle/cpuidle-calxeda.c so they can be used across platforms.
--- arch/arm/include/asm/cp15.h | 14 ++++++++++++++ drivers/cpuidle/cpuidle-calxeda.c | 14 -------------- 2 files changed, 14 insertions(+), 14 deletions(-)
arch/arm/include/asm/cp15.h +14 -0
@@ -59,6 +59,20 @@ static inline void set_cr(unsigned int val)isb (); } +static inline unsigned int get_auxcr (void ) +{ +unsigned int val; + asm("mrc p15, 0, %0, c1, c0, 1 @ get AUXCR" :"=r" (val)); +return val; +} + +static inline void set_auxcr (unsigned int val) +{ + asmvolatile ("mcr p15, 0, %0, c1, c0, 1 @ set AUXCR" + : :"r" (val)); +isb (); +} +#ifndef CONFIG_SMP extern void adjust_cr (unsigned long mask,unsigned long set);#endif
drivers/cpuidle/cpuidle-calxeda.c +0 -14
@@ -37,20 +37,6 @@ extern void *scu_base_addr;static struct cpuidle_device __percpu *calxeda_idle_cpuidle_devices; -static inline unsigned int get_auxcr (void ) -{ -unsigned int val; - asm("mrc p15, 0, %0, c1, c0, 1 @ get AUXCR" :"=r" (val) : :"cc" ); -return val; -} - -static inline void set_auxcr (unsigned int val) -{ - asmvolatile ("mcr p15, 0, %0, c1, c0, 1 @ set AUXCR" - : :"r" (val) :"cc" ); -isb (); -} - static noinline void calxeda_idle_restore(void) {set_cr (get_cr () |CR_C );
Signature · 1 line
1.8.1.2
PATCH v4 8/15 ARM: vexpress: introduce DCSCB support
In reply to Nicolas PitreThis adds basic CPU and cluster reset controls on RTSM for the A15x4-A7x4 model configuration using the Dual Cluster System Configuration Block (DCSCB).
The cache coherency interconnect (CCI) is not handled yet.
--- .../devicetree/bindings/arm/rtsm-dcscb.txt | 19 +++ arch/arm/mach-vexpress/Kconfig | 8 + arch/arm/mach-vexpress/Makefile | 1 + arch/arm/mach-vexpress/dcscb.c | 162 +++++++++++++++++++++ 4 files changed, 190 insertions(+) create mode 100644 Documentation/devicetree/bindings/arm/rtsm-dcscb.txt create mode 100644 arch/arm/mach-vexpress/dcscb.c
Documentation/devicetree/bindings/arm/rtsm-dcscb.txt new file +19 -0
@@ -0,0 +1,19 @@ +ARM Dual Cluster System Configuration Block +------------------------------------------- + +The Dual Cluster System Configuration Block (DCSCB) provides basic +functionality for controlling clocks, resets and configuration pins in +the Dual Cluster System implemented by the Real-Time System Model (RTSM). + +Required properties: + +- compatible : should be "arm,rtsm,dcscb" + +- reg : physical base address and the size of the registers window + +Example: + + dcscb at 60000000 { + compatible = "arm,rtsm,dcscb"; + reg = <0x60000000 0x1000>; + };
arch/arm/mach-vexpress/Kconfig +8 -0
@@ -52,4 +52,12 @@ config ARCH_VEXPRESS_CORTEX_A5_A9_ERRATA config ARCH_VEXPRESS_CA9X4 bool "Versatile Express Cortex-A9x4 tile" +config ARCH_VEXPRESS_DCSCB + bool "Dual Cluster System Control Block (DCSCB) support" + depends on CLUSTER_PM + help + Support for the Dual Cluster System Configuration Block (DCSCB). + This is needed to provide CPU and cluster power management + on RTSM. + endmenu
arch/arm/mach-vexpress/Makefile +1 -0
@@ -6,5 +6,6 @@ ccflags-$(CONFIG_ARCH_MULTIPLATFORM) := -I$(srctree)/$(src)/include \ obj-y := v2m.o reset.o obj-$(CONFIG_ARCH_VEXPRESS_CA9X4) += ct-ca9x4.o +obj-$(CONFIG_ARCH_VEXPRESS_DCSCB) += dcscb.o obj-$(CONFIG_SMP) += platsmp.o obj-$(CONFIG_HOTPLUG_CPU) += hotplug.o
arch/arm/mach-vexpress/dcscb.c new file +162 -0
162 lines not on this page · raw message
Signature · 1 line
1.8.1.2
PATCH v4 8/15 ARM: vexpress: introduce DCSCB support
Hi Nico,
Quoted 13 lines · Nicolas Pitre
arch/arm/mach-vexpress/dcscb.c new file +13
-0+static int dcscb_power_up (unsigned int cpu,unsigned int cluster) +{ +unsigned int rst_hold, cpumask = (1 << cpu); + +pr_debug ("%s: cpu %u cluster %u\n" , __func__, cpu, cluster); +if (cpu >=4 || cluster >=2 ) +return -EINVAL ; + +/* +* Since this is called with IRQs enabled, and no arch_spin_lock_irq +* variant exists, we need to disable IRQs manually here. +*/ +local_irq_disable ();
arch/arm/mach-vexpress/dcscb.c line 70
Quoted 10 lines
arch/arm/mach-vexpress/dcscb.c new file +10
-0+arch_spin_lock (&dcscb_lock); + + rst_hold =readl_relaxed (dcscb_base +RST_HOLD0 + cluster *4 ); +if (rst_hold & (1 <<8 )) { +/* remove cluster reset and add individual CPU's reset */ + rst_hold &= ~(1 <<8 ); + rst_hold |=0xf ; + } + rst_hold &= ~(cpumask | (cpumask <<4 )); +writel (rst_hold, dcscb_base +RST_HOLD0 + cluster *4 );
Why do you mix relaxed and non-relaxed I/O accessors here? Do you need the barriers implied by writel()?
Signature · 1 line
Catalin
PATCH v4 8/15 ARM: vexpress: introduce DCSCB support
Quoted 21 lines · Catalin Marinas
Hi Nico,
Quoted 18 lines · Nicolas Pitre
arch/arm/mach-vexpress/dcscb.c new file +18
-0+static int dcscb_power_up (unsigned int cpu,unsigned int cluster) +{ +unsigned int rst_hold, cpumask = (1 << cpu); + +pr_debug ("%s: cpu %u cluster %u\n" , __func__, cpu, cluster); +if (cpu >=4 || cluster >=2 ) +return -EINVAL ; + +/* +* Since this is called with IRQs enabled, and no arch_spin_lock_irq +* variant exists, we need to disable IRQs manually here. +*/ +local_irq_disable (); +arch_spin_lock (&dcscb_lock); + + rst_hold =readl_relaxed (dcscb_base +RST_HOLD0 + cluster *4 ); +if (rst_hold & (1 <<8 )) { +/* remove cluster reset and add individual CPU's reset */
Quoted 8 lines
arch/arm/mach-vexpress/dcscb.c line 70
Quoted 5 lines
arch/arm/mach-vexpress/dcscb.c new file +5
-0+ rst_hold &= ~(1 <<8 ); + rst_hold |=0xf ; + } + rst_hold &= ~(cpumask | (cpumask <<4 )); +writel (rst_hold, dcscb_base +RST_HOLD0 + cluster *4 );Why do you mix relaxed and non-relaxed I/O accessors here? Do you need the barriers implied by
writel()?
Most likely not. I'll change that.
For the record, this patch is not part of the pull request I sent recently.
Nicolas
PATCH v4 9/15 ARM: vexpress/dcscb: add CPU use counts to the power up/down API implementation
In reply to Nicolas PitreIt is possible for a CPU to be told to power up before it managed to power itself down. Solve this race with a usage count as mandated by the API definition.
--- arch/arm/mach-vexpress/dcscb.c | 77 +++++++++++++++++++++++++++++++++--------- 1 file changed, 61 insertions(+), 16 deletions(-)
arch/arm/mach-vexpress/dcscb.c +61 -16
@@ -44,6 +44,7 @@static arch_spinlock_t dcscb_lock = __ARCH_SPIN_LOCK_UNLOCKED;static void __iomem *dcscb_base; +static int dcscb_use_count[4 ][2 ];static int dcscb_power_up (unsigned int cpu,unsigned int cluster) {
@@ -60,14 +61,27 @@ static int dcscb_power_up(unsigned int cpu, unsigned int cluster)local_irq_disable ();arch_spin_lock (&dcscb_lock); - rst_hold =readl_relaxed (dcscb_base +RST_HOLD0 + cluster *4 ); -if (rst_hold & (1 <<8 )) { -/* remove cluster reset and add individual CPU's reset */ - rst_hold &= ~(1 <<8 ); - rst_hold |=0xf ; + dcscb_use_count[cpu][cluster]++; +if (dcscb_use_count[cpu][cluster] ==1 ) { + rst_hold =readl_relaxed (dcscb_base +RST_HOLD0 + cluster *4 ); +if (rst_hold & (1 <<8 )) { +/* remove cluster reset and add individual CPU's reset */ + rst_hold &= ~(1 <<8 ); + rst_hold |=0xf ; + } + rst_hold &= ~(cpumask | (cpumask <<4 )); +writel (rst_hold, dcscb_base +RST_HOLD0 + cluster *4 ); + }else if (dcscb_use_count[cpu][cluster] !=2 ) { +/* +* The only possible values are: +* 0 = CPU down +* 1 = CPU (still) up +* 2 = CPU requested to be up before it had a chance +* to actually make itself down. +* Any other value is a bug. +*/ +BUG (); } - rst_hold &= ~(cpumask | (cpumask <<4 )); -writel (rst_hold, dcscb_base +RST_HOLD0 + cluster *4 );arch_spin_unlock (&dcscb_lock);local_irq_enable ();
@@ -77,7 +91,8 @@ static int dcscb_power_up(unsigned int cpu, unsigned int cluster)static void dcscb_power_down (void ) { -unsigned int mpidr, cpu, cluster, rst_hold, cpumask, last_man; +unsigned int mpidr, cpu, cluster, rst_hold, cpumask; +bool last_man = false, skip_wfi = false; mpidr =read_cpuid_mpidr (); cpu =MPIDR_AFFINITY_LEVEL (mpidr,0 );
@@ -88,13 +103,26 @@ static void dcscb_power_down(void)BUG_ON (cpu >=4 || cluster >=2 );arch_spin_lock (&dcscb_lock); - rst_hold =readl_relaxed (dcscb_base +RST_HOLD0 + cluster *4 ); - rst_hold |= cpumask; -if (((rst_hold | (rst_hold >>4 )) &0xf ) ==0xf ) - rst_hold |= (1 <<8 ); -writel (rst_hold, dcscb_base +RST_HOLD0 + cluster *4 ); + dcscb_use_count[cpu][cluster]--; +if (dcscb_use_count[cpu][cluster] ==0 ) { + rst_hold =readl_relaxed (dcscb_base +RST_HOLD0 + cluster *4 ); + rst_hold |= cpumask; +if (((rst_hold | (rst_hold >>4 )) &0xf ) ==0xf ) { + rst_hold |= (1 <<8 ); + last_man = true; + } +writel (rst_hold, dcscb_base +RST_HOLD0 + cluster *4 ); + }else if (dcscb_use_count[cpu][cluster] ==1 ) { +/* +* A power_up request went ahead of us. +* Even if we do not want to shut this CPU down, +* the caller expects a certain state as if the WFI +* was aborted. So let's continue with cache cleaning. +*/ + skip_wfi = true; + }else +BUG ();arch_spin_unlock (&dcscb_lock); - last_man = (rst_hold & (1 <<8 ));/* * Now let's clean our L1 cache and shut ourself down.
@@ -121,8 +149,10 @@ static void dcscb_power_down(void)set_auxcr (get_auxcr () & ~(1 <<6 ));/* Now we are prepared for power-down, do it: */ -dsb (); -wfi (); +if (!skip_wfi) { +dsb (); +wfi (); + }/* Not dead@this point? Let our caller cope. */ }
@@ -132,6 +162,19 @@ static const struct mcpm_platform_ops dcscb_power_ops = { .power_down = dcscb_power_down, }; +static void __init dcscb_usage_count_init (void ) +{ +unsigned int mpidr, cpu, cluster; + + mpidr =read_cpuid_mpidr (); + cpu =MPIDR_AFFINITY_LEVEL (mpidr,0 ); + cluster =MPIDR_AFFINITY_LEVEL (mpidr,1 ); + +pr_debug ("%s: cpu %u cluster %u\n" , __func__, cpu, cluster); +BUG_ON (cpu >=4 || cluster >=2 ); + dcscb_use_count[cpu][cluster] =1 ; +} +static int __init dcscb_init (void ) {struct device_node *node;
@@ -144,6 +187,8 @@ static int __init dcscb_init(void)if (!dcscb_base)return -EADDRNOTAVAIL ; +dcscb_usage_count_init (); + ret =mcpm_platform_register (&dcscb_power_ops);if (ret) {iounmap (dcscb_base);
Signature · 1 line
1.8.1.2
PATCH v4 10/15 ARM: vexpress/dcscb: do not hardcode number of CPUs per cluster
In reply to Nicolas PitreIf 4 CPUs are assumed, the A15x1-A7x1 model configuration would never shut down the initial cluster as the 0xf reset bit mask will never be observed. Let's construct this mask based on the provided information in the DCSCB config register for the number of CPUs per cluster.
--- arch/arm/mach-vexpress/dcscb.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)
arch/arm/mach-vexpress/dcscb.c +10 -4
@@ -45,10 +45,12 @@ static arch_spinlock_t dcscb_lock = __ARCH_SPIN_LOCK_UNLOCKED;static void __iomem *dcscb_base;static int dcscb_use_count[4 ][2 ]; +static int dcscb_mcpm_cpu_mask[2 ];static int dcscb_power_up (unsigned int cpu,unsigned int cluster) {unsigned int rst_hold, cpumask = (1 << cpu); +unsigned int mcpm_mask = dcscb_mcpm_cpu_mask[cluster];pr_debug ("%s: cpu %u cluster %u\n" , __func__, cpu, cluster);if (cpu >=4 || cluster >=2 )
@@ -67,7 +69,7 @@ static int dcscb_power_up(unsigned int cpu, unsigned int cluster)if (rst_hold & (1 <<8 )) {/* remove cluster reset and add individual CPU's reset */ rst_hold &= ~(1 <<8 ); - rst_hold |=0xf ; + rst_hold |= mcpm_mask; } rst_hold &= ~(cpumask | (cpumask <<4 ));writel (rst_hold, dcscb_base +RST_HOLD0 + cluster *4 );
@@ -91,13 +93,14 @@ static int dcscb_power_up(unsigned int cpu, unsigned int cluster)static void dcscb_power_down (void ) { -unsigned int mpidr, cpu, cluster, rst_hold, cpumask; +unsigned int mpidr, cpu, cluster, rst_hold, cpumask, mcpm_mask;bool last_man = false, skip_wfi = false; mpidr =read_cpuid_mpidr (); cpu =MPIDR_AFFINITY_LEVEL (mpidr,0 ); cluster =MPIDR_AFFINITY_LEVEL (mpidr,1 ); cpumask = (1 << cpu); + mcpm_mask = dcscb_mcpm_cpu_mask[cluster];pr_debug ("%s: cpu %u cluster %u\n" , __func__, cpu, cluster);BUG_ON (cpu >=4 || cluster >=2 );
@@ -107,7 +110,7 @@ static void dcscb_power_down(void)if (dcscb_use_count[cpu][cluster] ==0 ) { rst_hold =readl_relaxed (dcscb_base +RST_HOLD0 + cluster *4 ); rst_hold |= cpumask; -if (((rst_hold | (rst_hold >>4 )) &0xf ) ==0xf ) { +if (((rst_hold | (rst_hold >>4 )) & mcpm_mask) == mcpm_mask) { rst_hold |= (1 <<8 ); last_man = true; }
@@ -178,6 +181,7 @@ static void __init dcscb_usage_count_init(void)static int __init dcscb_init (void ) {struct device_node *node; +unsigned int cfg;int ret; node =of_find_compatible_node (NULL ,NULL ,"arm,rtsm,dcscb" );
@@ -186,7 +190,9 @@ static int __init dcscb_init(void) dcscb_base=of_iomap (node,0 );if (!dcscb_base)return -EADDRNOTAVAIL ; - + cfg =readl_relaxed (dcscb_base +DCS_CFG_R ); + dcscb_mcpm_cpu_mask[0 ] = (1 << (((cfg >>16 ) >> (0 <<2 )) &0xf )) -1 ; + dcscb_mcpm_cpu_mask[1 ] = (1 << (((cfg >>16 ) >> (1 <<2 )) &0xf )) -1 ;dcscb_usage_count_init (); ret =mcpm_platform_register (&dcscb_power_ops);
Signature · 1 line
1.8.1.2
PATCH v4 11/15 drivers/bus: add ARM CCI support
In reply to Nicolas PitreFrom: Lorenzo Pieralisi <[email protected]>
On ARM multi-cluster systems coherency between cores running on different clusters is managed by the cache-coherent interconnect (CCI). It allows broadcasting of TLB invalidates and memory barriers and it guarantees cache coherency at system level.
This patch enables the basic infrastructure required in Linux to handle and programme the CCI component. The first implementation is based on a platform device, its relative DT compatible property and a simple programming interface.
Very basic for now. More functionalities will come later.
--- drivers/bus/Kconfig | 4 ++ drivers/bus/Makefile | 2 + drivers/bus/arm-cci.c | 112 ++++++++++++++++++++++++++++++++++++++++++++++++ include/linux/arm-cci.h | 30 +++++++++++++ 4 files changed, 148 insertions(+) create mode 100644 drivers/bus/arm-cci.c create mode 100644 include/linux/arm-cci.h
drivers/bus/Kconfig +4 -0
@@ -19,4 +19,8 @@ config OMAP_INTERCONNECT help Driver to enable OMAP interconnect error handling driver. + +config ARM_CCI + bool "ARM CCI driver support" + endmenu
drivers/bus/Makefile +2 -0
@@ -6,3 +6,5 @@ obj-$(CONFIG_OMAP_OCP2SCP) += omap-ocp2scp.o # Interconnect bus driver for OMAP SoCs. obj-$(CONFIG_OMAP_INTERCONNECT) += omap_l3_smx.o omap_l3_noc.o + +obj-$(CONFIG_ARM_CCI) += arm-cci.o
drivers/bus/arm-cci.c new file +112 -0
112 lines not on this page · raw message
include/linux/arm-cci.h new file +30 -0
@@ -0,0 +1,30 @@ +/* +* CCI support +* +* Copyright (C) 2012-2013 ARM Ltd. +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, write to the Free Software +* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +*/ + +#ifndef __LINUX_ARM_CCI_H +#define __LINUX_ARM_CCI_H + +#ifdef CONFIG_ARM_CCI +extern void disable_cci (int cluster); +#else +static inline void disable_cci (int cluster) { } +#endif + +#endif
Signature · 1 line
1.8.1.2
PATCH v4 11/15 drivers/bus: add ARM CCI support
drivers/bus/arm-cci.c line 48
Quoted 10 lines · Nicolas Pitre
drivers/bus/arm-cci.c new file +10
-0+void disable_cci (int cluster) +{ +u32 slave_reg = cluster ?CCI400_KF_OFFSET :CCI400_EAG_OFFSET ; +writel_relaxed (0x0 , info->baseaddr + slave_reg); + +while (readl_relaxed (info->baseaddr +CCI_STATUS_OFFSET ) + &STATUS_CHANGE_PENDING ) +barrier (); +} +EXPORT_SYMBOL_GPL (disable_cci);
This will blow up if the cci driver hasn't been probed - which I guess is fine.
Quoted 17 lines
drivers/bus/arm-cci.c new file +17
-0+ +static int cci_driver_probe (struct platform_device *pdev) +{ +struct resource *res; +int ret =0 ; + + info =kzalloc (sizeof (*info),GFP_KERNEL ); +if (!info) { +dev_err (&pdev->dev,"unable to allocate mem\n" ); +return -ENOMEM ; + } + + res =platform_get_resource (pdev,IORESOURCE_MEM ,0 ); +if (!res) { +dev_err (&pdev->dev,"No memory resource\n" ); + ret = -EINVAL ; + goto mem_free;
drivers/bus/arm-cci.c line 75
Quoted 10 lines
drivers/bus/arm-cci.c new file +10
-0+ } + +if (!request_mem_region (res->start,resource_size (res), +dev_name (&pdev->dev))) { +dev_err (&pdev->dev,"address 0x%x in use\n" , (u32 ) res->start); + ret = -EBUSY ; + goto mem_free; + } + + info->baseaddr =ioremap (res->start,resource_size (res));
As we are moving stuff over to the devm_* APIs, it would be a good idea to avoid introducing new code not using those APIs - otherwise it just creates yet more churn, something which Linus objects to. Can we please try in future to avoid creating stuff which then needs to be subsequently modified for the latest ways to do stuff...
PATCH v4 11/15 drivers/bus: add ARM CCI support
Quoted 40 lines · Russell King - ARM Linux
drivers/bus/arm-cci.c line 48
Quoted 10 lines · Nicolas Pitre
drivers/bus/arm-cci.c new file +10
-0+void disable_cci (int cluster) +{ +u32 slave_reg = cluster ?CCI400_KF_OFFSET :CCI400_EAG_OFFSET ; +writel_relaxed (0x0 , info->baseaddr + slave_reg); + +while (readl_relaxed (info->baseaddr +CCI_STATUS_OFFSET ) + &STATUS_CHANGE_PENDING ) +barrier (); +} +EXPORT_SYMBOL_GPL (disable_cci);This will blow up if the cci driver hasn't been probed - which I guess is fine.
Quoted 25 lines
drivers/bus/arm-cci.c new file +25
-0+ +static int cci_driver_probe (struct platform_device *pdev) +{ +struct resource *res; +int ret =0 ; + + info =kzalloc (sizeof (*info),GFP_KERNEL ); +if (!info) { +dev_err (&pdev->dev,"unable to allocate mem\n" ); +return -ENOMEM ; + } + + res =platform_get_resource (pdev,IORESOURCE_MEM ,0 ); +if (!res) { +dev_err (&pdev->dev,"No memory resource\n" ); + ret = -EINVAL ; + goto mem_free; + } + +if (!request_mem_region (res->start,resource_size (res), +dev_name (&pdev->dev))) { +dev_err (&pdev->dev,"address 0x%x in use\n" , (u32 ) res->start); + ret = -EBUSY ; + goto mem_free; + }
Quoted 8 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ + info->baseaddr = ioremap(res->start, resource_size(res));As we are moving stuff over to the devm_* APIs, it would be a good idea to avoid introducing new code not using those APIs - otherwise it just creates yet more churn, something which Linus objects to. Can we please try in future to avoid creating stuff which then needs to be subsequently modified for the latest ways to do stuff...
This driver is not part of the MCPM pull request. It was provided to illustrate how the MCPM layer is meant to be used. The on-going discussion about CPU/cluster/CCI deviice tree bindings by Lorenzo will significantly enhance this driver. And given the current timing it is likely that the enhanced driver will simply replace this one during the next cycle, so we should take your comment into account.
Nicolas
PATCH v4 12/15 ARM: CCI: ensure powerdown-time data is flushed from cache
In reply to Nicolas PitreFrom: Dave Martin <[email protected]>
Non-local variables used by the CCI management function called after disabling the cache must be flushed out to main memory in advance, otherwise incoherency of those values may occur if they are sitting in the cache of some other CPU when cci_disable() executes.
This patch adds the appropriate flushing to the CCI driver to ensure that the relevant data is available in RAM ahead of time.
Because this creates a dependency on arch-specific cacheflushing functions, this patch also makes ARM_CCI depend on ARM.
--- drivers/bus/Kconfig | 1 + drivers/bus/arm-cci.c | 13 +++++++++++++ 2 files changed, 14 insertions(+)
drivers/bus/Kconfig +1 -0
@@ -22,5 +22,6 @@ config OMAP_INTERCONNECT config ARM_CCI bool "ARM CCI driver support" + depends on ARM endmenu
drivers/bus/arm-cci.c +13 -0
@@ -21,6 +21,10 @@#include <linux/slab.h> #include <linux/arm-cci.h> +#include <asm/cacheflush.h> +#include <asm/memory.h> +#include <asm/outercache.h> +#define CCI_STATUS_OFFSET 0xc#define STATUS_CHANGE_PENDING (1 << 0)
@@ -78,6 +82,15 @@ static int cci_driver_probe(struct platform_device *pdev) goto ioremap_err; } +/* +* Multi-cluster systems may need this data when non-coherent, during +* cluster power-up/power-down. Make sure it reaches main memory: +*/ +__cpuc_flush_dcache_area (info,sizeof *info); +__cpuc_flush_dcache_area (&info,sizeof info); +outer_clean_range (virt_to_phys (info),virt_to_phys (info +1 )); +outer_clean_range (virt_to_phys (&info),virt_to_phys (&info +1 )); +platform_set_drvdata (pdev, info);pr_info ("CCI loaded@%p\n" , info->baseaddr);
Signature · 1 line
1.8.1.2
PATCH v4 12/15 ARM: CCI: ensure powerdown-time data is flushed from cache
drivers/bus/arm-cci.c static int cci_driver_probe(struct platform_device *pdev) · line 92
Quoted 8 lines · Nicolas Pitre
drivers/bus/arm-cci.c +8
-0+/* +* Multi-cluster systems may need this data when non-coherent, during +* cluster power-up/power-down. Make sure it reaches main memory: +*/ +__cpuc_flush_dcache_area (info,sizeof *info); +__cpuc_flush_dcache_area (&info,sizeof info); +outer_clean_range (virt_to_phys (info),virt_to_phys (info +1 )); +outer_clean_range (virt_to_phys (&info),virt_to_phys (&info +1 ));
This seems to be a recurring theme throughout these patches. What it's saying is that we need a proper way to do this, rather than keep on open coding this same thing time and time again. So, it's do something about that rather than having to go back and rework all this stuff later, creating "pointless churn" that Linus really hates.
PATCH v4 13/15 ARM: vexpress/dcscb: handle platform coherency exit/setup and CCI
In reply to Nicolas PitreFrom: Dave Martin <[email protected]>
Add the required code to properly handle race free platform coherency exit to the DCSCB power down method.
The power_up_setup callback is used to enable the CCI interface for the cluster being brought up. This must be done in assembly before the kernel environment is entered.
Thanks to Achin Gupta and Nicolas Pitre for their help and contributions.
--- arch/arm/mach-vexpress/Kconfig | 1 + arch/arm/mach-vexpress/Makefile | 2 +- arch/arm/mach-vexpress/dcscb.c | 74 ++++++++++++++++++++++++--------- arch/arm/mach-vexpress/dcscb_setup.S | 80 ++++++++++++++++++++++++++++++++++++ 4 files changed, 137 insertions(+), 20 deletions(-) create mode 100644 arch/arm/mach-vexpress/dcscb_setup.S
arch/arm/mach-vexpress/Kconfig +1 -0
@@ -55,6 +55,7 @@ config ARCH_VEXPRESS_CA9X4 config ARCH_VEXPRESS_DCSCB bool "Dual Cluster System Control Block (DCSCB) support" depends on CLUSTER_PM + select ARM_CCI help Support for the Dual Cluster System Configuration Block (DCSCB). This is needed to provide CPU and cluster power management
arch/arm/mach-vexpress/Makefile +1 -1
@@ -6,6 +6,6 @@ ccflags-$(CONFIG_ARCH_MULTIPLATFORM) := -I$(srctree)/$(src)/include \ obj-y := v2m.o reset.o obj-$(CONFIG_ARCH_VEXPRESS_CA9X4) += ct-ca9x4.o -obj-$(CONFIG_ARCH_VEXPRESS_DCSCB) += dcscb.o +obj-$(CONFIG_ARCH_VEXPRESS_DCSCB) += dcscb.o dcscb_setup.o obj-$(CONFIG_SMP) += platsmp.o obj-$(CONFIG_HOTPLUG_CPU) += hotplug.o
arch/arm/mach-vexpress/dcscb.c +55 -19
116 lines not on this page · raw message
arch/arm/mach-vexpress/dcscb_setup.S new file +80 -0
80 lines not on this page · raw message
Signature · 1 line
1.8.1.2
PATCH v4 14/15 ARM: Enable selection of SMP operations at boot time
In reply to Nicolas PitreFrom: Jon Medhurst <[email protected]>
Add a new 'smp_init' hook to machine_desc so platforms can specify a function to be used to setup smp ops instead of having a statically defined value.
--- arch/arm/include/asm/mach/arch.h | 3 +++ arch/arm/kernel/setup.c | 5 ++++- 2 files changed, 7 insertions(+), 1 deletion(-)
arch/arm/include/asm/mach/arch.h +3 -0
@@ -17,8 +17,10 @@ struct pt_regs;struct smp_operations ;#ifdef CONFIG_SMP #define smp_ops (ops) (&(ops)) +#define smp_init_ops (ops) (&(ops))#else #define smp_ops (ops) (struct smp_operations *)NULL +#define smp_init_ops (ops) (void (*)(void))NULL#endif struct machine_desc {
@@ -42,6 +44,7 @@ struct machine_desc { unsigned char reserve_lp2 :1; /* never has lp2 */char restart_mode;/* default restart mode */ struct smp_operations *smp;/* SMP operations */ +void (*smp_init)(void );void (*fixup)(struct tag *,char **,struct meminfo *);void (*reserve)(void );/* reserve mem blocks */
arch/arm/kernel/setup.c +4 -1
@@ -768,7 +768,10 @@ void __init setup_arch(char **cmdline_p)arm_dt_init_cpu_maps ();#ifdef CONFIG_SMP if (is_smp ()) { -smp_set_ops (mdesc->smp); +if (mdesc->smp_init) + (*mdesc->smp_init)(); +else +smp_set_ops (mdesc->smp);smp_init_cpus (); }#endif
Signature · 1 line
1.8.1.2
PATCH v4 14/15 ARM: Enable selection of SMP operations at boot time
Quoted 42 lines · Nicolas Pitre
From: Jon Medhurst <[email protected]>
Add a new '
smp_init' hook tomachine_descso platforms can specify a function to be used to setup smp ops instead of having a statically defined value.Signed-off-by Jon Medhurst <[email protected]>Signed-off-by Nicolas Pitre <[email protected]>Reviewed-by Santosh Shilimkar <[email protected]>--- arch/arm/include/asm/mach/arch.h | 3 +++ arch/arm/kernel/setup.c | 5 ++++- 2 files changed, 7 insertions(+), 1 deletion(-)arch/arm/include/asm/mach/arch.h +3
-0@@ -17,8 +17,10 @@ struct pt_regs;struct smp_operations ;#ifdef CONFIG_SMP #define smp_ops (ops) (&(ops)) +#define smp_init_ops (ops) (&(ops))#else #define smp_ops (ops) (struct smp_operations *)NULL +#define smp_init_ops (ops) (void (*)(void))NULL#endif struct machine_desc {@@ -42,6 +44,7 @@ struct machine_desc { unsigned char reserve_lp2 :1; /* never has lp2 */char restart_mode;/* default restart mode */ struct smp_operations *smp;/* SMP operations */ +void (*smp_init)(void );void (*fixup)(struct tag *,char **,struct meminfo *);void (*reserve)(void );/* reserve mem blocks */ arch/arm/kernel/setup.c +0
-0
arch/arm/kernel/setup.c void __init setup_arch(char **cmdline_p) · line 771
Quoted 5 lines
arch/arm/kernel/setup.c +1
-1@@ -768,7 +768,10 @@ void __init setup_arch(char **cmdline_p)arm_dt_init_cpu_maps ();#ifdef CONFIG_SMP if (is_smp ()) { -smp_set_ops (mdesc->smp); +if (mdesc->smp_init)
This will fail checkpatch: if() instead of if ().
arch/arm/kernel/setup.c void __init setup_arch(char **cmdline_p) · line 772
Quoted 1 line
arch/arm/kernel/setup.c +1
-0+ (*mdesc->smp_init)();
This is different calling style than init_early() below, which uses mdesc->init_early(). Please be consistent.
arch/arm/kernel/setup.c void __init setup_arch(char **cmdline_p) · line 774
Quoted 2 lines
arch/arm/kernel/setup.c +2
-0+else +smp_set_ops (mdesc->smp);
-Olof
PATCH v4 14/15 ARM: Enable selection of SMP operations at boot time
Quoted 42 lines · Olof Johansson
Quoted 41 lines · Nicolas Pitre
From: Jon Medhurst <[email protected]>
Add a new '
smp_init' hook tomachine_descso platforms can specify a function to be used to setup smp ops instead of having a statically defined value.Signed-off-by Jon Medhurst <[email protected]>Signed-off-by Nicolas Pitre <[email protected]>Reviewed-by Santosh Shilimkar <[email protected]>--- arch/arm/include/asm/mach/arch.h | 3 +++ arch/arm/kernel/setup.c | 5 ++++- 2 files changed, 7 insertions(+), 1 deletion(-)arch/arm/include/asm/mach/arch.h +3
-0@@ -17,8 +17,10 @@ struct pt_regs;struct smp_operations ;#ifdef CONFIG_SMP #define smp_ops (ops) (&(ops)) +#define smp_init_ops (ops) (&(ops))#else #define smp_ops (ops) (struct smp_operations *)NULL +#define smp_init_ops (ops) (void (*)(void))NULL#endif struct machine_desc {@@ -42,6 +44,7 @@ struct machine_desc { unsigned char reserve_lp2 :1; /* never has lp2 */char restart_mode;/* default restart mode */ struct smp_operations *smp;/* SMP operations */ +void (*smp_init)(void );void (*fixup)(struct tag *,char **,struct meminfo *);void (*reserve)(void );/* reserve mem blocks */ arch/arm/kernel/setup.c +0
-0
Quoted 8 lines
arch/arm/kernel/setup.c void __init setup_arch(char **cmdline_p) · line 771
Quoted 6 lines
arch/arm/kernel/setup.c +1
-1@@ -768,7 +768,10 @@ void __init setup_arch(char **cmdline_p)arm_dt_init_cpu_maps ();#ifdef CONFIG_SMP if (is_smp ()) { -smp_set_ops (mdesc->smp); +if (mdesc->smp_init)This will fail checkpatch:
if()instead of if ().
Indeed.
Quoted 4 lines
arch/arm/kernel/setup.c void __init setup_arch(char **cmdline_p) · line 772
Quoted 1 line
arch/arm/kernel/setup.c +1
-0+ (*mdesc->smp_init)();This is different calling style than
init_early()below, which usesmdesc->init_early(). Please be consistent.
Fixed in my tree now.
Thanks.
Nicolas
PATCH v4 14/15 ARM: Enable selection of SMP operations at boot time
In reply to Nicolas PitreQuoted 9 lines · Nicolas Pitre
From: Jon Medhurst <[email protected]>
Add a new '
smp_init' hook tomachine_descso platforms can specify a function to be used to setup smp ops instead of having a statically defined value.Signed-off-by Jon Medhurst <[email protected]>Signed-off-by Nicolas Pitre <[email protected]>Reviewed-by Santosh Shilimkar <[email protected]>
I've slightly amended this patch to make its usage more flexible, please see below.
Quoted 29 lines
arch/arm/include/asm/mach/arch.h +3
-0@@ -17,8 +17,10 @@ struct pt_regs;struct smp_operations ;#ifdef CONFIG_SMP #define smp_ops (ops) (&(ops)) +#define smp_init_ops (ops) (&(ops))#else #define smp_ops (ops) (struct smp_operations *)NULL +#define smp_init_ops (ops) (void (*)(void))NULL#endif struct machine_desc {@@ -42,6 +44,7 @@ struct machine_desc { unsigned char reserve_lp2 :1; /* never has lp2 */char restart_mode;/* default restart mode */ struct smp_operations *smp;/* SMP operations */ +void (*smp_init)(void );void (*fixup)(struct tag *,char **,struct meminfo *);void (*reserve)(void );/* reserve mem blocks */ arch/arm/kernel/setup.c +0
-0@@ -768,7 +768,10 @@ void __init setup_arch(char **cmdline_p)arm_dt_init_cpu_maps ();
arch/arm/kernel/setup.c void __init setup_arch(char **cmdline_p) · line 777
Quoted 10 lines
arch/arm/kernel/setup.c +4
-1#ifdef CONFIG_SMP if (is_smp ()) { -smp_set_ops (mdesc->smp); +if (mdesc->smp_init) + (*mdesc->smp_init)(); +else +smp_set_ops (mdesc->smp);smp_init_cpus (); }#endif
I've amended it with the following changes to deal with an issue highlighted by Tixy. If the runtime hook does not initialize the smp ops, the core may continue with a default. That should let MCPM, PSCI and Xen play well together.
arch/arm/include/asm/mach/arch.h +2 -2
@@ -19,7 +19,7 @@ struct smp_operations;#define smp_init_ops (ops) (&(ops))#else #define smp_ops (ops) (struct smp_operations *)NULL -#define smp_init_ops (ops) (void (*)(void))NULL +#define smp_init_ops (ops) (bool (*)(void))NULL#endif struct machine_desc {
@@ -43,7 +43,7 @@ struct machine_desc { unsigned char reserve_lp2 :1; /* never has lp2 */char restart_mode;/* default restart mode */ struct smp_operations *smp;/* SMP operations */ -void (*smp_init)(void ); +bool (*smp_init)(void );void (*fixup)(struct tag *,char **,struct meminfo *);void (*reserve)(void );/* reserve mem blocks */
arch/arm/kernel/setup.c +1 -3
@@ -768,9 +768,7 @@ void __init setup_arch(char **cmdline_p)arm_dt_init_cpu_maps ();#ifdef CONFIG_SMP if (is_smp ()) { -if (mdesc->smp_init) - mdesc->smp_init (); -else +if (!mdesc->smp_init || !mdesc->smp_init ())smp_set_ops (mdesc->smp);smp_init_cpus (); }
Nicolas
PATCH v4 14/15 ARM: Enable selection of SMP operations at boot time
Quoted 8 lines · Nicolas Pitre
Quoted 6 lines · Nicolas Pitre
From: Jon Medhurst <[email protected]>
Add a new '
smp_init' hook tomachine_descso platforms can specify a function to be used to setup smp ops instead of having a statically defined value.
Quoted 6 lines
Quoted 3 lines
Signed-off-by Jon Medhurst <[email protected]>Signed-off-by Nicolas Pitre <[email protected]>Reviewed-by Santosh Shilimkar <[email protected]>I've slightly amended this patch to make its usage more flexible, please see below.
Looks good to me.
Quoted 85 lines
Quoted 29 lines
arch/arm/include/asm/mach/arch.h +3
-0@@ -17,8 +17,10 @@ struct pt_regs;struct smp_operations ;#ifdef CONFIG_SMP #define smp_ops (ops) (&(ops)) +#define smp_init_ops (ops) (&(ops))#else #define smp_ops (ops) (struct smp_operations *)NULL +#define smp_init_ops (ops) (void (*)(void))NULL#endif struct machine_desc {@@ -42,6 +44,7 @@ struct machine_desc { unsigned char reserve_lp2 :1; /* never has lp2 */char restart_mode;/* default restart mode */ struct smp_operations *smp;/* SMP operations */ +void (*smp_init)(void );void (*fixup)(struct tag *,char **,struct meminfo *);void (*reserve)(void );/* reserve mem blocks */ arch/arm/kernel/setup.c +0
-0@@ -768,7 +768,10 @@ void __init setup_arch(char **cmdline_p)arm_dt_init_cpu_maps ();arch/arm/kernel/setup.c void __init setup_arch(char **cmdline_p) · line 777
Quoted 10 lines
arch/arm/kernel/setup.c +4
-1#ifdef CONFIG_SMP if (is_smp ()) { -smp_set_ops (mdesc->smp); +if (mdesc->smp_init) + (*mdesc->smp_init)(); +else +smp_set_ops (mdesc->smp);smp_init_cpus (); }#endif I've amended it with the following changes to deal with an issue highlighted by Tixy. If the runtime hook does not initialize the smp ops, the core may continue with a default. That should let MCPM, PSCI and Xen play well together.
arch/arm/include/asm/mach/arch.h +2
-2@@ -19,7 +19,7 @@ struct smp_operations;#define smp_init_ops (ops) (&(ops))#else #define smp_ops (ops) (struct smp_operations *)NULL -#define smp_init_ops (ops) (void (*)(void))NULL +#define smp_init_ops (ops) (bool (*)(void))NULL#endif struct machine_desc {@@ -43,7 +43,7 @@ struct machine_desc { unsigned char reserve_lp2 :1; /* never has lp2 */char restart_mode;/* default restart mode */ struct smp_operations *smp;/* SMP operations */ -void (*smp_init)(void ); +bool (*smp_init)(void );void (*fixup)(struct tag *,char **,struct meminfo *);void (*reserve)(void );/* reserve mem blocks */ arch/arm/kernel/setup.c +1
-3@@ -768,9 +768,7 @@ void __init setup_arch(char **cmdline_p)arm_dt_init_cpu_maps ();#ifdef CONFIG_SMP if (is_smp ()) { -if (mdesc->smp_init) - mdesc->smp_init (); -else +if (!mdesc->smp_init || !mdesc->smp_init ())smp_set_ops (mdesc->smp);smp_init_cpus (); }Nicolas
PATCH v4 15/15 ARM: vexpress: Select multi-cluster SMP operation if required
In reply to Nicolas PitreFrom: Jon Medhurst <[email protected]>
--- arch/arm/mach-vexpress/core.h | 2 ++ arch/arm/mach-vexpress/platsmp.c | 12 ++++++++++++ arch/arm/mach-vexpress/v2m.c | 2 +- 3 files changed, 15 insertions(+), 1 deletion(-)
arch/arm/mach-vexpress/core.h +2 -0
@@ -6,6 +6,8 @@void vexpress_dt_smp_map_io (void ); +void vexpress_smp_init_ops (void ); +extern struct smp_operations vexpress_smp_ops;extern void vexpress_cpu_die (unsigned int cpu);
arch/arm/mach-vexpress/platsmp.c +12 -0
@@ -12,6 +12,7 @@#include <linux/errno.h> #include <linux/smp.h> #include <linux/io.h> +#include <linux/of.h> #include <linux/of_fdt.h> #include <linux/vexpress.h>
@@ -206,3 +207,14 @@ struct smp_operations __initdata vexpress_smp_ops = { .cpu_die = vexpress_cpu_die,#endif }; + +void __init vexpress_smp_init_ops (void ) +{ +struct smp_operations *ops = &vexpress_smp_ops; +#ifdef CONFIG_CLUSTER_PM +extern struct smp_operations mcpm_smp_ops; +if (of_find_compatible_node (NULL ,NULL ,"arm,cci" )) + ops = &mcpm_smp_ops; +#endif +smp_set_ops (ops); +}
arch/arm/mach-vexpress/v2m.c +1 -1
@@ -494,7 +494,7 @@ static const char * const v2m_dt_match[] __initconst = {DT_MACHINE_START (VEXPRESS_DT ,"ARM-Versatile Express" ) .dt_compat = v2m_dt_match, - .smp = smp_ops(vexpress_smp_ops), + .smp_init = smp_init_ops(vexpress_smp_init_ops), .map_io = v2m_dt_map_io, .init_early = v2m_dt_init_early, .init_irq = v2m_dt_init_irq,
Signature · 1 line
1.8.1.2
PATCH v4 15/15 ARM: vexpress: Select multi-cluster SMP operation if required
Quoted 10 lines · Nicolas Pitre
From: Jon Medhurst <[email protected]>
Signed-off-by Jon Medhurst <[email protected]>Signed-off-by Nicolas Pitre <[email protected]>Reviewed-by Santosh Shilimkar <[email protected]>--- arch/arm/mach-vexpress/core.h | 2 ++ arch/arm/mach-vexpress/platsmp.c | 12 ++++++++++++ arch/arm/mach-vexpress/v2m.c | 2 +- 3 files changed, 15 insertions(+), 1 deletion(-)
For this and the other "ARM: vexpress" patches in the series:
Thanks!
Pawel
PATCH v4 15/15 ARM: vexpress: Select multi-cluster SMP operation if required
Quoted 12 lines · Pawel Moll
Quoted 10 lines · Nicolas Pitre
From: Jon Medhurst <[email protected]>
Signed-off-by Jon Medhurst <[email protected]>Signed-off-by Nicolas Pitre <[email protected]>Reviewed-by Santosh Shilimkar <[email protected]>--- arch/arm/mach-vexpress/core.h | 2 ++ arch/arm/mach-vexpress/platsmp.c | 12 ++++++++++++ arch/arm/mach-vexpress/v2m.c | 2 +- 3 files changed, 15 insertions(+), 1 deletion(-)
Quoted 3 lines
For this and the other "ARM: vexpress" patches in the series:
Acked-by Pawel Moll <[email protected]>
Thanks.
Nicolas
PATCH v4 15/15 ARM: vexpress: Select multi-cluster SMP operation if required
In reply to Nicolas PitreQuoted 37 lines · Nicolas Pitre
From: Jon Medhurst <[email protected]>
Signed-off-by Jon Medhurst <[email protected]>Signed-off-by Nicolas Pitre <[email protected]>Reviewed-by Santosh Shilimkar <[email protected]>--- arch/arm/mach-vexpress/core.h | 2 ++ arch/arm/mach-vexpress/platsmp.c | 12 ++++++++++++ arch/arm/mach-vexpress/v2m.c | 2 +- 3 files changed, 15 insertions(+), 1 deletion(-)arch/arm/mach-vexpress/core.h +2
-0@@ -6,6 +6,8 @@void vexpress_dt_smp_map_io (void ); +void vexpress_smp_init_ops (void ); +extern struct smp_operations vexpress_smp_ops;extern void vexpress_cpu_die (unsigned int cpu);arch/arm/mach-vexpress/platsmp.c +1
-0@@ -12,6 +12,7 @@#include <linux/errno.h> #include <linux/smp.h> #include <linux/io.h> +#include <linux/of.h> #include <linux/of_fdt.h> #include <linux/vexpress.h>
arch/arm/mach-vexpress/platsmp.c struct smp_operations __initdata vexpress_smp_ops = { · line 215
Quoted 9 lines
arch/arm/mach-vexpress/platsmp.c +6
-0@@ -206,3 +207,14 @@ struct smp_operations __initdata vexpress_smp_ops = { .cpu_die = vexpress_cpu_die,#endif }; + +void __init vexpress_smp_init_ops (void ) +{ +struct smp_operations *ops = &vexpress_smp_ops; +#ifdef CONFIG_CLUSTER_PM +extern struct smp_operations mcpm_smp_ops;
Seems appropriate to put this prototype in a header file instead.
arch/arm/mach-vexpress/platsmp.c struct smp_operations __initdata vexpress_smp_ops = { · line 216
Quoted 1 line
arch/arm/mach-vexpress/platsmp.c +1
-0+if (of_find_compatible_node (NULL ,NULL ,"arm,cci" ))
Another checkpatch error on if() whitespace.
Also, while bindings haven't been ironed out, checking for whether the node/device is enabled or disabled could be appropriate here?
Quoted 9 lines
arch/arm/mach-vexpress/platsmp.c +4
-0+ ops = &mcpm_smp_ops; +#endif +smp_set_ops (ops); +}arch/arm/mach-vexpress/v2m.c +0
-0
arch/arm/mach-vexpress/v2m.c static const char * const v2m_dt_match[] __initconst = { · line 500
Quoted 8 lines
arch/arm/mach-vexpress/v2m.c +1
-1@@ -494,7 +494,7 @@ static const char * const v2m_dt_match[] __initconst = {DT_MACHINE_START (VEXPRESS_DT ,"ARM-Versatile Express" ) .dt_compat = v2m_dt_match, - .smp = smp_ops(vexpress_smp_ops), + .smp_init = smp_init_ops(vexpress_smp_init_ops), .map_io = v2m_dt_map_io, .init_early = v2m_dt_init_early, .init_irq = v2m_dt_init_irq,
-Olof
PATCH v4 15/15 ARM: vexpress: Select multi-cluster SMP operation if required
Quoted 50 lines · Olof Johansson
Quoted 37 lines · Nicolas Pitre
From: Jon Medhurst <[email protected]>
Signed-off-by Jon Medhurst <[email protected]>Signed-off-by Nicolas Pitre <[email protected]>Reviewed-by Santosh Shilimkar <[email protected]>--- arch/arm/mach-vexpress/core.h | 2 ++ arch/arm/mach-vexpress/platsmp.c | 12 ++++++++++++ arch/arm/mach-vexpress/v2m.c | 2 +- 3 files changed, 15 insertions(+), 1 deletion(-)arch/arm/mach-vexpress/core.h +2
-0@@ -6,6 +6,8 @@void vexpress_dt_smp_map_io (void ); +void vexpress_smp_init_ops (void ); +extern struct smp_operations vexpress_smp_ops;extern void vexpress_cpu_die (unsigned int cpu);arch/arm/mach-vexpress/platsmp.c +1
-0@@ -12,6 +12,7 @@#include <linux/errno.h> #include <linux/smp.h> #include <linux/io.h> +#include <linux/of.h> #include <linux/of_fdt.h> #include <linux/vexpress.h> arch/arm/mach-vexpress/platsmp.c struct smp_operations __initdata vexpress_smp_ops = { · line 215
Quoted 9 lines
arch/arm/mach-vexpress/platsmp.c +6
-0@@ -206,3 +207,14 @@ struct smp_operations __initdata vexpress_smp_ops = { .cpu_die = vexpress_cpu_die,#endif }; + +void __init vexpress_smp_init_ops (void ) +{ +struct smp_operations *ops = &vexpress_smp_ops; +#ifdef CONFIG_CLUSTER_PM +extern struct smp_operations mcpm_smp_ops;Seems appropriate to put this prototype in a header file instead.
Quoted 6 lines
arch/arm/mach-vexpress/platsmp.c struct smp_operations __initdata vexpress_smp_ops = { · line 216
Quoted 1 line
arch/arm/mach-vexpress/platsmp.c +1
-0+if (of_find_compatible_node (NULL ,NULL ,"arm,cci" ))Another checkpatch error on
if()whitespace.Also, while bindings haven't been ironed out, checking for whether the node/device is enabled or disabled could be appropriate here?
Right.
I've amended this patch slightly so to:
1) Keep the default .smp = &vexpress_smp_ops and only perform an override in vexpress_smp_init_ops() when appropriate. This should remove one potential issue with Xen support that was highlighted recently.
2) Add a mcpm_smp_set_ops() instead of installing mcpm_smp_ops directly for better abstraction.
3) Added a comment about and checked for the CCI node being enabled.
That results in the patch below. Given those are minor changes, I kept the existing review tags.
From: Jon Medhurst <[email protected]>
Date: Wed, 30 Jan 2013 09:12:55 +0000
Subject: [PATCH] ARM: vexpress: Select multi-cluster SMP operation if required
arch/arm/common/mcpm_platsmp.c +5 -0
@@ -85,3 +85,8 @@ struct smp_operations __initdata mcpm_smp_ops = { .cpu_die = mcpm_cpu_die,#endif }; + +void __init mcpm_smp_set_ops (void ) +{ +smp_set_ops (&mcpm_smp_ops); +}
arch/arm/include/asm/mcpm_entry.h +2 -0
@@ -190,5 +190,7 @@ int __mcpm_cluster_state(unsigned int cluster);int __init mcpm_sync_init (void (*power_up_setup)(unsigned int affinity_level)); +void __init mcpm_smp_set_ops (void ); +#endif /* ! __ASSEMBLY__ */ #endif
arch/arm/mach-vexpress/core.h +2 -0
@@ -6,6 +6,8 @@void vexpress_dt_smp_map_io (void ); +void vexpress_smp_init_ops (void ); +extern struct smp_operations vexpress_smp_ops;extern void vexpress_cpu_die (unsigned int cpu);
arch/arm/mach-vexpress/platsmp.c +17 -0
@@ -12,9 +12,11 @@#include <linux/errno.h> #include <linux/smp.h> #include <linux/io.h> +#include <linux/of.h> #include <linux/of_fdt.h> #include <linux/vexpress.h> +#include <asm/mcpm_entry.h> #include <asm/smp_scu.h> #include <asm/mach/map.h>
@@ -203,3 +205,18 @@ struct smp_operations __initdata vexpress_smp_ops = { .cpu_die = vexpress_cpu_die,#endif }; + +void __init vexpress_smp_init_ops (void ) +{ +#ifdef CONFIG_MCPM +/* +* The best way to detect a multi-cluster configuration at the moment +* is to look for the presence of a CCI in the system. +* Override the default vexpress_smp_ops if so. +*/ +struct device_node *node; + node =of_find_compatible_node (NULL ,NULL ,"arm,cci" ); +if (node &&of_device_is_available (node)) +mcpm_smp_set_ops (); +#endif +}
arch/arm/mach-vexpress/v2m.c +1 -0
@@ -476,6 +476,7 @@ static const char * const v2m_dt_match[] __initconst = {DT_MACHINE_START (VEXPRESS_DT ,"ARM-Versatile Express" ) .dt_compat = v2m_dt_match, .smp = smp_ops(vexpress_smp_ops), + .smp_init = smp_init_ops(vexpress_smp_init_ops), .map_io = v2m_dt_map_io, .init_early = v2m_dt_init_early, .init_irq = irqchip_init,
PATCH v4 15/15 ARM: vexpress: Select multi-cluster SMP operation if required
Quoted 42 lines · Nicolas Pitre
I've amended this patch slightly so to:
1) Keep the default .smp =
&vexpress_smp_opsand only perform an override invexpress_smp_init_ops()when appropriate. This should remove one potential issue with Xen support that was highlighted recently.2) Add a
mcpm_smp_set_ops()instead of installingmcpm_smp_opsdirectly for better abstraction.3) Added a comment about and checked for the CCI node being enabled.
That results in the patch below. Given those are minor changes, I kept the existing review tags.
From: Jon Medhurst <[email protected]>
Date: Wed, 30 Jan 2013 09:12:55 +0000
Subject: [PATCH] ARM: vexpress: Select multi-cluster SMP operation if requiredSigned-off-by Jon Medhurst <[email protected]>Signed-off-by Nicolas Pitre <[email protected]>Reviewed-by Santosh Shilimkar <[email protected]>Acked-by Pawel Moll <[email protected]>arch/arm/common/mcpm_platsmp.c +5
-0@@ -85,3 +85,8 @@ struct smp_operations __initdata mcpm_smp_ops = { .cpu_die = mcpm_cpu_die,#endif }; + +void __init mcpm_smp_set_ops (void ) +{ +smp_set_ops (&mcpm_smp_ops); +}arch/arm/include/asm/mcpm_entry.h +0
-0
arch/arm/include/asm/mcpm_entry.h int __mcpm_cluster_state(unsigned int cluster); · line 196
Quoted 7 lines
arch/arm/include/asm/mcpm_entry.h +2
-0@@ -190,5 +190,7 @@ int __mcpm_cluster_state(unsigned int cluster);int __init mcpm_sync_init (void (*power_up_setup)(unsigned int affinity_level)); +void __init mcpm_smp_set_ops (void ); +#endif /* ! __ASSEMBLY__ */ #endif
Do the changes to the above mcpm files want to be in a separate patch as it's generic ARM code, not vexpress specific?
Quoted 38 lines
arch/arm/mach-vexpress/core.h +2
-0@@ -6,6 +6,8 @@void vexpress_dt_smp_map_io (void ); +void vexpress_smp_init_ops (void ); +extern struct smp_operations vexpress_smp_ops;extern void vexpress_cpu_die (unsigned int cpu);arch/arm/mach-vexpress/platsmp.c +7
-0@@ -12,9 +12,11 @@#include <linux/errno.h> #include <linux/smp.h> #include <linux/io.h> +#include <linux/of.h> #include <linux/of_fdt.h> #include <linux/vexpress.h> +#include <asm/mcpm_entry.h> #include <asm/smp_scu.h> #include <asm/mach/map.h> @@ -203,3 +205,18 @@ struct smp_operations __initdata vexpress_smp_ops = { .cpu_die = vexpress_cpu_die,#endif }; + +void __init vexpress_smp_init_ops (void ) +{ +#ifdef CONFIG_MCPM +/*
arch/arm/mach-vexpress/platsmp.c struct smp_operations __initdata vexpress_smp_ops = { · line 222
Quoted 10 lines
arch/arm/mach-vexpress/platsmp.c +10
-0+* The best way to detect a multi-cluster configuration at the moment +* is to look for the presence of a CCI in the system. +* Override the default vexpress_smp_ops if so. +*/ +struct device_node *node; + node =of_find_compatible_node (NULL ,NULL ,"arm,cci" ); +if (node &&of_device_is_available (node)) +mcpm_smp_set_ops (); +#endif +}
What now sets smp_ops if mcpm_smp_set_ops doesn't get called above? I know there are umpteen versions of the patch "ARM: Enable selection of SMP operations at boot time", but none of them seem to resolve that issue.
Quoted 12 lines
arch/arm/mach-vexpress/v2m.c +1
-0@@ -476,6 +476,7 @@ static const char * const v2m_dt_match[] __initconst = {DT_MACHINE_START (VEXPRESS_DT ,"ARM-Versatile Express" ) .dt_compat = v2m_dt_match, .smp = smp_ops(vexpress_smp_ops), + .smp_init = smp_init_ops(vexpress_smp_init_ops), .map_io = v2m_dt_map_io, .init_early = v2m_dt_init_early, .init_irq = irqchip_init,
Signature · 1 line
Tixy
PATCH v4 15/15 ARM: vexpress: Select multi-cluster SMP operation if required
Quoted 46 lines · Jon Medhurst (Tixy)
Quoted 45 lines · Nicolas Pitre
I've amended this patch slightly so to:
1) Keep the default .smp =
&vexpress_smp_opsand only perform an override invexpress_smp_init_ops()when appropriate. This should remove one potential issue with Xen support that was highlighted recently.2) Add a
mcpm_smp_set_ops()instead of installingmcpm_smp_opsdirectly for better abstraction.3) Added a comment about and checked for the CCI node being enabled.
That results in the patch below. Given those are minor changes, I kept the existing review tags.
From: Jon Medhurst <[email protected]>
Date: Wed, 30 Jan 2013 09:12:55 +0000
Subject: [PATCH] ARM: vexpress: Select multi-cluster SMP operation if requiredSigned-off-by Jon Medhurst <[email protected]>Signed-off-by Nicolas Pitre <[email protected]>Reviewed-by Santosh Shilimkar <[email protected]>Acked-by Pawel Moll <[email protected]>arch/arm/common/mcpm_platsmp.c +5
-0@@ -85,3 +85,8 @@ struct smp_operations __initdata mcpm_smp_ops = { .cpu_die = mcpm_cpu_die,#endif }; + +void __init mcpm_smp_set_ops (void ) +{ +smp_set_ops (&mcpm_smp_ops); +}arch/arm/include/asm/mcpm_entry.h +0
-0@@ -190,5 +190,7 @@ int __mcpm_cluster_state(unsigned int cluster);int __init mcpm_sync_init (void (*power_up_setup)(unsigned int affinity_level));
Quoted 8 lines
arch/arm/include/asm/mcpm_entry.h int __mcpm_cluster_state(unsigned int cluster); · line 196
Quoted 4 lines
arch/arm/include/asm/mcpm_entry.h +2
-0+void __init mcpm_smp_set_ops (void ); +#endif /* ! __ASSEMBLY__ */ #endif Do the changes to the above mcpm files want to be in a separate patch as it's generic ARM code, not vexpress specific?
Well, since it was so trivial I didn't do it, but the split might be a good idea nevertheless. So I inserted the following patch in the series, and corresponding changes are now removed from the patch above. What do you think?
Author: Nicolas Pitre <[email protected]> Date: Tue Apr 9 01:29:17 2013 -0400
ARM: mcpm: provide an interface to set the SMP ops at run time
This is cleaner than exporting the mcpm_smp_ops structure.
arch/arm/common/mcpm_platsmp.c +6 -1
@@ -76,7 +76,7 @@ static void mcpm_cpu_die(unsigned int cpu)#endif -struct smp_operations __initdata mcpm_smp_ops = { +static struct smp_operations __initdata mcpm_smp_ops = { .smp_init_cpus = simple_smp_init_cpus, .smp_boot_secondary = mcpm_boot_secondary, .smp_secondary_init = mcpm_secondary_init,
@@ -85,3 +85,8 @@ struct smp_operations __initdata mcpm_smp_ops = { .cpu_die = mcpm_cpu_die,#endif }; + +void __init mcpm_smp_set_ops (void ) +{ +smp_set_ops (&mcpm_smp_ops); +}
arch/arm/include/asm/mcpm_entry.h +2 -0
@@ -190,5 +190,7 @@ int __mcpm_cluster_state(unsigned int cluster);int __init mcpm_sync_init (void (*power_up_setup)(unsigned int affinity_level)); +void __init mcpm_smp_set_ops (void ); +#endif /* ! __ASSEMBLY__ */ #endif
PATCH v4 15/15 ARM: vexpress: Select multi-cluster SMP operation if required
Quoted 7 lines · Nicolas Pitre
Quoted 2 lines
Do the changes to the above mcpm files want to be in a separate patch as it's generic ARM code, not vexpress specific?
Well, since it was so trivial I didn't do it, but the split might be a good idea nevertheless. So I inserted the following patch in the series, and corresponding changes are now removed from the patch above. What do you think?
Looks fine. Did you miss my other comment about smp_ops not looking like they were getting set in the case of mcpm_smp_set_ops not getting called?
Quoted 44 lines
Author: Nicolas Pitre <[email protected]> Date: Tue Apr 9 01:29:17 2013 -0400ARM: mcpm: provide an interface to set the SMP ops at run time
This is cleaner than exporting the
mcpm_smp_opsstructure.Signed-off-by Nicolas Pitre <[email protected]>arch/arm/common/mcpm_platsmp.c +6
-1@@ -76,7 +76,7 @@ static void mcpm_cpu_die(unsigned int cpu)#endif -struct smp_operations __initdata mcpm_smp_ops = { +static struct smp_operations __initdata mcpm_smp_ops = { .smp_init_cpus = simple_smp_init_cpus, .smp_boot_secondary = mcpm_boot_secondary, .smp_secondary_init = mcpm_secondary_init,@@ -85,3 +85,8 @@ struct smp_operations __initdata mcpm_smp_ops = { .cpu_die = mcpm_cpu_die,#endif }; + +void __init mcpm_smp_set_ops (void ) +{ +smp_set_ops (&mcpm_smp_ops); +}arch/arm/include/asm/mcpm_entry.h +2
-0@@ -190,5 +190,7 @@ int __mcpm_cluster_state(unsigned int cluster);int __init mcpm_sync_init (void (*power_up_setup)(unsigned int affinity_level)); +void __init mcpm_smp_set_ops (void ); +#endif /* ! __ASSEMBLY__ */ #endif
Signature · 1 line
Tixy
PATCH v4 15/15 ARM: vexpress: Select multi-cluster SMP operation if required
Quoted 10 lines · Jon Medhurst (Tixy)
Quoted 7 lines · Nicolas Pitre
Quoted 2 lines
Do the changes to the above mcpm files want to be in a separate patch as it's generic ARM code, not vexpress specific?
Well, since it was so trivial I didn't do it, but the split might be a good idea nevertheless. So I inserted the following patch in the series, and corresponding changes are now removed from the patch above. What do you think?
Looks fine.
May I add your ACK?
Quoted 2 lines
Did you miss my other comment about
smp_opsnot looking like they were getting set in the case ofmcpm_smp_set_opsnot getting called?
You are right. See my change to #14/15 to fix that.
The VExpress patch now becomes this (only change is the return value):
Author: Jon Medhurst <[email protected]> Date: Wed Jan 30 09:12:55 2013 +0000
ARM: vexpress: Select multi-cluster SMP operation if required
arch/arm/mach-vexpress/core.h +2 -0
@@ -6,6 +6,8 @@void vexpress_dt_smp_map_io (void ); +bool vexpress_smp_init_ops (void ); +extern struct smp_operations vexpress_smp_ops;extern void vexpress_cpu_die (unsigned int cpu);
arch/arm/mach-vexpress/platsmp.c +20 -0
@@ -12,9 +12,11 @@#include <linux/errno.h> #include <linux/smp.h> #include <linux/io.h> +#include <linux/of.h> #include <linux/of_fdt.h> #include <linux/vexpress.h> +#include <asm/mcpm_entry.h> #include <asm/smp_scu.h> #include <asm/mach/map.h>
@@ -203,3 +205,21 @@ struct smp_operations __initdata vexpress_smp_ops = { .cpu_die = vexpress_cpu_die,#endif }; + +bool __init vexpress_smp_init_ops (void ) +{ +#ifdef CONFIG_MCPM +/* +* The best way to detect a multi-cluster configuration at the moment +* is to look for the presence of a CCI in the system. +* Override the default vexpress_smp_ops if so. +*/ +struct device_node *node; + node =of_find_compatible_node (NULL ,NULL ,"arm,cci" ); +if (node &&of_device_is_available (node)) { +mcpm_smp_set_ops (); +return true; + } +#endif +return false; +}
arch/arm/mach-vexpress/v2m.c +1 -0
@@ -476,6 +476,7 @@ static const char * const v2m_dt_match[] __initconst = {DT_MACHINE_START (VEXPRESS_DT ,"ARM-Versatile Express" ) .dt_compat = v2m_dt_match, .smp = smp_ops(vexpress_smp_ops), + .smp_init = smp_init_ops(vexpress_smp_init_ops), .map_io = v2m_dt_map_io, .init_early = v2m_dt_init_early, .init_irq = irqchip_init,
PATCH v4 15/15 ARM: vexpress: Select multi-cluster SMP operation if required
Quoted 6 lines · Nicolas Pitre
Quoted 4 lines · Jon Medhurst (Tixy)
Quoted 3 lines · Nicolas Pitre
Quoted 2 lines
Do the changes to the above mcpm files want to be in a separate patch as it's generic ARM code, not vexpress specific?
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Well, since it was so trivial I didn't do it, but the split might be a good idea nevertheless. So I inserted the following patch in the series, and corresponding changes are now removed from the patch above. What do you think?
Looks fine.
May I add your ACK?
Yes,
Quoted 85 lines
Quoted 2 lines
Did you miss my other comment about
smp_opsnot looking like they were getting set in the case ofmcpm_smp_set_opsnot getting called?You are right. See my change to #14/15 to fix that.
The VExpress patch now becomes this (only change is the return value):
Author: Jon Medhurst <[email protected]> Date: Wed Jan 30 09:12:55 2013 +0000ARM: vexpress: Select multi-cluster SMP operation if required
Signed-off-by Jon Medhurst <[email protected]>Signed-off-by Nicolas Pitre <[email protected]>Reviewed-by Santosh Shilimkar <[email protected]>Acked-by Pawel Moll <[email protected]>arch/arm/mach-vexpress/core.h +2
-0@@ -6,6 +6,8 @@void vexpress_dt_smp_map_io (void ); +bool vexpress_smp_init_ops (void ); +extern struct smp_operations vexpress_smp_ops;extern void vexpress_cpu_die (unsigned int cpu);arch/arm/mach-vexpress/platsmp.c +20
-0@@ -12,9 +12,11 @@#include <linux/errno.h> #include <linux/smp.h> #include <linux/io.h> +#include <linux/of.h> #include <linux/of_fdt.h> #include <linux/vexpress.h> +#include <asm/mcpm_entry.h> #include <asm/smp_scu.h> #include <asm/mach/map.h> @@ -203,3 +205,21 @@ struct smp_operations __initdata vexpress_smp_ops = { .cpu_die = vexpress_cpu_die,#endif }; + +bool __init vexpress_smp_init_ops (void ) +{ +#ifdef CONFIG_MCPM +/* +* The best way to detect a multi-cluster configuration at the moment +* is to look for the presence of a CCI in the system. +* Override the default vexpress_smp_ops if so. +*/ +struct device_node *node; + node =of_find_compatible_node (NULL ,NULL ,"arm,cci" ); +if (node &&of_device_is_available (node)) { +mcpm_smp_set_ops (); +return true; + } +#endif +return false; +}arch/arm/mach-vexpress/v2m.c +1
-0@@ -476,6 +476,7 @@ static const char * const v2m_dt_match[] __initconst = {DT_MACHINE_START (VEXPRESS_DT ,"ARM-Versatile Express" ) .dt_compat = v2m_dt_match, .smp = smp_ops(vexpress_smp_ops), + .smp_init = smp_init_ops(vexpress_smp_init_ops), .map_io = v2m_dt_map_io, .init_early = v2m_dt_init_early, .init_irq = irqchip_init,_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v4 15/15 ARM: vexpress: Select multi-cluster SMP operation if required
In reply to Nicolas Pitrearch/arm/mach-vexpress/platsmp.c struct smp_operations __initdata vexpress_smp_ops = { · line 217
Quoted 7 lines · Nicolas Pitre
arch/arm/mach-vexpress/platsmp.c +7
-0+void __init vexpress_smp_init_ops (void ) +{ +struct smp_operations *ops = &vexpress_smp_ops; +#ifdef CONFIG_CLUSTER_PM +extern struct smp_operations mcpm_smp_ops; +if (of_find_compatible_node (NULL ,NULL ,"arm,cci" )) + ops = &mcpm_smp_ops;
Hmm, so these patches haven't been subjected to checkpatch treatment? As a seasoned kernel programmer, I'm surprised you haven't checked that already.
PATCH v4 15/15 ARM: vexpress: Select multi-cluster SMP operation if required
Quoted 12 lines · Russell King - ARM Linux
arch/arm/mach-vexpress/platsmp.c struct smp_operations __initdata vexpress_smp_ops = { · line 217
Quoted 7 lines · Nicolas Pitre
arch/arm/mach-vexpress/platsmp.c +7
-0+void __init vexpress_smp_init_ops (void ) +{ +struct smp_operations *ops = &vexpress_smp_ops; +#ifdef CONFIG_CLUSTER_PM +extern struct smp_operations mcpm_smp_ops; +if (of_find_compatible_node (NULL ,NULL ,"arm,cci" )) + ops = &mcpm_smp_ops;Hmm, so these patches haven't been subjected to checkpatch treatment? As a seasoned kernel programmer, I'm surprised you haven't checked that already.
They have and fixes applied, although after this set was posted. I didn't think it was worth a repost.
Nicolas
v4 cover multi-cluster power management
In reply to Nicolas PitreQuoted 4 lines · Nicolas Pitre
This is version 4 of the patch series required to safely power up and down CPUs in a cluster as can be found in b.L systems. If nothing major is reported, I'll send a pull request to Russell for this set very soon.
Okay, so from spending an hour or so quickly scanning these patches, on the face of it there are only a few comments I have which I'm very surprised that your exhaustive team of apparant reviewers didn't already catch.
The biggest disappointment so far in this patch set is finding the re-use of the repeated cache handling stuff - when I've already made my feelings _well_ known towards re-using existing cache APIs (which are designed to handle a _purpose_) for different purposes "just because" they appear to do what you need on a particular CPU. I've stated my objection to this kind of behaviour many times in the past, I've even _purposely_ broken stuff (mtd stuff) when that's happened in the past, and I've not changed my views on this one bit. It's really surprising to me that someone who has been around for such a long time, and who knows my views on this, is pushing a patch set which goes against that, and somehow thinks that I won't have a problem with it... really?
What I suggest for the time being is to provide new inline function(s) in arch/arm/include/cacheflush.h which are purposed for your application, document them in that file, and call the implementation you're currently using. That means if we do have to change it in the future (for example, we don't need to do anything in the dcache flushing stuff) we don't have to hunt through all the code to find _your_ different use of that function and fix it - we can just fix it in one place and we have the reference there for what your code expects.
Again, this is nothing new - I've made my position on this kind of stuff _exceedingly_ plain in the past.
That all said, my biggest concern so far from this set is the independent issue of moving the cache handling into generic code, so that we avoid the existing problem where every platform ends up implementing that stuff time and time again - and the impact that will have on this MCPM code.
Currently as I see it, the two things are mutually incompatible with each other - and having discussed with Will, we'd come to the conclusion that I'd merge what I had because the comments alone on how the cpu hotplug stuff is supposed to work are a valuable improvement, even though the code changes don't completely solve all the issues.
However, MCPMm gets in the way of that. So... that presents a dilema... it's a case of this or that but not both. Or do you think MCPM can survive with additional LOUIS flushing before the cpu die callback is called?
v4 cover multi-cluster power management
Quoted 8 lines · Russell King - ARM Linux
What I suggest for the time being is to provide new inline function(s) in
arch/arm/include/cacheflush.hwhich are purposed for your application, document them in that file, and call the implementation you're currently using. That means if we do have to change it in the future (for example, we don't need to do anything in the dcache flushing stuff) we don't have to hunt through all the code to find _your_ different use of that function and fix it - we can just fix it in one place and we have the reference there for what your code expects.
What about the patch below? Once you tell me it is fine to you I'll adapt the MCPM series to it.
----- >8
From: Nicolas Pitre <[email protected]>
Date: Tue, 23 Apr 2013 16:45:40 -0400
Subject: [PATCH] ARM: cacheflush: add synchronization helpers for mixed cache state accesses
Algorithms used by the MCPM layer rely on state variables which are accessed while the cache is either active or inactive, depending on the code path and the active state.
This patch introduces generic cache maintenance helpers to provide the necessary cache synchronization for such state variables to always hit main memory in a ordered way.
arch/arm/include/asm/cacheflush.h +75 -0
@@ -363,4 +363,79 @@ static inline void flush_cache_vunmap(unsigned long start, unsigned long end) flush_cache_all(); } +/* +* Memory synchronization helpers for mixed cached vs non cached accesses. +* +* Some synchronization algorithms have to set states in memory with the +* cache enabled or disabled depending on the code path. It is crucial +* to always ensure proper cache maintenance to update main memory right +* away in that case. +* +* Any cached write must be followed by a cache clean operation. +* Any cached read must be preceded by a cache invalidate operation. +* Yet, in the read case, a cache flush i.e. atomic clean+invalidate +* operation is needed to avoid discarding possible concurrent writes to the +* accessed memory. +* +* Also, in order to prevent a cached writer from interfering with an +* adjacent non-cached writer, each state variable must be located to +* a separate cache line. +*/ + +/* +* This needs to be >= the max cache writeback size of all +* supported platforms included in the current kernel configuration. +* This is used to align state variables to their own cache lines. +*/ +#define __CACHE_WRITEBACK_ORDER 6/* guessed from existing platforms */ +#define __CACHE_WRITEBACK_GRANULE (1 << __CACHE_WRITEBACK_ORDER) + +/* +* There is no __cpuc_clean_dcache_area but we use it anyway for +* code intent clarity, and alias it to __cpuc_flush_dcache_area. +*/ +#define __cpuc_clean_dcache_area __cpuc_flush_dcache_area + +/* +* Ensure preceding writes to *p by this CPU are visible to +* subsequent reads by other CPUs: +*/ +static inline void __sync_cache_range_w (volatile void *p,size_t size) +{ +char *_p = (char *)p; + +__cpuc_clean_dcache_area (_p, size); +outer_clean_range (__pa (_p),__pa (_p + size)); +} + +/* +* Ensure preceding writes to *p by other CPUs are visible to +* subsequent reads by this CPU. We must be careful not to +* discard data simultaneously written by another CPU, hence the +* usage of flush rather than invalidate operations. +*/ +static inline void __sync_cache_range_r (volatile void *p,size_t size) +{ +char *_p = (char *)p; + +#ifdef CONFIG_OUTER_CACHE +if (outer_cache.flush_range) { +/* +* Ensure dirty data migrated from other CPUs into our cache +* are cleaned out safely before the outer cache is cleaned: +*/ +__cpuc_clean_dcache_area (_p, size); + +/* Clean and invalidate stale data for *p from outer ... */ +outer_flush_range (__pa (_p),__pa (_p + size)); + } +#endif + +/* ... and inner cache: */ +__cpuc_flush_dcache_area (_p, size); +} + +#define sync_cache_w (ptr) __sync_cache_range_w(ptr, sizeof *(ptr)) +#define sync_cache_r (ptr) __sync_cache_range_r(ptr, sizeof *(ptr)) +#endif
v4 cover multi-cluster power management
Quoted 11 lines · Nicolas Pitre
Quoted 8 lines · Russell King - ARM Linux
What I suggest for the time being is to provide new inline function(s) in
arch/arm/include/cacheflush.hwhich are purposed for your application, document them in that file, and call the implementation you're currently using. That means if we do have to change it in the future (for example, we don't need to do anything in the dcache flushing stuff) we don't have to hunt through all the code to find _your_ different use of that function and fix it - we can just fix it in one place and we have the reference there for what your code expects.
Quoted 2 lines
What about the patch below? Once you tell me it is fine to you I'll adapt the MCPM series to it.
Yes, that looks like a saner solution. Thanks.
v4 cover multi-cluster power management
Quoted 12 lines · Russell King - ARM Linux
Quoted 11 lines · Nicolas Pitre
Quoted 8 lines · Russell King - ARM Linux
What I suggest for the time being is to provide new inline function(s) in
arch/arm/include/cacheflush.hwhich are purposed for your application, document them in that file, and call the implementation you're currently using. That means if we do have to change it in the future (for example, we don't need to do anything in the dcache flushing stuff) we don't have to hunt through all the code to find _your_ different use of that function and fix it - we can just fix it in one place and we have the reference there for what your code expects.
Quoted 4 lines
Quoted 2 lines
What about the patch below? Once you tell me it is fine to you I'll adapt the MCPM series to it.
Yes, that looks like a saner solution. Thanks.
May I add your ACK to the patch?
Nicolas
v4 cover multi-cluster power management
Quoted 14 lines · Nicolas Pitre
Quoted 12 lines · Russell King - ARM Linux
Quoted 11 lines · Nicolas Pitre
Quoted 8 lines · Russell King - ARM Linux
What I suggest for the time being is to provide new inline function(s) in
arch/arm/include/cacheflush.hwhich are purposed for your application, document them in that file, and call the implementation you're currently using. That means if we do have to change it in the future (for example, we don't need to do anything in the dcache flushing stuff) we don't have to hunt through all the code to find _your_ different use of that function and fix it - we can just fix it in one place and we have the reference there for what your code expects.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
What about the patch below? Once you tell me it is fine to you I'll adapt the MCPM series to it.
Yes, that looks like a saner solution. Thanks.
May I add your ACK to the patch?
Yes, sure.
v4 cover multi-cluster power management
Quoted 18 lines · Russell King - ARM Linux
Quoted 17 lines · Nicolas Pitre
Quoted 15 lines · Russell King - ARM Linux
Quoted 13 lines · Nicolas Pitre
Quoted 8 lines · Russell King - ARM Linux
What I suggest for the time being is to provide new inline function(s) in
arch/arm/include/cacheflush.hwhich are purposed for your application, document them in that file, and call the implementation you're currently using. That means if we do have to change it in the future (for example, we don't need to do anything in the dcache flushing stuff) we don't have to hunt through all the code to find _your_ different use of that function and fix it - we can just fix it in one place and we have the reference there for what your code expects.What about the patch below? Once you tell me it is fine to you I'll adapt the MCPM series to it.
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
Yes, that looks like a saner solution. Thanks.
May I add your ACK to the patch?
Yes, sure.
Acked-by Russell King <[email protected]>
Thanks. I've adapted the series to this, and mcpm_set_entry_vector() now looks like this:
+extern unsigned long mcpm_entry_vectors[MAX_NR_CLUSTERS][MAX_CPUS_PER_CLUSTER]; +
+void mcpm_set_entry_vector(unsigned cpu, unsigned cluster, void *ptr) +{
+ unsigned long val = ptr ? virt_to_phys(ptr) : 0;
+ mcpm_entry_vectors[cluster][cpu] = val;
+ sync_cache_w(&mcpm_entry_vectors[cluster][cpu]);
+}
Similarly for the CCI driver:
+ /*
+ * Multi-cluster systems may need this data when non-coherent, during
+ * cluster power-up/power-down. Make sure it reaches main memory:
+ */
+ sync_cache_w(info);
+ sync_cache_w(&info);
This is also much prettier.
Assorted small changes for comments that followed the v4 post have been integrated as well. I don't think they justify a repost of the whole thing though. They are:
- Replace custom cache sync helpers with the sync_cache_*() ones from cacheflush.h.
- Remove volatile annotations since explicit cache ops act as memory barriers now.
- Rename mcpm_entry.h to mcpm.h since this is not solely about entry but more general.
- Export the __CACHE_WRITEBACK_* constants via asm_offsets.h since they're needed by assembly code.
- Add some more comments and clarify some existing ones.
I managed to retest this on RTSM after fighting with the license system.
The latest MCPM patches are available here:
git://git.linaro.org/people/nico/linux mcpm
Diffstat:
Documentation/arm/cluster-pm-race-avoidance.txt | 498 ++++++++++++++++++ Documentation/arm/vlocks.txt | 211 ++++++++ arch/arm/Kconfig | 8 + arch/arm/common/Makefile | 1 + arch/arm/common/mcpm_entry.c | 263 +++++++++ arch/arm/common/mcpm_head.S | 219 ++++++++ arch/arm/common/mcpm_platsmp.c | 92 ++++ arch/arm/common/vlock.S | 108 ++++ arch/arm/common/vlock.h | 29 + arch/arm/include/asm/cacheflush.h | 75 +++ arch/arm/include/asm/mcpm.h | 209 ++++++++ arch/arm/kernel/asm-offsets.c | 4 + 12 files changed, 1717 insertions(+)
The corresponding DCSCB support for RTSM is here:
git://git.linaro.org/people/nico/linux mcpm+dcscb
although that part is probably more suitable for ARM-SOC.
Do you want those patches to be posted again, or should I send a new pull request?
Nicolas
v4 cover multi-cluster power management
Quoted 20 lines · Nicolas Pitre
The latest MCPM patches are available here:
git://git.linaro.org/people/nico/linuxmcpmDiffstat:
Documentation/arm/cluster-pm-race-avoidance.txt | 498 ++++++++++++++++++ Documentation/arm/vlocks.txt | 211 ++++++++ arch/arm/Kconfig | 8 + arch/arm/common/Makefile | 1 + arch/arm/common/mcpm_entry.c | 263 +++++++++ arch/arm/common/mcpm_head.S | 219 ++++++++ arch/arm/common/mcpm_platsmp.c | 92 ++++ arch/arm/common/vlock.S | 108 ++++ arch/arm/common/vlock.h | 29 + arch/arm/include/asm/cacheflush.h | 75 +++ arch/arm/include/asm/mcpm.h | 209 ++++++++ arch/arm/kernel/asm-offsets.c | 4 + 12 files changed, 1717 insertions(+)
Quoted 8 lines
The corresponding DCSCB support for RTSM is here:
git://git.linaro.org/people/nico/linuxmcpm+dcscbalthough that part is probably more suitable for ARM-SOC.
Do you want those patches to be posted again, or should I send a new pull request?
I've been doing some further digging in these patches this evening, via
https://git.linaro.org/gitweb?p=people/nico/linux.git;a=shortlog;h=refs/heads/mcpm
I think they're now in pretty good shape and are ready to be pulled.
v4 cover multi-cluster power management
Quoted 27 lines · Russell King - ARM Linux
Quoted 26 lines · Nicolas Pitre
The latest MCPM patches are available here:
git://git.linaro.org/people/nico/linuxmcpmDiffstat:
Documentation/arm/cluster-pm-race-avoidance.txt | 498 ++++++++++++++++++ Documentation/arm/vlocks.txt | 211 ++++++++ arch/arm/Kconfig | 8 + arch/arm/common/Makefile | 1 + arch/arm/common/mcpm_entry.c | 263 +++++++++ arch/arm/common/mcpm_head.S | 219 ++++++++ arch/arm/common/mcpm_platsmp.c | 92 ++++ arch/arm/common/vlock.S | 108 ++++ arch/arm/common/vlock.h | 29 + arch/arm/include/asm/cacheflush.h | 75 +++ arch/arm/include/asm/mcpm.h | 209 ++++++++ arch/arm/kernel/asm-offsets.c | 4 + 12 files changed, 1717 insertions(+)The corresponding DCSCB support for RTSM is here:
git://git.linaro.org/people/nico/linuxmcpm+dcscbalthough that part is probably more suitable for ARM-SOC.
Quoted 7 lines
Quoted 2 lines
Do you want those patches to be posted again, or should I send a new pull request?
I've been doing some further digging in these patches this evening, via
https://git.linaro.org/gitweb?p=people/nico/linux.git;a=shortlog;h=refs/heads/mcpmI think they're now in pretty good shape and are ready to be pulled.
Great! Please feel free to pull the mcpm branch. I've nothing to add to it.
Nicolas
v4 cover multi-cluster power management
In reply to Nicolas PitreQuoted 11 lines · Nicolas Pitre
Quoted 8 lines · Russell King - ARM Linux
What I suggest for the time being is to provide new inline function(s) in
arch/arm/include/cacheflush.hwhich are purposed for your application, document them in that file, and call the implementation you're currently using. That means if we do have to change it in the future (for example, we don't need to do anything in the dcache flushing stuff) we don't have to hunt through all the code to find _your_ different use of that function and fix it - we can just fix it in one place and we have the reference there for what your code expects.
Quoted 2 lines
What about the patch below? Once you tell me it is fine to you I'll adapt the MCPM series to it.
This looks appropriate for mcpm's needs, and generic enough for other code to reuse. Thanks for that.
Quoted 100 lines
----- >8
From: Nicolas Pitre <[email protected]>
Date: Tue, 23 Apr 2013 16:45:40 -0400
Subject: [PATCH] ARM: cacheflush: add synchronization helpers for mixed cache state accessesAlgorithms used by the MCPM layer rely on state variables which are accessed while the cache is either active or inactive, depending on the code path and the active state.
This patch introduces generic cache maintenance helpers to provide the necessary cache synchronization for such state variables to always hit main memory in a ordered way.
Signed-off-by Nicolas Pitre <[email protected]>arch/arm/include/asm/cacheflush.h +75
-0@@ -363,4 +363,79 @@ static inline void flush_cache_vunmap(unsigned long start, unsigned long end) flush_cache_all(); } +/* +* Memory synchronization helpers for mixed cached vs non cached accesses. +* +* Some synchronization algorithms have to set states in memory with the +* cache enabled or disabled depending on the code path. It is crucial +* to always ensure proper cache maintenance to update main memory right +* away in that case. +* +* Any cached write must be followed by a cache clean operation. +* Any cached read must be preceded by a cache invalidate operation. +* Yet, in the read case, a cache flush i.e. atomic clean+invalidate +* operation is needed to avoid discarding possible concurrent writes to the +* accessed memory. +* +* Also, in order to prevent a cached writer from interfering with an +* adjacent non-cached writer, each state variable must be located to +* a separate cache line. +*/ + +/* +* This needs to be >= the max cache writeback size of all +* supported platforms included in the current kernel configuration. +* This is used to align state variables to their own cache lines. +*/ +#define __CACHE_WRITEBACK_ORDER 6/* guessed from existing platforms */ +#define __CACHE_WRITEBACK_GRANULE (1 << __CACHE_WRITEBACK_ORDER) + +/* +* There is no __cpuc_clean_dcache_area but we use it anyway for +* code intent clarity, and alias it to __cpuc_flush_dcache_area. +*/ +#define __cpuc_clean_dcache_area __cpuc_flush_dcache_area + +/* +* Ensure preceding writes to *p by this CPU are visible to +* subsequent reads by other CPUs: +*/ +static inline void __sync_cache_range_w (volatile void *p,size_t size) +{ +char *_p = (char *)p; + +__cpuc_clean_dcache_area (_p, size); +outer_clean_range (__pa (_p),__pa (_p + size)); +} + +/* +* Ensure preceding writes to *p by other CPUs are visible to +* subsequent reads by this CPU. We must be careful not to +* discard data simultaneously written by another CPU, hence the +* usage of flush rather than invalidate operations. +*/ +static inline void __sync_cache_range_r (volatile void *p,size_t size) +{ +char *_p = (char *)p; + +#ifdef CONFIG_OUTER_CACHE +if (outer_cache.flush_range) { +/* +* Ensure dirty data migrated from other CPUs into our cache +* are cleaned out safely before the outer cache is cleaned: +*/ +__cpuc_clean_dcache_area (_p, size); + +/* Clean and invalidate stale data for *p from outer ... */ +outer_flush_range (__pa (_p),__pa (_p + size)); + } +#endif + +/* ... and inner cache: */ +__cpuc_flush_dcache_area (_p, size); +} + +#define sync_cache_w (ptr) __sync_cache_range_w(ptr, sizeof *(ptr)) +#define sync_cache_r (ptr) __sync_cache_range_r(ptr, sizeof *(ptr)) +#endif
v4 cover multi-cluster power management
In reply to Russell King - ARM LinuxQuoted 11 lines · Russell King - ARM Linux
That all said, my biggest concern so far from this set is the independent issue of moving the cache handling into generic code, so that we avoid the existing problem where every platform ends up implementing that stuff time and time again - and the impact that will have on this MCPM code.
Currently as I see it, the two things are mutually incompatible with each other - and having discussed with Will, we'd come to the conclusion that I'd merge what I had because the comments alone on how the cpu hotplug stuff is supposed to work are a valuable improvement, even though the code changes don't completely solve all the issues.
Quoted 4 lines
However, MCPMm gets in the way of that. So... that presents a dilema... it's a case of this or that but not both. Or do you think MCPM can survive with additional LOUIS flushing before the cpu die callback is called?
We can have both for now. MCPM most certainly can survive it.
The cache should be mostly clean when the machine backend flushes up to LOUIS in that case so that shouldn't be very costly to do it again. And the removing of a CPU via the hotplug path is hardly a performance critical operation.
Nicolas