Skip to contentSkip to conversation

Introduce logical CPU mapping

RFC PATCH New Will Deacon 5 participants 20 messages Aug 9, 2011 – Aug 11, 2011

Subsystems (6)
lore
Share & tools
lore
#0 lore raw

cover Introduce logical CPU mapping

Hello,

This patch series is an RFC following a discussion I had with Russell the other day:

http://lists.infradead.org/pipermail/linux-arm-kernel/2011-August/060370.html

The approach I've taken is fairly simple but is enough for booting on a CPU with physical ID != 0 on platforms which can handle it.

Note that I haven't touched the OMAP4 or Tegra platforms since it's not clear from the code how they would be adapted to make use of the physical CPU number.

I've tested this with a Cortex-A9 on the vexpress platform but other platforms have been compile-tested only.

This is an RFC, so I'd value any feedback or comments.

Thanks,

Will

Will Deacon (8):
ARM: cpuid: add support for reading MPIDR
ARM: smp: populate logical CPU mapping during boot
ARM: gic: convert logical CPU numbers into physical numbers
ARM: exynos4: convert logical CPU numbers to physical numbers
ARM: msm: convert logical CPU numbers to physical numbers
ARM: shmobile: convert logical CPU numbers to physical numbers
ARM: ux500: convert logical CPU numbers to physical numbers
ARM: versatile: convert logical CPU numbers to physical numbers

 arch/arm/common/gic.c               |   12 +++++++++---
 arch/arm/include/asm/cputype.h      |    6 ++++++
 arch/arm/include/asm/smp.h          |    6 ++++++
 arch/arm/kernel/smp.c               |   15 +++++++++++++++
 arch/arm/mach-exynos4/hotplug.c     |    2 +-
 arch/arm/mach-exynos4/platsmp.c     |    2 +-
 arch/arm/mach-msm/hotplug.c         |    2 +-
 arch/arm/mach-msm/platsmp.c         |    2 +-
 arch/arm/mach-realview/hotplug.c    |    2 +-
 arch/arm/mach-shmobile/smp-sh73a0.c |    6 +++++-
 arch/arm/mach-ux500/hotplug.c       |    2 +-
 arch/arm/mach-ux500/platsmp.c       |    2 +-
 arch/arm/mach-vexpress/hotplug.c    |    2 +-
 arch/arm/plat-versatile/platsmp.c   |    2 +-
 14 files changed, 50 insertions(+), 13 deletions(-)
#1 lore raw

RFC PATCH 1/8 ARM: cpuid: add support for reading MPIDR

The MPIDR register forms part of the CPUID interface and allows software to determine the physical ID of the CPU on which it is currently executing.

This patch adds support for reading the MPIDR to cputype.h

---
 arch/arm/include/asm/cputype.h |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)
arch/arm/include/asm/cputype.h +6 -0
index cd4458f..cb47d28 100644
@@ -8,6 +8,7 @@
 #define CPUID_CACHETYPE	1
 #define CPUID_TCM	2
 #define CPUID_TLBTYPE	3
+#define CPUID_MPIDR	5
 
 #define CPUID_EXT_PFR0	"c1, 0"
 #define CPUID_EXT_PFR1	"c1, 1"
@@ -70,6 +71,11 @@ static inline unsigned int __attribute_const__ read_cpuid_tcmstatus(void)
 	return read_cpuid(CPUID_TCM);
 }
 
+static inline unsigned int __attribute_const__ read_cpuid_mpidr(void)
+{
+	return read_cpuid(CPUID_MPIDR);
+}
+
 /*
  * Intel's XScale3 core supports some v6 features (supersections, L2)
  * but advertises itself as v5 as it does not support the v6 ISA.  For
Signature · 1 line
1.7.0.4
#2 lore raw

RFC PATCH 2/8 ARM: smp: populate logical CPU mapping during boot

In reply to Will Deacon

To allow booting Linux on a CPU with physical ID != 0, we need to provide a mapping from the logical CPU number to the physical CPU number.

This patch adds such a mapping and populates it during boot.

---
 arch/arm/include/asm/smp.h |    6 ++++++
 arch/arm/kernel/smp.c      |   15 +++++++++++++++
 2 files changed, 21 insertions(+), 0 deletions(-)
arch/arm/include/asm/smp.h +6 -0
index e42d96a..674ebcd 100644
@@ -66,6 +66,12 @@ extern void platform_secondary_init(unsigned int cpu);
 extern void platform_smp_prepare_cpus(unsigned int);
 
 /*
+ * Logical CPU mapping.
+ */
+extern int __cpu_logical_map[NR_CPUS];
+#define cpu_logical_map(cpu)	__cpu_logical_map[cpu]
+
+/*
  * Initial data for bringing up a secondary CPU.
  */
 struct secondary_data {
arch/arm/kernel/smp.c +15 -0
index d88ff02..0016665 100644
@@ -39,6 +39,7 @@
 #include <asm/tlbflush.h>
 #include <asm/ptrace.h>
 #include <asm/localtimer.h>
+#include <asm/smp_plat.h>
 
 /*
  * as from 2.5, kernels no longer have an init_tasks structure
@@ -259,6 +260,20 @@ void __ref cpu_die(void)
 }
 #endif /* CONFIG_HOTPLUG_CPU */
 
+int __cpu_logical_map[NR_CPUS];
+
+void __init smp_setup_processor_id(void)
+{
+	int i;
+	u32 cpu = is_smp() ? read_cpuid_mpidr() & 0xff : 0;
+
+	cpu_logical_map(0) = cpu;
+	for (i = 1; i < NR_CPUS; ++i)
+		cpu_logical_map(i) = i == cpu ? 0 : i;
+
+	printk(KERN_INFO "Booting Linux on physical CPU %d\n", cpu);
+}
+
 /*
  * Called by both boot and secondaries to move global data into
  * per-processor storage.
Signature · 1 line
1.7.0.4
#3 lore raw

RFC PATCH 3/8 ARM: gic: convert logical CPU numbers into physical numbers

In reply to Will Deacon

The GIC driver must convert logical CPU numbers passed in from Linux into physical CPU numbers that are understood by the hardware.

This patch uses the new cpu_logical_map macro for performing the conversion inside the GIC driver.

---
 arch/arm/common/gic.c |   12 +++++++++---
 1 files changed, 9 insertions(+), 3 deletions(-)
arch/arm/common/gic.c +9 -3
index 3227ca9..6822c10 100644
@@ -180,7 +180,7 @@ static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
 		return -EINVAL;
 
 	mask = 0xff << shift;
-	bit = 1 << (cpu + shift);
+	bit = 1 << (cpu_logical_map(cpu) + shift);
 
 	spin_lock(&irq_controller_lock);
 	val = readl_relaxed(reg) & ~mask;
@@ -260,7 +260,8 @@ static void __init gic_dist_init(struct gic_chip_data *gic,
 {
 	unsigned int gic_irqs, irq_limit, i;
 	void __iomem *base = gic->dist_base;
-	u32 cpumask = 1 << smp_processor_id();
+	u32 cpu = cpu_logical_map(smp_processor_id());
+	u32 cpumask = 1 << cpu;
 
 	cpumask |= cpumask << 8;
 	cpumask |= cpumask << 16;
@@ -382,7 +383,12 @@ void __cpuinit gic_enable_ppi(unsigned int irq)
 #ifdef CONFIG_SMP
 void gic_raise_softirq(const struct cpumask *mask, unsigned int irq)
 {
-	unsigned long map = *cpus_addr(*mask);
+	int cpu;
+	unsigned long map = 0;
+
+	/* Convert our logical CPU mask into a physical one. */
+	for_each_cpu(cpu, mask)
+		map |= 1 << cpu_logical_map(cpu);
 
 	/*
 	 * Ensure that stores to Normal memory are visible to the
Signature · 1 line
1.7.0.4
#4 lore raw

RFC PATCH 3/8 ARM: gic: convert logical CPU numbers into physical numbers

Quoted 25 lines · Will Deacon

The GIC driver must convert logical CPU numbers passed in from Linux into physical CPU numbers that are understood by the hardware.

This patch uses the new cpu_logical_map macro for performing the conversion inside the GIC driver.

---
 arch/arm/common/gic.c |   12 +++++++++---
 1 files changed, 9 insertions(+), 3 deletions(-)
arch/arm/common/gic.c +1 -1
index 3227ca9..6822c10 100644
@@ -180,7 +180,7 @@ static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
 		return -EINVAL;
 
 	mask = 0xff << shift;
-	bit = 1 << (cpu + shift);
+	bit = 1 << (cpu_logical_map(cpu) + shift);
 
 	spin_lock(&irq_controller_lock);
 	val = readl_relaxed(reg) & ~mask;

arch/arm/common/gic.c static void __init gic_dist_init(struct gic_chip_data *gic, · line 264

Quoted 6 lines
arch/arm/common/gic.c +2 -1
@@ -260,7 +260,8 @@ static void __init gic_dist_init(struct gic_chip_data *gic,
 {
 	unsigned int gic_irqs, irq_limit, i;
 	void __iomem *base = gic->dist_base;
-	u32 cpumask = 1 << smp_processor_id();
+	u32 cpu = cpu_logical_map(smp_processor_id());
+	u32 cpumask = 1 << cpu;

Just a heads up - I need to add an #ifdef CONFIG_SMP otherwise this will fail to build for a UP configuration.

Will

#5 lore raw

RFC PATCH 4/8 ARM: exynos4: convert logical CPU numbers to physical numbers

In reply to Will Deacon

This patch uses the new cpu_logical_map() macro for converting logical CPU numbers into physical numbers when dealing with the pen_release variable in the SMP boot and CPU hotplug paths.

---
 arch/arm/mach-exynos4/hotplug.c |    2 +-
 arch/arm/mach-exynos4/platsmp.c |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
arch/arm/mach-exynos4/hotplug.c +1 -1
index 7490789..da70e7e 100644
@@ -75,7 +75,7 @@ static inline void platform_do_lowpower(unsigned int cpu, int *spurious)
 		    :
 		    : "memory", "cc");
 
-		if (pen_release == cpu) {
+		if (pen_release == cpu_logical_map(cpu)) {
 			/*
 			 * OK, proper wakeup, we're done
 			 */
arch/arm/mach-exynos4/platsmp.c +1 -1
index 7c2282c..0733814 100644
@@ -126,7 +126,7 @@ int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)
 	 * Note that "pen_release" is the hardware CPU ID, whereas
 	 * "cpu" is Linux's internal ID.
 	 */
-	write_pen_release(cpu);
+	write_pen_release(cpu_logical_map(cpu));
 
 	if (!(__raw_readl(S5P_ARM_CORE1_STATUS) & S5P_CORE_LOCAL_PWR_EN)) {
 		__raw_writel(S5P_CORE_LOCAL_PWR_EN,
Signature · 1 line
1.7.0.4
#6 lore raw

RFC PATCH 4/8 ARM: exynos4: convert logical CPU numbers to physical numbers

Quoted 46 lines · Will Deacon

This patch uses the new cpu_logical_map() macro for converting logical CPU numbers into physical numbers when dealing with the pen_release variable in the SMP boot and CPU hotplug paths.

---
?arch/arm/mach-exynos4/hotplug.c | ? ?2 +-
?arch/arm/mach-exynos4/platsmp.c | ? ?2 +-
?2 files changed, 2 insertions(+), 2 deletions(-)

arch/arm/mach-exynos4/hotplug.c +0 -0
index 7490789..da70e7e 100644
@@ -75,7 +75,7 @@ static inline void platform_do_lowpower(unsigned int cpu, int *spurious)

? ? ? ? ? ? ? ? ? ?:
? ? ? ? ? ? ? ? ? ?: "memory", "cc");

- ? ? ? ? ? ? ? if (pen_release == cpu) {
+ ? ? ? ? ? ? ? if (pen_release == cpu_logical_map(cpu)) { ? ? ? ? ? ? ? ? ? ? ? ?/*
? ? ? ? ? ? ? ? ? ? ? ? * OK, proper wakeup, we're done ? ? ? ? ? ? ? ? ? ? ? ? */

arch/arm/mach-exynos4/platsmp.c +0 -0
index 7c2282c..0733814 100644
@@ -126,7 +126,7 @@ int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)

? ? ? ? * Note that "pen_release" is the hardware CPU ID, whereas ? ? ? ? * "cpu" is Linux's internal ID.
? ? ? ? */
- ? ? ? write_pen_release(cpu);
+ ? ? ? write_pen_release(cpu_logical_map(cpu));

? ? ? ?if (!(__raw_readl(S5P_ARM_CORE1_STATUS) & S5P_CORE_LOCAL_PWR_EN)) { ? ? ? ? ? ? ? ?__raw_writel(S5P_CORE_LOCAL_PWR_EN,
--
1.7.0.4

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#9 lore raw

RFC PATCH 4/8 ARM: exynos4: convert logical CPU numbers to physical numbers

Quoted 9 lines · Kyungmin Park
Quoted 6 lines · Will Deacon
Quoted 2 lines · Kyungmin Park

Cheers for that. Did you have a go at booting on a CPU other than physical CPU 0?

No, how to test it?

Well, it depends on how your bootloader is configured. You want to throw CPU1 at the kernel whilst putting CPU0 into the pen.

Of course, that might not be possible on your platform, but at least with these patches it's not the core code getting in the way.

Will

#10 lore raw

RFC PATCH 5/8 ARM: msm: convert logical CPU numbers to physical numbers

In reply to Will Deacon

This patch uses the new cpu_logical_map() macro for converting logical CPU numbers into physical numbers when dealing with the pen_release variable in the SMP boot and CPU hotplug paths.

---
 arch/arm/mach-msm/hotplug.c |    2 +-
 arch/arm/mach-msm/platsmp.c |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
arch/arm/mach-msm/hotplug.c +1 -1
index 5a31f70..41c252d 100644
@@ -37,7 +37,7 @@ static inline void platform_do_lowpower(unsigned int cpu)
 		    :
 		    : "memory", "cc");
 
-		if (pen_release == cpu) {
+		if (pen_release == cpu_logical_map(cpu)) {
 			/*
 			 * OK, proper wakeup, we're done
 			 */
arch/arm/mach-msm/platsmp.c +1 -1
index 1a1af9e..e3375ee 100644
@@ -117,7 +117,7 @@ int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)
 	 * Note that "pen_release" is the hardware CPU ID, whereas
 	 * "cpu" is Linux's internal ID.
 	 */
-	pen_release = cpu;
+	pen_release = cpu_logical_map(cpu);
 	__cpuc_flush_dcache_area((void *)&pen_release, sizeof(pen_release));
 	outer_clean_range(__pa(&pen_release), __pa(&pen_release + 1));
 
Signature · 1 line
1.7.0.4
#11 lore raw

RFC PATCH 5/8 ARM: msm: convert logical CPU numbers to physical numbers

Quoted 6 lines · Will Deacon

This patch uses the new cpu_logical_map() macro for converting logical CPU numbers into physical numbers when dealing with the pen_release variable in the SMP boot and CPU hotplug paths.

For MSM8660:

Signature · 2 lines
Sent by an employee of the Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum.
#12 lore raw

RFC PATCH 5/8 ARM: msm: convert logical CPU numbers to physical numbers

Quoted 5 lines · David Brown
Quoted 4 lines · Will Deacon

This patch uses the new cpu_logical_map() macro for converting logical CPU numbers into physical numbers when dealing with the pen_release variable in the SMP boot and CPU hotplug paths.

Quoted 7 lines
Quoted 2 lines

For MSM8660:

Thanks David!

Curious: did you try booting with a CPU other than physical CPU 0 or is that not possible on your platform?

Cheers,

Will

#13 lore raw

RFC PATCH 5/8 ARM: msm: convert logical CPU numbers to physical numbers

Quoted 11 lines · Will Deacon
Quoted 10 lines · David Brown
Quoted 6 lines · Will Deacon

This patch uses the new cpu_logical_map() macro for converting logical CPU numbers into physical numbers when dealing with the pen_release variable in the SMP boot and CPU hotplug paths.

For MSM8660:

Quoted 7 lines
Quoted 2 lines

Thanks David!

Curious: did you try booting with a CPU other than physical CPU 0 or is that not possible on your platform?

I haven't. I'll look into whether that is possible.

David

Signature · 2 lines
Sent by an employee of the Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum.
#14 lore raw

RFC PATCH 6/8 ARM: shmobile: convert logical CPU numbers to physical numbers

In reply to Will Deacon

This patch uses the new cpu_logical_map() macro for converting logical CPU numbers into physical numbers when dealing with the SCU in the SMP boot path for sh73a0.

---
 arch/arm/mach-shmobile/smp-sh73a0.c |    6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)
arch/arm/mach-shmobile/smp-sh73a0.c +5 -1
index 3ffdbc9..be1ade7 100644
@@ -74,6 +74,8 @@ void __cpuinit sh73a0_secondary_init(unsigned int cpu)
 
 int __cpuinit sh73a0_boot_secondary(unsigned int cpu)
 {
+	cpu = cpu_logical_map(cpu);
+
 	/* enable cache coherency */
 	modify_scu_cpu_psr(0, 3 << (cpu * 8));
 
@@ -87,6 +89,8 @@ int __cpuinit sh73a0_boot_secondary(unsigned int cpu)
 
 void __init sh73a0_smp_prepare_cpus(void)
 {
+	int cpu = cpu_logical_map(0);
+
 	scu_enable(scu_base_addr());
 
 	/* Map the reset vector (in headsmp.S) */
@@ -94,5 +98,5 @@ void __init sh73a0_smp_prepare_cpus(void)
 	__raw_writel(__pa(shmobile_secondary_vector), __io(SBAR));
 
 	/* enable cache coherency on CPU0 */
-	modify_scu_cpu_psr(0, 3 << (0 * 8));
+	modify_scu_cpu_psr(0, 3 << (cpu * 8));
 }
Signature · 1 line
1.7.0.4
#15 lore raw

RFC PATCH 7/8 ARM: ux500: convert logical CPU numbers to physical numbers

In reply to Will Deacon

This patch uses the new cpu_logical_map() macro for converting logical CPU numbers into physical numbers when dealing with the pen_release variable in the SMP boot and CPU hotplug paths.

---
 arch/arm/mach-ux500/hotplug.c |    2 +-
 arch/arm/mach-ux500/platsmp.c |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
arch/arm/mach-ux500/hotplug.c +1 -1
index dd8037e..572015e 100644
@@ -24,7 +24,7 @@ static inline void platform_do_lowpower(unsigned int cpu)
 	for (;;) {
 		__asm__ __volatile__("dsb\n\t" "wfi\n\t"
 				: : : "memory");
-		if (pen_release == cpu) {
+		if (pen_release == cpu_logical_map(cpu)) {
 			/*
 			 * OK, proper wakeup, we're done
 			 */
arch/arm/mach-ux500/platsmp.c +1 -1
index a33df5f..27e5281 100644
@@ -96,7 +96,7 @@ int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)
 	 * the holding pen - release it, then wait for it to flag
 	 * that it has been released by resetting pen_release.
 	 */
-	write_pen_release(cpu);
+	write_pen_release(cpu_logical_map(cpu));
 
 	gic_raise_softirq(cpumask_of(cpu), 1);
 
Signature · 1 line
1.7.0.4
#17 lore raw

RFC PATCH 8/8 ARM: versatile: convert logical CPU numbers to physical numbers

In reply to Will Deacon

This patch uses the new cpu_logical_map() macro for converting logical CPU numbers into physical numbers when dealing with the pen_release variable in the SMP boot and CPU hotplug paths.

---
 arch/arm/mach-realview/hotplug.c  |    2 +-
 arch/arm/mach-vexpress/hotplug.c  |    2 +-
 arch/arm/plat-versatile/platsmp.c |    2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
arch/arm/mach-realview/hotplug.c +1 -1
index a87523d..ac1aed2 100644
@@ -69,7 +69,7 @@ static inline void platform_do_lowpower(unsigned int cpu, int *spurious)
 		    :
 		    : "memory", "cc");
 
-		if (pen_release == cpu) {
+		if (pen_release == cpu_logical_map(cpu)) {
 			/*
 			 * OK, proper wakeup, we're done
 			 */
arch/arm/mach-vexpress/hotplug.c +1 -1
index ea4cbfb..2866ed8 100644
@@ -70,7 +70,7 @@ static inline void platform_do_lowpower(unsigned int cpu, int *spurious)
 		    :
 		    : "memory", "cc");
 
-		if (pen_release == cpu) {
+		if (pen_release == cpu_logical_map(cpu)) {
 			/*
 			 * OK, proper wakeup, we're done
 			 */
arch/arm/plat-versatile/platsmp.c +1 -1
index 51ecfea..92f18d3 100644
@@ -77,7 +77,7 @@ int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)
 	 * since we haven't sent them a soft interrupt, they shouldn't
 	 * be there.
 	 */
-	write_pen_release(cpu);
+	write_pen_release(cpu_logical_map(cpu));
 
 	/*
 	 * Send the secondary CPU a soft interrupt, thereby causing
Signature · 1 line
1.7.0.4
#18 lore raw

cover Introduce logical CPU mapping

In reply to Will Deacon

Will,

Quoted 5 lines · Will Deacon

Hello,

This patch series is an RFC following a discussion I had with Russell the other day:

Quoted 9 lines

http://lists.infradead.org/pipermail/linux-arm-kernel/2011-August/060370.html

The approach I've taken is fairly simple but is enough for booting on a CPU with physical ID != 0 on platforms which can handle it.

Note that I haven't touched the OMAP4 or Tegra platforms since it's not clear from the code how they would be adapted to make use of the physical CPU number.

On OMAP, we use hardware register hand-shake for boot between a boot CPU and non-boot CPU. On OMAP4, boot CPU has to be always CPU0 to ensure that security and firewall initialisation is taken care. Other CPU can't do those things.

Regards
Santosh

#19 lore raw

cover Introduce logical CPU mapping

Quoted 1 line · Santosh

Will,

Hi Santosh,

Quoted 11 lines · Will Deacon
Quoted 10 lines

Hello,

This patch series is an RFC following a discussion I had with Russell the other day:

http://lists.infradead.org/pipermail/linux-arm-kernel/2011-August/060370.html

The approach I've taken is fairly simple but is enough for booting on a CPU with physical ID != 0 on platforms which can handle it.

Quoted 8 lines
Quoted 4 lines

Note that I haven't touched the OMAP4 or Tegra platforms since it's not clear from the code how they would be adapted to make use of the physical CPU number.

On OMAP, we use hardware register hand-shake for boot between a boot CPU and non-boot CPU. On OMAP4, boot CPU has to be always CPU0 to ensure that security and firewall initialisation is taken care. Other CPU can't do those things.

Understood. We can leave OMAP like it is then and assume that bootloaders will always ensure physical CPU0 is used for booting.

Cheers,

Will