Skip to contentSkip to conversation

SMP support for CNS3xxx + some SMP SCU cleanups

PATCH New Anton Vorontsov 4 participants 7 messages Jul 27, 2010 – Jul 31, 2010

Subsystems (3)
Arm (32-bit) Arm sub-architectures Arm/versatile express platform
lore
Share & tools
lore

Anton Vorontsov

author
#0 lore raw

v3 cover SMP support for CNS3xxx + some SMP SCU cleanups

Hello Russell,

These patches were posted before, i.e.

1/3 ARM: Factor out secondary CPU bootup code from VExpress platforms
http://lists.infradead.org/pipermail/linux-arm-kernel/2010-June/018410.html
2/3 ARM: RealView: Switch to generic SCU routines
http://lists.infradead.org/pipermail/linux-arm-kernel/2010-June/018411.html
3/3 ARM: cns3xxx: Add support for SMP
http://lists.infradead.org/pipermail/linux-arm-kernel/2010-June/018412.html

In this updated series I split patch 1/3 into two, i.e. now it is:

1/4 ARM: SCU: Add generic routines for secondary CPU bootup and
3/4 ARM: VExpress: Switch to generic SCU routines

This was done so that new feature addition doesn't depend on a cleanup patch.

I tested 4/4 on a RealView PBX platform, so it should be pretty much OK. 3/4 was only compile-tested, but it should be OK as generic SCU code was mostly copied from VExpress platforms.

In the end, I consider patches 1/4 and 2/4 as 100% safe, so I'd like them to go into 2.6.36 via your patch tracking system. What do you think?

Thanks,

Signature · 3 lines
Anton Vorontsov
email: cbouatmailru at gmail.com
irc://irc.freenode.net/bd2
#1 lore raw

PATCH 1/4 ARM: SCU: Add generic routines for secondary CPU bootup

For CNS3xxx we want to reuse the original ARM approach of booting secondary CPUs. This patch turns VExpress' code into "SCU library", so that now platform code can call the generic routines.

Note that this patch doesn't convert VExpress platform to the generic routines. Plus, there are also mach-{omap2,ux500,realview} platforms that might benefit from this change, but we'll convert them via separate patches.

Signed-off-by Anton Vorontsov <[email protected]>
---
 arch/arm/include/asm/smp_scu.h |    9 +++
 arch/arm/kernel/Makefile       |    2 +-
 arch/arm/kernel/smp_scu.c      |  136 ++++++++++++++++++++++++++++++++++++++++
 arch/arm/kernel/smp_scu_head.S |   37 +++++++++++
 4 files changed, 183 insertions(+), 1 deletions(-)
 create mode 100644 arch/arm/kernel/smp_scu_head.S
arch/arm/include/asm/smp_scu.h +9 -0
index 2376835..271b252 100644
@@ -1,7 +1,16 @@
 #ifndef __ASMARM_ARCH_SCU_H
 #define __ASMARM_ARCH_SCU_H
 
+#include <linux/init.h>
+
 unsigned int scu_get_core_count(void __iomem *);
 void scu_enable(void __iomem *);
 
+extern volatile int __cpuinitdata scu_pen_release;
+void scu_secondary_startup(void);
+void __cpuinit scu_secondary_init(unsigned int cpu);
+int __cpuinit scu_boot_secondary(unsigned int cpu, struct task_struct *idle);
+void __init scu_init_cpus(void __iomem *scu_base);
+void __init scu_prepare_cpus(void __iomem *scu_base, unsigned int max_cpus);
+
 #endif
arch/arm/kernel/Makefile +1 -1
index 26d302c..2fc7cb1 100644
@@ -28,7 +28,7 @@ obj-$(CONFIG_ARTHUR)		+= arthur.o
 obj-$(CONFIG_ISA_DMA)		+= dma-isa.o
 obj-$(CONFIG_PCI)		+= bios32.o isa.o
 obj-$(CONFIG_SMP)		+= smp.o
-obj-$(CONFIG_HAVE_ARM_SCU)	+= smp_scu.o
+obj-$(CONFIG_HAVE_ARM_SCU)	+= smp_scu.o smp_scu_head.o
 obj-$(CONFIG_HAVE_ARM_TWD)	+= smp_twd.o
 obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
 obj-$(CONFIG_KEXEC)		+= machine_kexec.o relocate_kernel.o
arch/arm/kernel/smp_scu.c +136 -0

149 lines not on this page · raw message

arch/arm/kernel/smp_scu_head.S new file +37 -0
new file mode 100644
index 0000000..200010b
@@ -0,0 +1,37 @@
+/*
+ *  Copyright (c) 2003 ARM Limited
+ *  All Rights Reserved
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+#include <linux/linkage.h>
+#include <linux/init.h>
+
+	__INIT
+
+/*
+ * SCU specific entry point for secondary CPUs.  This provides
+ * a "holding pen" into which all secondary cores are held until we're
+ * ready for them to initialise.
+ */
+ENTRY(scu_secondary_startup)
+	mrc	p15, 0, r0, c0, c0, 5
+	and	r0, r0, #15
+	adr	r4, 1f
+	ldmia	r4, {r5, r6}
+	sub	r4, r4, r5
+	add	r6, r6, r4
+pen:	ldr	r7, [r6]
+	cmp	r7, r0
+	bne	pen
+
+	/*
+	 * we've been released from the holding pen: secondary_stack
+	 * should now contain the SVC stack for this core
+	 */
+	b	secondary_startup
+
+1:	.long	.
+	.long	scu_pen_release
Signature · 1 line
1.7.0.5
#2 lore raw

PATCH 1/4 ARM: SCU: Add generic routines for secondary CPU bootup

Quoted 3 lines · Anton Vorontsov

For CNS3xxx we want to reuse the original ARM approach of booting secondary CPUs. This patch turns VExpress' code into "SCU library", so that now platform code can call the generic routines.

I'm not sure this should become a "SCU library" - the name seems wrong. When you consider that the SCU is the 'snoop control unit', which has no relation to how secondary CPUs are booted, it makes little sense.

The secondary CPU boot protocol has much more to do with the boot loader than with the hardware.

I also suspect that we can get rid of the 'holding pen' stuff and just rely on the IPI being sent to the intended target CPU. I don't think anyone has really updated this since the initial ARM SMP implementation.

#3 lore raw

PATCH 1/4 ARM: SCU: Add generic routines for secondary CPU bootup

Quoted 6 lines · Russell King - ARM Linux
Quoted 3 lines · Anton Vorontsov

For CNS3xxx we want to reuse the original ARM approach of booting secondary CPUs. This patch turns VExpress' code into "SCU library", so that now platform code can call the generic routines.

I'm not sure this should become a "SCU library" - the name seems wrong.

Well, how would you name it? Would you also move these routines into a separate file (is there any preference for the file name)?

Quoted 5 lines

When you consider that the SCU is the 'snoop control unit', which has no relation to how secondary CPUs are booted, it makes little sense.

The secondary CPU boot protocol has much more to do with the boot loader than with the hardware.

Yup.

Quoted 3 lines

I also suspect that we can get rid of the 'holding pen' stuff and just rely on the IPI being sent to the intended target CPU. I don't think anyone has really updated this since the initial ARM SMP implementation.

Sure, but I belive that this patch set is a step into the right direction, basically it just gets rid of code duplication, and then we can improve the bootup code for all interested machines.

With these patches, machines don't* bother with the holding pen, it just becomes an implementation detail of the SCU/secondary CPU bootup code.

So it makes sense to consolidate the code anyway (which exactly what these patches do), right?

* Except realview, it also uses scu_pen_release for hotplug handling.

Signature · 3 lines
Anton Vorontsov
email: cbouatmailru at gmail.com
irc://irc.freenode.net/bd2
#4 lore raw

PATCH 2/4 ARM: cns3xxx: Add support for SMP

In reply to Anton Vorontsov

Thanks to the previous refactorings, nothing fancy needs to be done, just use generic SCU routines.

Signed-off-by Anton Vorontsov <[email protected]>
---
 arch/arm/Kconfig                         |    2 +-
 arch/arm/mach-cns3xxx/Kconfig            |    1 +
 arch/arm/mach-cns3xxx/Makefile           |    1 +
 arch/arm/mach-cns3xxx/include/mach/smp.h |   31 +++++++++++++
 arch/arm/mach-cns3xxx/platsmp.c          |   69 ++++++++++++++++++++++++++++++
 5 files changed, 103 insertions(+), 1 deletions(-)
 create mode 100644 arch/arm/mach-cns3xxx/include/mach/smp.h
 create mode 100644 arch/arm/mach-cns3xxx/platsmp.c
arch/arm/Kconfig +1 -1
index 4b54156..7bc76de 100644
@@ -1101,7 +1101,7 @@ config SMP
 	bool "Symmetric Multi-Processing (EXPERIMENTAL)"
 	depends on EXPERIMENTAL && (REALVIEW_EB_ARM11MP || REALVIEW_EB_A9MP ||\
 		 MACH_REALVIEW_PB11MP || MACH_REALVIEW_PBX || ARCH_OMAP4 ||\
-		 ARCH_U8500 || ARCH_VEXPRESS_CA9X4)
+		 ARCH_U8500 || ARCH_VEXPRESS_CA9X4 || ARCH_CNS3XXX)
 	depends on GENERIC_CLOCKEVENTS
 	select USE_GENERIC_SMP_HELPERS
 	select HAVE_ARM_SCU if (ARCH_REALVIEW || ARCH_OMAP4 || ARCH_U8500 || ARCH_VEXPRESS_CA9X4)
arch/arm/mach-cns3xxx/Kconfig +1 -0
index 9ebfcc4..0ba82aa 100644
@@ -3,6 +3,7 @@ menu "CNS3XXX platform type"
 
 config MACH_CNS3420VB
 	bool "Support for CNS3420 Validation Board"
+	select HAVE_ARM_SCU if SMP
 	help
 	  Include support for the Cavium Networks CNS3420 MPCore Platform
 	  Baseboard.
arch/arm/mach-cns3xxx/Makefile +1 -0
index 11033f1..ba4ee7b 100644
@@ -1,3 +1,4 @@
 obj-$(CONFIG_ARCH_CNS3XXX)		+= core.o pm.o devices.o
+obj-$(CONFIG_SMP)			+= platsmp.o
 obj-$(CONFIG_PCI)			+= pcie.o
 obj-$(CONFIG_MACH_CNS3420VB)		+= cns3420vb.o
arch/arm/mach-cns3xxx/include/mach/smp.h new file +31 -0
new file mode 100644
index 0000000..afb2fef
@@ -0,0 +1,31 @@
+/*
+ * Copyright 2002 ARM Ltd.
+ * Copyright 2008 Cavium Networks
+ *
+ * This file is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, Version 2, as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef __MACH_SMP_H
+#define __MACH_SMP_H
+
+#include <asm/hardware/gic.h>
+
+#define hard_smp_processor_id()				\
+	({						\
+		unsigned int cpunum;			\
+		__asm__("mrc p15, 0, %0, c0, c0, 5"	\
+			: "=r" (cpunum));		\
+		cpunum &= 0x0F;				\
+	})
+
+/*
+ * We use IRQ1 as the IPI
+ */
+static inline void smp_cross_call(const struct cpumask *mask)
+{
+	gic_raise_softirq(mask, 1);
+}
+
+#endif /* __MACH_SMP_H */
arch/arm/mach-cns3xxx/platsmp.c new file +69 -0

69 lines not on this page · raw message

Signature · 1 line
1.7.0.5
#5 lore raw

PATCH 3/4 ARM: VExpress: Switch to generic SCU routines

In reply to Anton Vorontsov

This shrinks platform-specific code quite a bit.

Signed-off-by Anton Vorontsov <[email protected]>
---
 arch/arm/mach-vexpress/Makefile  |    2 +-
 arch/arm/mach-vexpress/headsmp.S |   39 ----------
 arch/arm/mach-vexpress/platsmp.c |  150 ++++----------------------------------
 3 files changed, 15 insertions(+), 176 deletions(-)
 delete mode 100644 arch/arm/mach-vexpress/headsmp.S
arch/arm/mach-vexpress/Makefile +1 -1
index 1b71b77..6533d11 100644
@@ -4,5 +4,5 @@
 
 obj-y					:= v2m.o
 obj-$(CONFIG_ARCH_VEXPRESS_CA9X4)	+= ct-ca9x4.o
-obj-$(CONFIG_SMP)			+= platsmp.o headsmp.o
+obj-$(CONFIG_SMP)			+= platsmp.o
 obj-$(CONFIG_LOCAL_TIMERS)		+= localtimer.o
arch/arm/mach-vexpress/headsmp.S deleted +0 -39
deleted file mode 100644
index 8a78ff6..0000000
@@ -1,39 +0,0 @@
-/*
- *  linux/arch/arm/mach-vexpress/headsmp.S
- *
- *  Copyright (c) 2003 ARM Limited
- *  All Rights Reserved
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-#include <linux/linkage.h>
-#include <linux/init.h>
-
-	__INIT
-
-/*
- * Versatile Express specific entry point for secondary CPUs.  This
- * provides a "holding pen" into which all secondary cores are held
- * until we're ready for them to initialise.
- */
-ENTRY(vexpress_secondary_startup)
-	mrc	p15, 0, r0, c0, c0, 5
-	and	r0, r0, #15
-	adr	r4, 1f
-	ldmia	r4, {r5, r6}
-	sub	r4, r4, r5
-	add	r6, r6, r4
-pen:	ldr	r7, [r6]
-	cmp	r7, r0
-	bne	pen
-
-	/*
-	 * we've been released from the holding pen: secondary_stack
-	 * should now contain the SVC stack for this core
-	 */
-	b	secondary_startup
-
-1:	.long	.
-	.long	pen_release
arch/arm/mach-vexpress/platsmp.c +14 -136

188 lines not on this page · raw message

Signature · 1 line
1.7.0.5
#6 lore raw

PATCH 4/4 ARM: RealView: Switch to generic SCU routines

In reply to Anton Vorontsov

This shrinks platform-specific code quite a bit.

Signed-off-by Anton Vorontsov <[email protected]>
---
 arch/arm/mach-realview/Makefile  |    2 +-
 arch/arm/mach-realview/headsmp.S |   39 ---------
 arch/arm/mach-realview/hotplug.c |    5 +-
 arch/arm/mach-realview/platsmp.c |  161 +++-----------------------------------
 4 files changed, 14 insertions(+), 193 deletions(-)
 delete mode 100644 arch/arm/mach-realview/headsmp.S
arch/arm/mach-realview/Makefile +1 -1
index a01b76b..d456163 100644
@@ -8,6 +8,6 @@ obj-$(CONFIG_MACH_REALVIEW_PB11MP)	+= realview_pb11mp.o
 obj-$(CONFIG_MACH_REALVIEW_PB1176)	+= realview_pb1176.o
 obj-$(CONFIG_MACH_REALVIEW_PBA8)	+= realview_pba8.o
 obj-$(CONFIG_MACH_REALVIEW_PBX)		+= realview_pbx.o
-obj-$(CONFIG_SMP)			+= platsmp.o headsmp.o
+obj-$(CONFIG_SMP)			+= platsmp.o
 obj-$(CONFIG_HOTPLUG_CPU)		+= hotplug.o
 obj-$(CONFIG_LOCAL_TIMERS)		+= localtimer.o
arch/arm/mach-realview/headsmp.S deleted +0 -39
deleted file mode 100644
index 4075473..0000000
@@ -1,39 +0,0 @@
-/*
- *  linux/arch/arm/mach-realview/headsmp.S
- *
- *  Copyright (c) 2003 ARM Limited
- *  All Rights Reserved
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-#include <linux/linkage.h>
-#include <linux/init.h>
-
-	__INIT
-
-/*
- * Realview specific entry point for secondary CPUs.  This provides
- * a "holding pen" into which all secondary cores are held until we're
- * ready for them to initialise.
- */
-ENTRY(realview_secondary_startup)
-	mrc	p15, 0, r0, c0, c0, 5
-	and	r0, r0, #15
-	adr	r4, 1f
-	ldmia	r4, {r5, r6}
-	sub	r4, r4, r5
-	add	r6, r6, r4
-pen:	ldr	r7, [r6]
-	cmp	r7, r0
-	bne	pen
-
-	/*
-	 * we've been released from the holding pen: secondary_stack
-	 * should now contain the SVC stack for this core
-	 */
-	b	secondary_startup
-
-1:	.long	.
-	.long	pen_release
arch/arm/mach-realview/hotplug.c +2 -3
index f95521a..cf2c1ea 100644
@@ -13,10 +13,9 @@
 #include <linux/smp.h>
 #include <linux/completion.h>
 
+#include <asm/smp_scu.h>
 #include <asm/cacheflush.h>
 
-extern volatile int pen_release;
-
 static DECLARE_COMPLETION(cpu_killed);
 
 static inline void cpu_enter_lowpower(void)

@@ -72,7 +71,7 @@ static inline void platform_do_lowpower(unsigned int cpu)
 		    :
 		    : "memory", "cc");
 
-		if (pen_release == cpu) {
+		if (scu_pen_release == cpu) {
 			/*
 			 * OK, proper wakeup, we're done
 			 */
arch/arm/mach-realview/platsmp.c +11 -150

209 lines not on this page · raw message

Signature · 1 line
1.7.0.5