Skip to contentSkip to conversation

SMP support for Armada 375 and 38x

PATCH Superseded Thomas Petazzoni 3 participants 14 messages Mar 28, 2014

Subsystems (5)

v1 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

cover SMP support for Armada 375 and 38x

Jason, Andrew, Gregory, Sebastian,

This patch series adds SMP support for the Armada 375 and 38x. Most of the patches are fairly straight-forward, but here is a short overview of what's going on:

* Patches 1, 2, 3 make some preparatory clean-ups. Actually, patches 2 and 3 are not directly related to adding Armada 375 and 38x SMP support, but they are related to the SMP support of Armada XP.

* Patches 4, 5 and 6 add a few bits needed for the SMP support.

* Patch 7 is adding the SMP support itself. Since both Armada 375 and 38x are Cortex-A9, their SMP logic is fairly similar, the only difference being how to define the boot address of the secondary CPU. Therefore, we've used a single smp_operations implementation.

* Patch 8 add a workaround for a SMP booting bug affecting Armada 375 Z1 stepping. It requires using the Crypto SRAM loaded with a few instructions of code as a replacement of the BootROM for booting the secondary processor. For now, the workaround is unconditionally applied on all Armada 375 platforms, but as soon as we get access to more recent platforms, we'll add a mvebu-soc-id based test to only enable the workaround if needed.

* Patch 9 and 10 add the necessary Device Tree bits to enable SMP on Armada 375 and 38x.

In terms of dependencies, we will have 4 interconnected patch series:
Armada 375/38x coherency support, the PMSU rework series, the cpuidle series, and this Armada 375/38x SMP support series. The relations are as follows

  ---------------        ---------------------
  | pmsu-rework |        | 375/38x coherency |
  ---------------        ---------------------
        ||   ||                 ||
	||   \\____________     ||
	\/                \/    \/
  ---------------        ---------------
  |   cpuidle   |        | 375/38x SMP |
  ---------------        ---------------

In other words, this 375/38x SMP support series needs *both* the 375/38x coherency patch series to be applied, and the pmsu-rework to be applied.

If needed, I can merge the 375/38x coherency and 375/38x SMP support together, but I believe shorter patch series are easier to review. Just let me know what works best for you.

Thanks!

Thomas

Gregory CLEMENT (4):
ARM: mvebu: add function to set the resume boot address for Armada 375
ARM: mvebu: add SMP support for Armada 375 and Armada 38x
ARM: mvebu: add workaround for SMP support for Armada 375 stepping Z1
ARM: mvebu: add SMP support in the Armada 375 device tree

Thomas Petazzoni (6):
ARM: mvebu: move Armada XP specific SMP initialization to platsmp.c
irqchip: irq-armada-370-xp: do the set_smp_cross_call() in the driver
irqchip: irq-armada-370-xp: use cpu notifier to initialize secondary
CPUs
ARM: mvebu: add Armada 375/38x support in cpu-reset
ARM: mvebu: add Armada 38x compatible string to pmsu
ARM: mvebu: add SMP support in the Armada 38x device tree

 .../devicetree/bindings/arm/armada-370-xp-pmsu.txt |  9 +-
 .../devicetree/bindings/arm/armada-cpu-reset.txt   |  2 +
 arch/arm/boot/dts/armada-375.dtsi                  |  5 ++
 arch/arm/boot/dts/armada-38x.dtsi                  | 10 +++
 arch/arm/mach-mvebu/Kconfig                        |  4 +
 arch/arm/mach-mvebu/Makefile                       |  2 +-
 arch/arm/mach-mvebu/armada-370-xp.h                |  2 -
 arch/arm/mach-mvebu/armada-375.h                   | 18 ++++
 arch/arm/mach-mvebu/armada-380.h                   | 18 ++++
 arch/arm/mach-mvebu/board-v7.c                     |  2 +
 arch/arm/mach-mvebu/common.h                       |  6 ++
 arch/arm/mach-mvebu/cpu-reset.c                    |  4 +
 arch/arm/mach-mvebu/headsmp-a9.S                   | 34 ++++++++
 arch/arm/mach-mvebu/platsmp-a9.c                   | 95 ++++++++++++++++++++++
 arch/arm/mach-mvebu/platsmp.c                      | 18 ++--
 arch/arm/mach-mvebu/pmsu.c                         | 27 +-----
 arch/arm/mach-mvebu/system-controller.c            | 15 +++-
 drivers/irqchip/irq-armada-370-xp.c                | 24 +++++-
 18 files changed, 252 insertions(+), 43 deletions(-)
 create mode 100644 arch/arm/mach-mvebu/armada-375.h
 create mode 100644 arch/arm/mach-mvebu/armada-380.h
 create mode 100644 arch/arm/mach-mvebu/headsmp-a9.S
 create mode 100644 arch/arm/mach-mvebu/platsmp-a9.c
Signature · 1 line
1.8.3.2
#1 lore raw

PATCH 1/10 ARM: mvebu: move Armada XP specific SMP initialization to platsmp.c

The pmsu.c driver contained an armada_xp_boot_cpu() function that sets the boot address of a secondary CPUs and deasserts the reset. However, the Armada 375 needs a slightly different logic, so it makes more sense to move this code into the Armada XP specific platsmp.c.

In order to achieve this, the mvebu_pmsu_set_cpu_boot_addr() function is exported. It will be needed for both the Armada XP and Armada 38x SMP implementations.

---
 arch/arm/mach-mvebu/common.h  |  1 +
 arch/arm/mach-mvebu/platsmp.c | 10 +++++++++-
 arch/arm/mach-mvebu/pmsu.c    | 26 +-------------------------
 3 files changed, 11 insertions(+), 26 deletions(-)
arch/arm/mach-mvebu/common.h +1 -0
index cfb129b..2b88eb0 100644
@@ -19,6 +19,7 @@
 
 void mvebu_restart(enum reboot_mode mode, const char *cmd);
 int mvebu_cpu_reset_deassert(int cpu);
+void mvebu_pmsu_set_cpu_boot_addr(int hw_cpu, void *boot_addr);
 
 void armada_xp_cpu_die(unsigned int cpu);
 
arch/arm/mach-mvebu/platsmp.c +9 -1
index a6da03f..e98075f 100644
@@ -77,9 +77,17 @@ static void armada_xp_secondary_init(unsigned int cpu)
 
 static int armada_xp_boot_secondary(unsigned int cpu, struct task_struct *idle)
 {
+	int ret, hw_cpu;
+
 	pr_info("Booting CPU %d\n", cpu);
 
-	armada_xp_boot_cpu(cpu, armada_xp_secondary_startup);
+	hw_cpu = cpu_logical_map(cpu);
+	mvebu_pmsu_set_cpu_boot_addr(hw_cpu, armada_xp_secondary_startup);
+	ret = mvebu_cpu_reset_deassert(hw_cpu);
+	if (ret) {
+		pr_warn("unable to boot CPU: %d\n", ret);
+		return ret;
+	}
 
 	return 0;
 }
arch/arm/mach-mvebu/pmsu.c +1 -25
index 8361281..a1508de 100644
@@ -41,36 +41,12 @@ static struct of_device_id of_pmsu_table[] = {
 	{ /* end of list */ },
 };
 
-static void mvebu_pmsu_set_cpu_boot_addr(int hw_cpu, void *boot_addr)
+void mvebu_pmsu_set_cpu_boot_addr(int hw_cpu, void *boot_addr)
 {
 	writel(virt_to_phys(boot_addr), pmsu_mp_base +
 		PMSU_BOOT_ADDR_REDIRECT_OFFSET(hw_cpu));
 }
 
-#ifdef CONFIG_SMP
-int armada_xp_boot_cpu(unsigned int cpu_id, void *boot_addr)
-{
-	int hw_cpu, ret;
-
-	if (!pmsu_mp_base) {
-		pr_warn("Can't boot CPU. PMSU is uninitialized\n");
-		return -ENODEV;
-	}
-
-	hw_cpu = cpu_logical_map(cpu_id);
-
-	mvebu_pmsu_set_cpu_boot_addr(hw_cpu, boot_addr);
-
-	ret = mvebu_cpu_reset_deassert(hw_cpu);
-	if (ret) {
-		pr_warn("unable to boot CPU: %d\n", ret);
-		return ret;
-	}
-
-	return 0;
-}
-#endif
-
 static int __init armada_370_xp_pmsu_init(void)
 {
 	struct device_node *np;
Signature · 1 line
1.8.3.2
#2 lore raw

PATCH 2/10 irqchip: irq-armada-370-xp: do the set_smp_cross_call() in the driver

In reply to Thomas Petazzoni

Instead of having the SoC code in arch/arm/mach-mvebu/platsmp.c do the set_smp_cross_call() to register the IPI-triggering function, it makes more sense to do exactly what the GIC driver is doing: let the irqchip driver do it. This way, it avoids having to expose the armada_mpic_send_doorbell() function between the irqchip driver and the SoC code.

---
 arch/arm/mach-mvebu/armada-370-xp.h | 1 -
 arch/arm/mach-mvebu/platsmp.c       | 2 --
 drivers/irqchip/irq-armada-370-xp.c | 6 +++++-
 3 files changed, 5 insertions(+), 4 deletions(-)
arch/arm/mach-mvebu/armada-370-xp.h +0 -1
index 991ab32..dd3b8a1 100644
@@ -21,7 +21,6 @@
 #define ARMADA_370_MAX_CPUS 1
 #define ARMADA_XP_MAX_CPUS  4
 
-void armada_mpic_send_doorbell(const struct cpumask *mask, unsigned int irq);
 void armada_xp_mpic_smp_cpu_init(void);
 void armada_xp_secondary_startup(void);
 extern struct smp_operations armada_xp_smp_ops;
arch/arm/mach-mvebu/platsmp.c +0 -2
index e98075f..81bf517 100644
@@ -98,8 +98,6 @@ static void __init armada_xp_smp_init_cpus(void)
 
 	if (ncores == 0 || ncores > ARMADA_XP_MAX_CPUS)
 		panic("Invalid number of CPUs in DT\n");
-
-	set_smp_cross_call(armada_mpic_send_doorbell);
 }
 
 static void __init armada_xp_smp_prepare_cpus(unsigned int max_cpus)
drivers/irqchip/irq-armada-370-xp.c +5 -1
index abb850b..0bf5aef 100644
@@ -315,7 +315,8 @@ static int armada_370_xp_mpic_irq_map(struct irq_domain *h,
 }
 
 #ifdef CONFIG_SMP
-void armada_mpic_send_doorbell(const struct cpumask *mask, unsigned int irq)
+static void armada_mpic_send_doorbell(const struct cpumask *mask,
+				      unsigned int irq)
 {
 	int cpu;
 	unsigned long map = 0;
@@ -534,6 +535,9 @@ static int __init armada_370_xp_mpic_of_init(struct device_node *node,
 	if (parent_irq <= 0) {
 		irq_set_default_host(armada_370_xp_mpic_domain);
 		set_handle_irq(armada_370_xp_handle_irq);
+#ifdef CONFIG_SMP
+		set_smp_cross_call(armada_mpic_send_doorbell);
+#endif
 	} else {
 		irq_set_chained_handler(parent_irq,
 					armada_370_xp_mpic_handle_cascade_irq);
Signature · 1 line
1.8.3.2
#3 lore raw

PATCH 3/10 irqchip: irq-armada-370-xp: use cpu notifier to initialize secondary CPUs

In reply to Thomas Petazzoni

Some irqchip initialization must be done on secondary CPUs. On mvebu platforms, this is currently achieved by having the arch/arm/mach-mvebu/platsmp.c code directly call into a function exported by the irqchip driver, which isn't really nice.

This commit changes this by using the same solution as the one used in the GIC driver: the irqchip driver registers a CPU notifier, which is used to do the secondary CPU IRQ initialization. This way, the irqchip driver is completely autonomous, and the function no longer needs to be exposed from the irqchip driver to the SoC code.

---
 arch/arm/mach-mvebu/armada-370-xp.h |  1 -
 arch/arm/mach-mvebu/platsmp.c       |  6 ------
 drivers/irqchip/irq-armada-370-xp.c | 18 +++++++++++++++++-
 3 files changed, 17 insertions(+), 8 deletions(-)
arch/arm/mach-mvebu/armada-370-xp.h +0 -1
index dd3b8a1..883b348 100644
@@ -21,7 +21,6 @@
 #define ARMADA_370_MAX_CPUS 1
 #define ARMADA_XP_MAX_CPUS  4
 
-void armada_xp_mpic_smp_cpu_init(void);
 void armada_xp_secondary_startup(void);
 extern struct smp_operations armada_xp_smp_ops;
 #endif
arch/arm/mach-mvebu/platsmp.c +0 -6
index 81bf517..10d3890 100644
@@ -70,11 +70,6 @@ static void __init set_secondary_cpus_clock(void)
 	}
 }
 
-static void armada_xp_secondary_init(unsigned int cpu)
-{
-	armada_xp_mpic_smp_cpu_init();
-}
-
 static int armada_xp_boot_secondary(unsigned int cpu, struct task_struct *idle)
 {
 	int ret, hw_cpu;
@@ -130,7 +125,6 @@ static void __init armada_xp_smp_prepare_cpus(unsigned int max_cpus)
 struct smp_operations armada_xp_smp_ops __initdata = {
 	.smp_init_cpus		= armada_xp_smp_init_cpus,
 	.smp_prepare_cpus	= armada_xp_smp_prepare_cpus,
-	.smp_secondary_init	= armada_xp_secondary_init,
 	.smp_boot_secondary	= armada_xp_boot_secondary,
 #ifdef CONFIG_HOTPLUG_CPU
 	.cpu_die		= armada_xp_cpu_die,
drivers/irqchip/irq-armada-370-xp.c +17 -1
index 0bf5aef..20d9655 100644
@@ -19,6 +19,7 @@
 #include <linux/irq.h>
 #include <linux/interrupt.h>
 #include <linux/irqchip/chained_irq.h>
+#include <linux/cpu.h>
 #include <linux/io.h>
 #include <linux/of_address.h>
 #include <linux/of_irq.h>
@@ -336,7 +337,7 @@ static void armada_mpic_send_doorbell(const struct cpumask *mask,
 		ARMADA_370_XP_SW_TRIG_INT_OFFS);
 }
 
-void armada_xp_mpic_smp_cpu_init(void)
+static void armada_xp_mpic_smp_cpu_init(void)
 {
 	/* Clear pending IPIs */
 	writel(0, per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_CAUSE_OFFS);
@@ -348,6 +349,20 @@ void armada_xp_mpic_smp_cpu_init(void)
 	/* Unmask IPI interrupt */
 	writel(0, per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
 }
+
+static int armada_xp_mpic_secondary_init(struct notifier_block *nfb,
+					 unsigned long action, void *hcpu)
+{
+	if (action == CPU_STARTING || action == CPU_STARTING_FROZEN)
+		armada_xp_mpic_smp_cpu_init();
+	return NOTIFY_OK;
+}
+
+static struct notifier_block armada_370_xp_mpic_cpu_notifier = {
+	.notifier_call = armada_xp_mpic_secondary_init,
+	.priority = 100,
+};
+
 #endif /* CONFIG_SMP */
 
 static struct irq_domain_ops armada_370_xp_mpic_irq_ops = {
@@ -537,6 +552,7 @@ static int __init armada_370_xp_mpic_of_init(struct device_node *node,
 		set_handle_irq(armada_370_xp_handle_irq);
 #ifdef CONFIG_SMP
 		set_smp_cross_call(armada_mpic_send_doorbell);
+		register_cpu_notifier(&armada_370_xp_mpic_cpu_notifier);
 #endif
 	} else {
 		irq_set_chained_handler(parent_irq,
Signature · 1 line
1.8.3.2
#4 lore raw

PATCH 4/10 ARM: mvebu: add Armada 375/38x support in cpu-reset

In reply to Thomas Petazzoni

This commit adds new compatible strings for the Armada 375 and Armada 38x SOCs to the cpu-reset code of mach-mvebu. It will allow the upcoming SMP support for these SOCs to start the secondary CPUs.

---
 .../devicetree/bindings/arm/armada-cpu-reset.txt       |  2 ++
 arch/arm/mach-mvebu/armada-375.h                       | 18 ++++++++++++++++++
 arch/arm/mach-mvebu/armada-380.h                       | 18 ++++++++++++++++++
 arch/arm/mach-mvebu/cpu-reset.c                        |  4 ++++
 4 files changed, 42 insertions(+)
 create mode 100644 arch/arm/mach-mvebu/armada-375.h
 create mode 100644 arch/arm/mach-mvebu/armada-380.h
Documentation/devicetree/bindings/arm/armada-cpu-reset.txt +2 -0
index 384cbf8..27e70df 100644
@@ -7,6 +7,8 @@ Required properties:
   values are:
     marvell,armada-370-cpu-reset
     marvell,armada-xp-cpu-reset
+    marvell,armada-375-cpu-reset
+    marvell,armada-380-cpu-reset
 
 - reg: should be register base and length as documented in the
   datasheet for the CPU reset registers
arch/arm/mach-mvebu/armada-375.h new file +18 -0
new file mode 100644
index 0000000..3da7575
@@ -0,0 +1,18 @@
+/*
+ * Generic definitions for Marvell Armada 375 SoCs
+ *
+ * Copyright (C) 2014 Marvell
+ *
+ * Gregory CLEMENT <[email protected]>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#ifndef __MACH_ARMADA_375_H
+#define __MACH_ARMADA_375_H
+
+#define ARMADA_375_MAX_CPUS 2
+
+#endif /* __MACH_ARMADA_375_H */
arch/arm/mach-mvebu/armada-380.h new file +18 -0
new file mode 100644
index 0000000..fdfc34f
@@ -0,0 +1,18 @@
+/*
+ * Generic definitions for Marvell Armada 38x SoCs
+ *
+ * Copyright (C) 2014 Marvell
+ *
+ * Gregory CLEMENT <[email protected]>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#ifndef __MACH_ARMADA_38X_H
+#define __MACH_ARMADA_38X_H
+
+#define ARMADA_380_MAX_CPUS 2
+
+#endif /* __MACH_ARMADA_38X_H */
arch/arm/mach-mvebu/cpu-reset.c +4 -0
index e3821b1..8b84c94 100644
@@ -16,10 +16,14 @@
 #include <linux/io.h>
 #include <linux/resource.h>
 #include "armada-370-xp.h"
+#include "armada-375.h"
+#include "armada-380.h"
 
 static struct of_device_id of_cpu_reset_table[] = {
 	{.compatible = "marvell,armada-370-cpu-reset", .data = (void*) ARMADA_370_MAX_CPUS },
 	{.compatible = "marvell,armada-xp-cpu-reset",  .data = (void*) ARMADA_XP_MAX_CPUS },
+	{.compatible = "marvell,armada-375-cpu-reset", .data = (void*) ARMADA_375_MAX_CPUS },
+	{.compatible = "marvell,armada-380-cpu-reset", .data = (void*) ARMADA_380_MAX_CPUS },
 	{ /* end of list */ },
 };
 
Signature · 1 line
1.8.3.2
#5 lore raw

PATCH 5/10 ARM: mvebu: add function to set the resume boot address for Armada 375

In reply to Thomas Petazzoni

From: Gregory CLEMENT <[email protected]>

In order to boot the secondary CPUs on Armada 375, we need to set the boot address of these CPUs, through a register part of the System Controller (this deviates from the Armada XP design, where the boot address was defined using a register part of the PMSU unit).

Therefore, this commit adds a new helper function in the System Controller driver to set the secondary CPU boot address.

Moreover, it moves the System Controller initialization as an early_initcall(), since arch_initcall() is too late for an SMP-related initialization.

---
 arch/arm/mach-mvebu/common.h            |  1 +
 arch/arm/mach-mvebu/system-controller.c | 15 ++++++++++++++-
 2 files changed, 15 insertions(+), 1 deletion(-)
arch/arm/mach-mvebu/common.h +1 -0
index 2b88eb0..b67fb7a 100644
@@ -20,6 +20,7 @@
 void mvebu_restart(enum reboot_mode mode, const char *cmd);
 int mvebu_cpu_reset_deassert(int cpu);
 void mvebu_pmsu_set_cpu_boot_addr(int hw_cpu, void *boot_addr);
+void mvebu_system_controller_set_cpu_boot_addr(void *boot_addr);
 
 void armada_xp_cpu_die(unsigned int cpu);
 
arch/arm/mach-mvebu/system-controller.c +14 -1
index 1806187..3d4489e 100644
@@ -37,6 +37,8 @@ struct mvebu_system_controller {
 
 	u32 rstoutn_mask_reset_out_en;
 	u32 system_soft_reset;
+
+	u32 resume_boot_addr;
 };
 static struct mvebu_system_controller *mvebu_sc;
 
@@ -52,6 +54,7 @@ static const struct mvebu_system_controller armada_375_system_controller = {
 	.system_soft_reset_offset = 0x58,
 	.rstoutn_mask_reset_out_en = 0x1,
 	.system_soft_reset = 0x1,
+	.resume_boot_addr = 0xd4,
 };
 
 static const struct mvebu_system_controller orion_system_controller = {
@@ -98,6 +101,16 @@ void mvebu_restart(enum reboot_mode mode, const char *cmd)
 		;
 }
 
+#ifdef CONFIG_SMP
+void mvebu_system_controller_set_cpu_boot_addr(void *boot_addr)
+{
+	BUG_ON(system_controller_base == NULL);
+	BUG_ON(mvebu_sc->resume_boot_addr == 0);
+	writel(virt_to_phys(boot_addr), system_controller_base +
+	       mvebu_sc->resume_boot_addr);
+}
+#endif
+
 static int __init mvebu_system_controller_init(void)
 {
 	struct device_node *np;
@@ -115,4 +128,4 @@ static int __init mvebu_system_controller_init(void)
 	return 0;
 }
 
-arch_initcall(mvebu_system_controller_init);
+early_initcall(mvebu_system_controller_init);
Signature · 1 line
1.8.3.2
#6 lore raw

PATCH 6/10 ARM: mvebu: add Armada 38x compatible string to pmsu

In reply to Thomas Petazzoni

Since the Armada 38x PMSU registers are slightly different than the Armada 370/XP PMSU ones, we introduce a new compatible string "armada-380-pmsu" in the PMSU driver. These differences are not visible for the current usage of the PMSU, but they might become visible in the future.

---
 Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt | 9 ++++++---
 arch/arm/mach-mvebu/pmsu.c                                   | 1 +
 2 files changed, 7 insertions(+), 3 deletions(-)
Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt +6 -3
index 9761887..26799ef 100644
@@ -1,11 +1,14 @@
 Power Management Service Unit(PMSU)
 -----------------------------------
-Available on Marvell SOCs: Armada 370 and Armada XP
+Available on Marvell SOCs: Armada 370, Armada 38x and Armada XP
 
 Required properties:
 
-- compatible: should be "marvell,armada-370-pmsu", whereas
-  "marvell,armada-370-xp-pmsu" is deprecated and will be removed
+- compatible: should be one of:
+  - "marvell,armada-370-pmsu" for Armada 370 or Armada XP
+  - "marvell,armada-380-pmsu" for Armada 38x
+  - "marvell,armada-370-xp-pmsu" was used for Armada 370/XP but is now
+    deprecated and will be removed
 
 - reg: Should contain PMSU registers location and length.
 
arch/arm/mach-mvebu/pmsu.c +1 -0
index a1508de..a8796b8 100644
@@ -38,6 +38,7 @@ static void __iomem *pmsu_mp_base;
 static struct of_device_id of_pmsu_table[] = {
 	{ .compatible = "marvell,armada-370-pmsu", },
 	{ .compatible = "marvell,armada-370-xp-pmsu", },
+	{ .compatible = "marvell,armada-380-pmsu", },
 	{ /* end of list */ },
 };
 
Signature · 1 line
1.8.3.2
#7 lore raw

PATCH 7/10 ARM: mvebu: add SMP support for Armada 375 and Armada 38x

In reply to Thomas Petazzoni

From: Gregory CLEMENT <[email protected]>

This commit adds the SMP support for Armada 375 and Armada 38x. It turns out that the SMP logic for both of these SOCs are fairly similar, the only differences being:

* A different method to set the secondary CPU boot address

* An Armada 375 specific workaround needed for the early Z1 stepping, added by the following patch.

Other than that, the patch is fairly straightforward and adds the usual platsmp and headsmp code, defining the smp_operations structure that is referenced from the DT_MACHINE structures.

---
 arch/arm/mach-mvebu/Kconfig      |  4 +++
 arch/arm/mach-mvebu/Makefile     |  2 +-
 arch/arm/mach-mvebu/board-v7.c   |  2 ++
 arch/arm/mach-mvebu/common.h     |  4 +++
 arch/arm/mach-mvebu/headsmp-a9.S | 22 ++++++++++++++
 arch/arm/mach-mvebu/platsmp-a9.c | 63 ++++++++++++++++++++++++++++++++++++++++
 6 files changed, 96 insertions(+), 1 deletion(-)
 create mode 100644 arch/arm/mach-mvebu/headsmp-a9.S
 create mode 100644 arch/arm/mach-mvebu/platsmp-a9.c
arch/arm/mach-mvebu/Kconfig +4 -0
index 8634222..e88d951 100644
@@ -45,6 +45,8 @@ config MACH_ARMADA_375
 	select ARMADA_375_CLK
 	select CPU_V7
 	select HAVE_ARM_SCU
+	select HAVE_ARM_TWD
+	select HAVE_SMP
 	select MACH_MVEBU_V7
 	select NEON
 	select PINCTRL_ARMADA_375
@@ -60,6 +62,8 @@ config MACH_ARMADA_38X
 	select ARMADA_38X_CLK
 	select CPU_V7
 	select HAVE_ARM_SCU
+	select HAVE_ARM_TWD
+	select HAVE_SMP
 	select MACH_MVEBU_V7
 	select NEON
 	select PINCTRL_ARMADA_38X
arch/arm/mach-mvebu/Makefile +1 -1
index 0e0eab5..18d623f 100644
@@ -6,5 +6,5 @@ AFLAGS_coherency_ll.o		:= -Wa,-march=armv7-a
 obj-y				 += system-controller.o mvebu-soc-id.o cpu-reset.o
 obj-$(CONFIG_MACH_MVEBU_V7)      += board-v7.o
 obj-$(CONFIG_ARCH_MVEBU)	 += coherency.o coherency_ll.o pmsu.o
-obj-$(CONFIG_SMP)                += platsmp.o headsmp.o
+obj-$(CONFIG_SMP)                += platsmp.o headsmp.o platsmp-a9.o headsmp-a9.o
 obj-$(CONFIG_HOTPLUG_CPU)        += hotplug.o
arch/arm/mach-mvebu/board-v7.c +2 -0
index 6260cb8..9ded4b9 100644
@@ -146,6 +146,7 @@ static const char * const armada_375_dt_compat[] = {
 };
 
 DT_MACHINE_START(ARMADA_375_DT, "Marvell Armada 375 (Device Tree)")
+	.smp		= smp_ops(mvebu_cortex_a9_smp_ops),
 	.init_time	= mvebu_timer_and_clk_init,
 	.restart	= mvebu_restart,
 	.dt_compat	= armada_375_dt_compat,
@@ -158,6 +159,7 @@ static const char * const armada_38x_dt_compat[] = {
 };
 
 DT_MACHINE_START(ARMADA_38X_DT, "Marvell Armada 380/385 (Device Tree)")
+	.smp		= smp_ops(mvebu_cortex_a9_smp_ops),
 	.init_time	= mvebu_timer_and_clk_init,
 	.restart	= mvebu_restart,
 	.dt_compat	= armada_38x_dt_compat,
arch/arm/mach-mvebu/common.h +4 -0
index b67fb7a..c05de82 100644
@@ -24,4 +24,8 @@ void mvebu_system_controller_set_cpu_boot_addr(void *boot_addr);
 
 void armada_xp_cpu_die(unsigned int cpu);
 
+#ifdef CONFIG_SMP
+extern struct smp_operations mvebu_cortex_a9_smp_ops;
+#endif
+
 #endif
arch/arm/mach-mvebu/headsmp-a9.S new file +22 -0
new file mode 100644
index 0000000..78e66c9
@@ -0,0 +1,22 @@
+/*
+ * SMP support: Entry point for secondary CPUs of Marvell EBU
+ * Cortex-A9 based SOCs (Armada 375 and Armada 38x).
+ *
+ * Copyright (C) 2014 Marvell
+ *
+ * Gregory CLEMENT <[email protected]>
+ * Thomas Petazzoni <[email protected]>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#include <linux/linkage.h>
+#include <linux/init.h>
+
+	__CPUINIT
+ENTRY(mvebu_cortex_a9_secondary_startup)
+	bl      v7_invalidate_l1
+	b	secondary_startup
+ENDPROC(mvebu_cortex_a9_secondary_startup)
arch/arm/mach-mvebu/platsmp-a9.c new file +63 -0

63 lines not on this page · raw message

Signature · 1 line
1.8.3.2
#8 lore raw

PATCH 7/10 ARM: mvebu: add SMP support for Armada 375 and Armada 38x

arch/arm/mach-mvebu/Kconfig config MACH_ARMADA_375 · line 48

Quoted 9 lines · Thomas Petazzoni
arch/arm/mach-mvebu/Kconfig +1 -0
index 8634222..e88d951 100644
@@ -45,6 +45,8 @@ config MACH_ARMADA_375
 	select ARMADA_375_CLK
 	select CPU_V7
 	select HAVE_ARM_SCU
+	select HAVE_ARM_TWD

This should have "if SMP" appended to it (same for HAVE_ARM_SCU).

Quoted 14 lines
arch/arm/mach-mvebu/board-v7.c +1 -0
index 6260cb8..9ded4b9 100644
@@ -146,6 +146,7 @@ static const char * const armada_375_dt_compat[] = {
 };
 
 DT_MACHINE_START(ARMADA_375_DT, "Marvell Armada 375 (Device Tree)")
+	.smp		= smp_ops(mvebu_cortex_a9_smp_ops),
 	.init_time	= mvebu_timer_and_clk_init,
 	.restart	= mvebu_restart,
 	.dt_compat	= armada_375_dt_compat,

arch/arm/mach-mvebu/board-v7.c static const char * const armada_38x_dt_compat[] = { · line 165

Quoted 7 lines
arch/arm/mach-mvebu/board-v7.c +1 -0
@@ -158,6 +159,7 @@ static const char * const armada_38x_dt_compat[] = {
 };
 
 DT_MACHINE_START(ARMADA_38X_DT, "Marvell Armada 380/385 (Device Tree)")
+	.smp		= smp_ops(mvebu_cortex_a9_smp_ops),
 	.init_time	= mvebu_timer_and_clk_init,
 	.restart	= mvebu_restart,
 	.dt_compat	= armada_38x_dt_compat,

You may want to look into using the enable-method property in devicetree so that you don't have to set the smp ops here. The patch to support that is sitting in -next as 6c3ff8b11a16 (ARM: Introduce CPU_METHOD_OF_DECLARE() for cpu hotplug/smp, 2013-10-30).

Quoted 14 lines
arch/arm/mach-mvebu/headsmp-a9.S new file +8 -0
new file mode 100644
index 0000000..78e66c9
@@ -0,0 +1,22 @@
+/*
+ * SMP support: Entry point for secondary CPUs of Marvell EBU
+ * Cortex-A9 based SOCs (Armada 375 and Armada 38x).
+ *
+ * Copyright (C) 2014 Marvell
+ *
+ * Gregory CLEMENT <[email protected]>
+ * Thomas Petazzoni <[email protected]>

arch/arm/mach-mvebu/headsmp-a9.S line 18

Quoted 10 lines
arch/arm/mach-mvebu/headsmp-a9.S new file +10 -0
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#include <linux/linkage.h>
+#include <linux/init.h>
+
+	__CPUINIT

__CPUINIT is dead. Don't use it.

Quoted 5 lines
arch/arm/mach-mvebu/headsmp-a9.S new file +4 -0
+ENTRY(mvebu_cortex_a9_secondary_startup)
+	bl      v7_invalidate_l1
+	b	secondary_startup
+ENDPROC(mvebu_cortex_a9_secondary_startup)
Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation
#9 lore raw

PATCH 7/10 ARM: mvebu: add SMP support for Armada 375 and Armada 38x

Dear Stephen Boyd,

Quoted 12 lines · Stephen Boyd

arch/arm/mach-mvebu/Kconfig config MACH_ARMADA_375 · line 48

Quoted 9 lines · Thomas Petazzoni
arch/arm/mach-mvebu/Kconfig +1 -0
index 8634222..e88d951 100644
@@ -45,6 +45,8 @@ config MACH_ARMADA_375
 	select ARMADA_375_CLK
 	select CPU_V7
 	select HAVE_ARM_SCU
+	select HAVE_ARM_TWD

This should have "if SMP" appended to it (same for HAVE_ARM_SCU).

Thanks, will do. The HAVE_ARM_SCU addition is part of a separate series, the Armada 375/38x coherency series, so I'll fix it in there.

Quoted 15 lines
Quoted 15 lines
arch/arm/mach-mvebu/board-v7.c +1 -0
index 6260cb8..9ded4b9 100644
@@ -146,6 +146,7 @@ static const char * const armada_375_dt_compat[] = {
 };
 
 DT_MACHINE_START(ARMADA_375_DT, "Marvell Armada 375 (Device Tree)")
+	.smp		= smp_ops(mvebu_cortex_a9_smp_ops),
 	.init_time	= mvebu_timer_and_clk_init,
 	.restart	= mvebu_restart,
 	.dt_compat	= armada_375_dt_compat,
@@ -158,6 +159,7 @@ static const char * const armada_38x_dt_compat[] = {
 };
 
Quoted 10 lines

arch/arm/mach-mvebu/board-v7.c static const char * const armada_38x_dt_compat[] = { · line 165

Quoted 5 lines
arch/arm/mach-mvebu/board-v7.c +1 -0
 DT_MACHINE_START(ARMADA_38X_DT, "Marvell Armada 380/385 (Device Tree)")
+	.smp		= smp_ops(mvebu_cortex_a9_smp_ops),
 	.init_time	= mvebu_timer_and_clk_init,
 	.restart	= mvebu_restart,
 	.dt_compat	= armada_38x_dt_compat,

You may want to look into using the enable-method property in devicetree so that you don't have to set the smp ops here. The patch to support that is sitting in -next as 6c3ff8b11a16 (ARM: Introduce CPU_METHOD_OF_DECLARE() for cpu hotplug/smp, 2013-10-30).

Hm, ok, I'll have a look into that.

Quoted 18 lines
Quoted 18 lines
arch/arm/mach-mvebu/headsmp-a9.S new file +12 -0
new file mode 100644
index 0000000..78e66c9
@@ -0,0 +1,22 @@
+/*
+ * SMP support: Entry point for secondary CPUs of Marvell EBU
+ * Cortex-A9 based SOCs (Armada 375 and Armada 38x).
+ *
+ * Copyright (C) 2014 Marvell
+ *
+ * Gregory CLEMENT <[email protected]>
+ * Thomas Petazzoni <[email protected]>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
Quoted 8 lines

arch/arm/mach-mvebu/headsmp-a9.S line 18

Quoted 6 lines
arch/arm/mach-mvebu/headsmp-a9.S new file +6 -0
+ */
+
+#include <linux/linkage.h>
+#include <linux/init.h>
+
+	__CPUINIT

__CPUINIT is dead. Don't use it.

Sure, will fix, thanks.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#10 lore raw

PATCH 8/10 ARM: mvebu: add workaround for SMP support for Armada 375 stepping Z1

In reply to Thomas Petazzoni

From: Gregory CLEMENT <[email protected]>

Due to internal bootrom issue, CPU[1] initial jump code (four instructions) should be placed in SRAM memory of the SoC. In order to achieve this, we have to unmap the BootROM and at some specific location where the BootROM was place, create a specific MBus window for the SRAM. This SRAM is initialized with a few instructions of code that allows to jump into the real secondary CPU boot address.

This workaround will most likely be disabled when newer steppings of the Armada 375 will be made available, in which case a dynamic test based on mvebu-soc-id will be added.

---
 arch/arm/mach-mvebu/headsmp-a9.S | 12 ++++++++++++
 arch/arm/mach-mvebu/platsmp-a9.c | 32 ++++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+)
arch/arm/mach-mvebu/headsmp-a9.S +12 -0
index 78e66c9..5925366 100644
@@ -16,6 +16,18 @@
 #include <linux/init.h>
 
 	__CPUINIT
+#define CPU_RESUME_ADDR_REG 0xf10182d4
+
+.global armada_375_smp_cpu1_enable_code_start
+.global armada_375_smp_cpu1_enable_code_end
+
+armada_375_smp_cpu1_enable_code_start:
+	ldr     r0, [pc, #4]
+	ldr     r1, [r0]
+	mov     pc, r1
+	.word   CPU_RESUME_ADDR_REG
+armada_375_smp_cpu1_enable_code_end:
+
 ENTRY(mvebu_cortex_a9_secondary_startup)
 	bl      v7_invalidate_l1
 	b	secondary_startup
arch/arm/mach-mvebu/platsmp-a9.c +32 -0
index f29eb95..6c311de 100644
@@ -16,11 +16,36 @@
 #include <linux/io.h>
 #include <linux/of.h>
 #include <linux/smp.h>
+#include <linux/mbus.h>
 #include <asm/smp_scu.h>
 #include <asm/smp_plat.h>
 #include "common.h"
 #include "pmsu.h"
 
+#define CRYPT0_ENG_ID   41
+#define CRYPT0_ENG_ATTR 0x1
+#define SRAM_PHYS_BASE  0xFFFF0000
+
+#define BOOTROM_BASE    0xFFF00000
+#define BOOTROM_SIZE    0x100000
+
+extern unsigned char armada_375_smp_cpu1_enable_code_end;
+extern unsigned char armada_375_smp_cpu1_enable_code_start;
+
+void armada_375_smp_cpu1_enable_wa(void)
+{
+	void __iomem *sram_virt_base;
+
+	mvebu_mbus_del_window(BOOTROM_BASE, BOOTROM_SIZE);
+	mvebu_mbus_add_window_by_id(CRYPT0_ENG_ID, CRYPT0_ENG_ATTR,
+				SRAM_PHYS_BASE, SZ_64K);
+	sram_virt_base = ioremap(SRAM_PHYS_BASE, SZ_64K);
+
+	memcpy(sram_virt_base, &armada_375_smp_cpu1_enable_code_start,
+	       &armada_375_smp_cpu1_enable_code_end
+	       - &armada_375_smp_cpu1_enable_code_start);
+}
+
 extern void mvebu_cortex_a9_secondary_startup(void);
 
 static int __cpuinit mvebu_cortex_a9_boot_secondary(unsigned int cpu,
@@ -55,7 +80,14 @@ static int __cpuinit mvebu_cortex_a9_boot_secondary(unsigned int cpu,
 	return 0;
 }
 
+static void __init mvebu_cortex_a9_smp_prepare_cpus(unsigned int max_cpus)
+{
+	if (of_machine_is_compatible("marvell,armada375"))
+		armada_375_smp_cpu1_enable_wa();
+}
+
 struct smp_operations mvebu_cortex_a9_smp_ops __initdata = {
+	.smp_prepare_cpus	= mvebu_cortex_a9_smp_prepare_cpus,
 	.smp_boot_secondary	= mvebu_cortex_a9_boot_secondary,
 #ifdef CONFIG_HOTPLUG_CPU
 	.cpu_die		= armada_xp_cpu_die,
Signature · 1 line
1.8.3.2
#11 lore raw

PATCH 9/10 ARM: mvebu: add SMP support in the Armada 375 device tree

In reply to Thomas Petazzoni

From: Gregory CLEMENT <[email protected]>

Improve the Armada 375 Device Tree to add the CPU reset Device Tree node, which is necessary to get SMP working.

---
 arch/arm/boot/dts/armada-375.dtsi | 5 +++++
 1 file changed, 5 insertions(+)
arch/arm/boot/dts/armada-375.dtsi +5 -0
index eceab89..b9b901d 100644
@@ -325,6 +325,11 @@
 				clocks = <&coreclk 0>;
 			};
 
+			cpurst at 20800 {
+				compatible = "marvell,armada-375-cpu-reset";
+				reg = <0x20800 0x10>;
+			};
+
 			coherency-fabric at 21010 {
 				compatible = "marvell,armada-375-coherency-fabric";
 				reg = <0x21010 0x1c>;
Signature · 1 line
1.8.3.2
#12 lore raw

PATCH 10/10 ARM: mvebu: add SMP support in the Armada 38x device tree

In reply to Thomas Petazzoni

This commit improves the Armada 38x Device Tree to add the CPU reset and PMSU Device Tree nodes, which are necessary to get SMP working.

---
 arch/arm/boot/dts/armada-38x.dtsi | 10 ++++++++++
 1 file changed, 10 insertions(+)
arch/arm/boot/dts/armada-38x.dtsi +10 -0
index 2332b20..2b2145e 100644
@@ -272,11 +272,21 @@
 				clock-names = "nbclk", "fixed";
 			};
 
+			cpurst at 20800 {
+				compatible = "marvell,armada-380-cpu-reset";
+				reg = <0x20800 0x10>;
+			};
+
 			coherency-fabric at 21010 {
 				compatible = "marvell,armada-380-coherency-fabric";
 				reg = <0x21010 0x1c>;
 			};
 
+			pmsu at 22000 {
+				compatible = "marvell,armada-380-pmsu";
+				reg = <0x22000 0x1000>;
+			};
+
 			eth1: ethernet at 30000 {
 				compatible = "marvell,armada-370-neta";
 				reg = <0x30000 0x4000>;
Signature · 1 line
1.8.3.2
maintainer
#13 lore raw

cover SMP support for Armada 375 and 38x

In reply to Thomas Petazzoni
Quoted 25 lines · Thomas Petazzoni

Jason, Andrew, Gregory, Sebastian,

This patch series adds SMP support for the Armada 375 and 38x. Most of the patches are fairly straight-forward, but here is a short overview of what's going on:

* Patches 1, 2, 3 make some preparatory clean-ups. Actually, patches 2 and 3 are not directly related to adding Armada 375 and 38x SMP support, but they are related to the SMP support of Armada XP.

* Patches 4, 5 and 6 add a few bits needed for the SMP support.

* Patch 7 is adding the SMP support itself. Since both Armada 375 and 38x are Cortex-A9, their SMP logic is fairly similar, the only difference being how to define the boot address of the secondary CPU. Therefore, we've used a single smp_operations implementation.

* Patch 8 add a workaround for a SMP booting bug affecting Armada 375 Z1 stepping. It requires using the Crypto SRAM loaded with a few instructions of code as a replacement of the BootROM for booting the secondary processor. For now, the workaround is unconditionally applied on all Armada 375 platforms, but as soon as we get access to more recent platforms, we'll add a mvebu-soc-id based test to only enable the workaround if needed.

Quoted 2 lines

* Patch 9 and 10 add the necessary Device Tree bits to enable SMP on Armada 375 and 38x.

Not my area of expertise, but it all looks O.K. The only comment, which i suspect Jason will say sometime soon, is that it would be nice if the workaround was triggered on the stepping, i.e. the same comment he made before for another similar workaround.

Andrew