Skip to contentSkip to conversation

SMP support for Armada XP

PATCH Superseded Gregory CLEMENT 4 participants 26 messages Oct 29, 2012 – Nov 19, 2012

Subsystems (4)

v2 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v2 cover SMP support for Armada XP

Hello,

The purpose of this patch set is to add the SMP support for the Armada XP SoCs. Beside the SMP support itself brought by the last 3 patches, this patch set also adds the support for the coherency fabric unit and the power management service unit.

The coherency fabric is responsible for ensuring hardware coherency between all CPUs and between CPUs and I/O masters. This unit is also available for Armada 370 and will be used in an incoming patch set for hardware I/O cache coherency.

The power management service unit is responsible for powering down and waking up CPUs and other SOC units.

The original code is from Yehuda Yitschak, it was reworked by myself and reviewed by Yehuda.

This patch set is based on 3.7-rc3 and depends one the framework clock support (the last version was posted 2 weeks ago:
http://thread.gmane.org/gmane.linux.kernel/1375701). The git branch called ArmadaXP-SMP-for-3.8-V2 is also available at
https://github.com/MISL-EBU-System-SW/mainline-public.git.

Arnd, Olof, as this code is SoC specific will you be able to pull it through Jason tree? Or should I push the part added in arch/arm/mm/proc-v7.S to support PJ4B CPU through Russell tree?

Changelog:
V1 -> V2:
- Rebased on to v3.7-rc3
- Fixed typos found by Alexandre Belloni
- Added clk_prepare_enable() before getting rate clk in set_secondary_cpus_clock()
- Add explanation in the binding documentation about the per-CPU interrupt registers: the address of the virtual register must be used.
- Removed the armada_xp prefix in the coherency.c file to be more compliant with the name convention of the other files.
- Coherency_init is now called from armada_370_xp_dt_init() and is no more an early_init() call. As the device tree is not available from an early_init(), it was useless to call coherency_init() so early. The need to be able to call some function very early during the boot were already resolved by using the hard code address of the register.

Regards,

Yehuda Yitschak (5):
arm: mvebu: Added support for coherency fabric in mach-mvebu
arm: mvebu: Added initial support for power managmement service unit
arm: mvebu: Added IPI support via doorbells
arm: mm: Added support for PJ4B cpu and init routines
arm: mvebu: Added SMP support for Armada XP

 .../devicetree/bindings/arm/armada-370-xp-mpic.txt |   12 +-
 .../devicetree/bindings/arm/armada-370-xp-pmsu.txt |   20 ++++
 .../devicetree/bindings/arm/coherency-fabric.txt   |   16 +++
 arch/arm/boot/dts/armada-370-xp.dtsi               |    5 +
 arch/arm/boot/dts/armada-xp.dtsi                   |   12 +-
 arch/arm/configs/mvebu_defconfig                   |    3 +
 arch/arm/mach-mvebu/Kconfig                        |    3 +-
 arch/arm/mach-mvebu/Makefile                       |    4 +-
 arch/arm/mach-mvebu/armada-370-xp.c                |    3 +
 arch/arm/mach-mvebu/armada-370-xp.h                |   10 ++
 arch/arm/mach-mvebu/coherency.c                    |   89 ++++++++++++++
 arch/arm/mach-mvebu/coherency.h                    |   21 ++++
 arch/arm/mach-mvebu/common.h                       |    6 +
 arch/arm/mach-mvebu/headsmp.S                      |   66 +++++++++++
 arch/arm/mach-mvebu/hotplug.c                      |   30 +++++
 arch/arm/mach-mvebu/irq-armada-370-xp.c            |   92 ++++++++++++++-
 arch/arm/mach-mvebu/platsmp.c                      |  124 ++++++++++++++++++++
 arch/arm/mach-mvebu/pmsu.c                         |   78 ++++++++++++
 arch/arm/mach-mvebu/pmsu.h                         |   16 +++
 arch/arm/mm/Kconfig                                |    4 +
 arch/arm/mm/proc-v7.S                              |   46 ++++++++
 21 files changed, 648 insertions(+), 12 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt
 create mode 100644 Documentation/devicetree/bindings/arm/coherency-fabric.txt
 create mode 100644 arch/arm/mach-mvebu/coherency.c
 create mode 100644 arch/arm/mach-mvebu/coherency.h
 create mode 100644 arch/arm/mach-mvebu/headsmp.S
 create mode 100644 arch/arm/mach-mvebu/hotplug.c
 create mode 100644 arch/arm/mach-mvebu/platsmp.c
 create mode 100644 arch/arm/mach-mvebu/pmsu.c
 create mode 100644 arch/arm/mach-mvebu/pmsu.h
Signature · 1 line
1.7.9.5
#1 lore raw

PATCH v2 1/5 arm: mvebu: Added support for coherency fabric in mach-mvebu

From: Yehuda Yitschak <[email protected]>

---
 .../devicetree/bindings/arm/coherency-fabric.txt   |   16 ++++
 arch/arm/boot/dts/armada-370-xp.dtsi               |    5 ++
 arch/arm/mach-mvebu/Makefile                       |    2 +-
 arch/arm/mach-mvebu/coherency.c                    |   89 ++++++++++++++++++++
 arch/arm/mach-mvebu/coherency.h                    |   21 +++++
 arch/arm/mach-mvebu/common.h                       |    2 +
 6 files changed, 134 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/devicetree/bindings/arm/coherency-fabric.txt
 create mode 100644 arch/arm/mach-mvebu/coherency.c
 create mode 100644 arch/arm/mach-mvebu/coherency.h
Documentation/devicetree/bindings/arm/coherency-fabric.txt new file +16 -0
new file mode 100644
index 0000000..2bfbf67
@@ -0,0 +1,16 @@
+Coherency fabric
+----------------
+Available on Marvell SOCs: Armada 370 and Armada XP
+
+Required properties:
+
+- compatible: "marvell,coherency-fabric"
+- reg: Should contain,coherency fabric registers location and length.
+
+Example:
+
+coherency-fabric at d0020200 {
+	compatible = "marvell,coherency-fabric";
+	reg = <0xd0020200 0xb0>;
+};
+
arch/arm/boot/dts/armada-370-xp.dtsi +5 -0
index 94b4b9e..b0d075b 100644
@@ -36,6 +36,11 @@
 	      interrupt-controller;
 	};
 
+	coherency-fabric at d0020200 {
+		compatible = "marvell,coherency-fabric";
+		reg = <0xd0020200 0xb0>;
+	};
+
 	soc {
 		#address-cells = <1>;
 		#size-cells = <1>;
arch/arm/mach-mvebu/Makefile +1 -1
index 57f996b..abd6d3b 100644
@@ -2,4 +2,4 @@ ccflags-$(CONFIG_ARCH_MULTIPLATFORM) := -I$(srctree)/$(src)/include \
 	-I$(srctree)/arch/arm/plat-orion/include
 
 obj-y += system-controller.o
-obj-$(CONFIG_MACH_ARMADA_370_XP) += armada-370-xp.o irq-armada-370-xp.o addr-map.o
+obj-$(CONFIG_MACH_ARMADA_370_XP) += armada-370-xp.o irq-armada-370-xp.o addr-map.o coherency.o
arch/arm/mach-mvebu/coherency.c new file +89 -0

89 lines not on this page · raw message

arch/arm/mach-mvebu/coherency.h new file +21 -0
new file mode 100644
index 0000000..6182192
@@ -0,0 +1,21 @@
+/*
+ * arch/arm/mach-mvebu/include/mach/coherency.h
+ *
+ *
+ * Coherency fabric (Aurora) support for Armada 370 and XP platforms.
+ *
+ * Copyright (C) 2012 Marvell
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#ifndef __MACH_370_XP_COHERENCY_H
+#define __MACH_370_XP_COHERENCY_H
+
+int set_cpu_coherent(int cpu_id, int smp_group_id);
+int coherency_get_cpu_count(void);
+int coherency_init(void);
+
+#endif	/* __MACH_370_XP_COHERENCY_H */
arch/arm/mach-mvebu/common.h +2 -0
index 281fab3..ea08919 100644
@@ -21,4 +21,6 @@ void mvebu_clocks_init(void);
 void armada_370_xp_init_irq(void);
 void armada_370_xp_handle_irq(struct pt_regs *regs);
 
+
+int armada_370_xp_coherency_init(void);
 #endif
Signature · 1 line
1.7.9.5
#2 lore raw

PATCH v2 1/5 arm: mvebu: Added support for coherency fabric in mach-mvebu

Hi Gregory,

Quoted 24 lines · Gregory CLEMENT
arch/arm/mach-mvebu/coherency.c new file +18 -0
new file mode 100644
index 0000000..69e130d
@@ -0,0 +1,89 @@
+/*
+ * Coherency fabric (Aurora) support for Armada 370 and XP platforms.
+ *
+ * Copyright (C) 2012 Marvell
+ *
+ * Yehuda Yitschak <[email protected]>
+ * Gregory Clement <[email protected]>
+ * Thomas Petazzoni <[email protected]>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ *
+ * The Armada 370 and Armada XP SOCs have a coherency fabric which is
+ * responsible for ensuring hardware coherency between all CPUs and between
+ * CPUs and I/O masters. This file initializes the coherency fabric and
+ * supplies basic routines for configuring and controlling hardware coherency
+ */

[...]

Quoted 12 lines
arch/arm/mach-mvebu/coherency.c new file +12 -0
+int set_cpu_coherent(unsigned int hw_cpu_id, int smp_group_id)
+{
+	int reg;
+
+	if (!coherency_base) {
+		pr_warn("Can't make CPU %d cache coherent.\n", hw_cpu_id);
+		pr_warn("Coherency fabric is not initialized\n");
+		return 1;
+	}
+
+	/* Enable the CPU in coherency fabric */
+	reg = readl(coherency_base + COHERENCY_FABRIC_CTL_OFFSET);

arch/arm/mach-mvebu/coherency.c line 76

Quoted 10 lines
arch/arm/mach-mvebu/coherency.c new file +10 -0
+	reg |= 1 << (24 + hw_cpu_id);
+	writel(reg, coherency_base + COHERENCY_FABRIC_CTL_OFFSET);
+
+	/* Add CPU to SMP group */
+	reg = readl(coherency_base + COHERENCY_FABRIC_CFG_OFFSET);
+	reg |= 1 << (16 + hw_cpu_id + (smp_group_id == 0 ? 8 : 0));
+	writel(reg, coherency_base + COHERENCY_FABRIC_CFG_OFFSET);
+
+	return 0;
+}

These writels may expand to code containing calls to outer_sync(), which will attempt to take a spinlock for the aurora l2. Given that the CPU isn't coherent, how does this play out with the exclusive store instruction in the lock?

Will

#3 lore raw

PATCH v2 1/5 arm: mvebu: Added support for coherency fabric in mach-mvebu

Quoted 1 line · Will Deacon

Hi Gregory,

Hi Will,

Quoted 48 lines · Gregory CLEMENT
Quoted 24 lines
arch/arm/mach-mvebu/coherency.c new file +18 -0
new file mode 100644
index 0000000..69e130d
@@ -0,0 +1,89 @@
+/*
+ * Coherency fabric (Aurora) support for Armada 370 and XP platforms.
+ *
+ * Copyright (C) 2012 Marvell
+ *
+ * Yehuda Yitschak <[email protected]>
+ * Gregory Clement <[email protected]>
+ * Thomas Petazzoni <[email protected]>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ *
+ * The Armada 370 and Armada XP SOCs have a coherency fabric which is
+ * responsible for ensuring hardware coherency between all CPUs and between
+ * CPUs and I/O masters. This file initializes the coherency fabric and
+ * supplies basic routines for configuring and controlling hardware coherency
+ */

[...]

Quoted 19 lines
arch/arm/mach-mvebu/coherency.c new file +19 -0
+int set_cpu_coherent(unsigned int hw_cpu_id, int smp_group_id)
+{
+	int reg;
+
+	if (!coherency_base) {
+		pr_warn("Can't make CPU %d cache coherent.\n", hw_cpu_id);
+		pr_warn("Coherency fabric is not initialized\n");
+		return 1;
+	}
+
+	/* Enable the CPU in coherency fabric */
+	reg = readl(coherency_base + COHERENCY_FABRIC_CTL_OFFSET);
+	reg |= 1 << (24 + hw_cpu_id);
+	writel(reg, coherency_base + COHERENCY_FABRIC_CTL_OFFSET);
+
+	/* Add CPU to SMP group */
+	reg = readl(coherency_base + COHERENCY_FABRIC_CFG_OFFSET);
+	reg |= 1 << (16 + hw_cpu_id + (smp_group_id == 0 ? 8 : 0));
+	writel(reg, coherency_base + COHERENCY_FABRIC_CFG_OFFSET);
Quoted 8 lines

arch/arm/mach-mvebu/coherency.c line 76

Quoted 3 lines
arch/arm/mach-mvebu/coherency.c new file +3 -0
+
+	return 0;
+}

These writels may expand to code containing calls to outer_sync(), which will attempt to take a spinlock for the aurora l2. Given that the CPU isn't coherent, how does this play out with the exclusive store instruction in the lock?

I forward this question to the Marvell experts.

Quoted 8 lines

Will

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Signature · 4 lines
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
#4 lore raw

PATCH v2 1/5 arm: mvebu: Added support for coherency fabric in mach-mvebu

In reply to Will Deacon

Hi Will,

Quoted 49 lines · Will Deacon

Hi Gregory,

Quoted 24 lines · Gregory CLEMENT
arch/arm/mach-mvebu/coherency.c new file +18 -0
new file mode 100644
index 0000000..69e130d
@@ -0,0 +1,89 @@
+/*
+ * Coherency fabric (Aurora) support for Armada 370 and XP platforms.
+ *
+ * Copyright (C) 2012 Marvell
+ *
+ * Yehuda Yitschak <[email protected]>
+ * Gregory Clement <[email protected]>
+ * Thomas Petazzoni <[email protected]>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ *
+ * The Armada 370 and Armada XP SOCs have a coherency fabric which is
+ * responsible for ensuring hardware coherency between all CPUs and between
+ * CPUs and I/O masters. This file initializes the coherency fabric and
+ * supplies basic routines for configuring and controlling hardware coherency
+ */

[...]

Quoted 19 lines
arch/arm/mach-mvebu/coherency.c new file +19 -0
+int set_cpu_coherent(unsigned int hw_cpu_id, int smp_group_id)
+{
+	int reg;
+
+	if (!coherency_base) {
+		pr_warn("Can't make CPU %d cache coherent.\n", hw_cpu_id);
+		pr_warn("Coherency fabric is not initialized\n");
+		return 1;
+	}
+
+	/* Enable the CPU in coherency fabric */
+	reg = readl(coherency_base + COHERENCY_FABRIC_CTL_OFFSET);
+	reg |= 1 << (24 + hw_cpu_id);
+	writel(reg, coherency_base + COHERENCY_FABRIC_CTL_OFFSET);
+
+	/* Add CPU to SMP group */
+	reg = readl(coherency_base + COHERENCY_FABRIC_CFG_OFFSET);
+	reg |= 1 << (16 + hw_cpu_id + (smp_group_id == 0 ? 8 : 0));
+	writel(reg, coherency_base + COHERENCY_FABRIC_CFG_OFFSET);
Quoted 8 lines

arch/arm/mach-mvebu/coherency.c line 76

Quoted 3 lines
arch/arm/mach-mvebu/coherency.c new file +3 -0
+
+	return 0;
+}

These writels may expand to code containing calls to outer_sync(), which will attempt to take a spinlock for the aurora l2. Given that the CPU isn't coherent, how does this play out with the exclusive store instruction in the lock?

I dug a little this subject: and I am not sure there is problem. In SMP mode, only the system cache mode of Aurora is used. In this mode, outer_cache.sync is void then outer_sync() won't call any function, so there will be no access to any spinlock.

Gregory

#5 lore raw

PATCH v2 1/5 arm: mvebu: Added support for coherency fabric in mach-mvebu

Quoted 30 lines · Gregory CLEMENT
Quoted 20 lines · Will Deacon
Quoted 19 lines · Gregory CLEMENT
arch/arm/mach-mvebu/coherency.c new file +19 -0
+int set_cpu_coherent(unsigned int hw_cpu_id, int smp_group_id)
+{
+	int reg;
+
+	if (!coherency_base) {
+		pr_warn("Can't make CPU %d cache coherent.\n", hw_cpu_id);
+		pr_warn("Coherency fabric is not initialized\n");
+		return 1;
+	}
+
+	/* Enable the CPU in coherency fabric */
+	reg = readl(coherency_base + COHERENCY_FABRIC_CTL_OFFSET);
+	reg |= 1 << (24 + hw_cpu_id);
+	writel(reg, coherency_base + COHERENCY_FABRIC_CTL_OFFSET);
+
+	/* Add CPU to SMP group */
+	reg = readl(coherency_base + COHERENCY_FABRIC_CFG_OFFSET);
+	reg |= 1 << (16 + hw_cpu_id + (smp_group_id == 0 ? 8 : 0));
+	writel(reg, coherency_base + COHERENCY_FABRIC_CFG_OFFSET);
Quoted 8 lines

arch/arm/mach-mvebu/coherency.c line 76

Quoted 3 lines
arch/arm/mach-mvebu/coherency.c new file +3 -0
+
+	return 0;
+}

These writels may expand to code containing calls to outer_sync(), which will attempt to take a spinlock for the aurora l2. Given that the CPU isn't coherent, how does this play out with the exclusive store instruction in the lock?

Quoted 4 lines

I dug a little this subject: and I am not sure there is problem. In SMP mode, only the system cache mode of Aurora is used. In this mode, outer_cache.sync is void then outer_sync() won't call any function, so there will be no access to any spinlock.

Hmm, that is pretty subtle and it doesn't really solve the bigger picture. printk takes logbuf_lock, for example, and I'm sure that by the time you get to this code you will have relied on exclusives behaving correctly.

Will

#6 lore raw

PATCH v2 1/5 arm: mvebu: Added support for coherency fabric in mach-mvebu

Quoted 31 lines · Will Deacon
Quoted 30 lines · Gregory CLEMENT
Quoted 28 lines · Will Deacon
Quoted 12 lines · Gregory CLEMENT
arch/arm/mach-mvebu/coherency.c new file +12 -0
+int set_cpu_coherent(unsigned int hw_cpu_id, int smp_group_id)
+{
+	int reg;
+
+	if (!coherency_base) {
+		pr_warn("Can't make CPU %d cache coherent.\n", hw_cpu_id);
+		pr_warn("Coherency fabric is not initialized\n");
+		return 1;
+	}
+
+	/* Enable the CPU in coherency fabric */
+	reg = readl(coherency_base + COHERENCY_FABRIC_CTL_OFFSET);

arch/arm/mach-mvebu/coherency.c line 76

Quoted 10 lines
arch/arm/mach-mvebu/coherency.c new file +10 -0
+	reg |= 1 << (24 + hw_cpu_id);
+	writel(reg, coherency_base + COHERENCY_FABRIC_CTL_OFFSET);
+
+	/* Add CPU to SMP group */
+	reg = readl(coherency_base + COHERENCY_FABRIC_CFG_OFFSET);
+	reg |= 1 << (16 + hw_cpu_id + (smp_group_id == 0 ? 8 : 0));
+	writel(reg, coherency_base + COHERENCY_FABRIC_CFG_OFFSET);
+
+	return 0;
+}

These writels may expand to code containing calls to outer_sync(), which will attempt to take a spinlock for the aurora l2. Given that the CPU isn't coherent, how does this play out with the exclusive store instruction in the lock?

Quoted 9 lines
Quoted 4 lines

I dug a little this subject: and I am not sure there is problem. In SMP mode, only the system cache mode of Aurora is used. In this mode, outer_cache.sync is void then outer_sync() won't call any function, so there will be no access to any spinlock.

Hmm, that is pretty subtle and it doesn't really solve the bigger picture. printk takes logbuf_lock, for example, and I'm sure that by the time you get to this code you will have relied on exclusives behaving correctly.

Hi Will,
I get an answer from Marvell engineers:
"STREX on non-shareable and/or non-cacheable memory regions is supported."

Gregory

#7 lore raw

PATCH v2 1/5 arm: mvebu: Added support for coherency fabric in mach-mvebu

Quoted 13 lines · Gregory CLEMENT
Quoted 12 lines · Will Deacon
Quoted 10 lines · Gregory CLEMENT
Quoted 4 lines · Will Deacon

These writels may expand to code containing calls to outer_sync(), which will attempt to take a spinlock for the aurora l2. Given that the CPU isn't coherent, how does this play out with the exclusive store instruction in the lock?

I dug a little this subject: and I am not sure there is problem. In SMP mode, only the system cache mode of Aurora is used. In this mode, outer_cache.sync is void then outer_sync() won't call any function, so there will be no access to any spinlock.

Quoted 8 lines
Quoted 4 lines

Hmm, that is pretty subtle and it doesn't really solve the bigger picture. printk takes logbuf_lock, for example, and I'm sure that by the time you get to this code you will have relied on exclusives behaving correctly.

Hi Will,
I get an answer from Marvell engineers:
"STREX on non-shareable and/or non-cacheable memory regions is supported."

Interesting, thanks for asking them about this. Does this mean that:

1. When not running coherently (i.e. before initialising the coherency fabric), memory is treated as non-shareable, non-cacheable?

2. If (1), then are exclusive accesses the only way to achieve coherent memory accesses in this scenario?

If so, you still have a problem with write locks, where the unlock code does a regular str to clear the status. atomic_{read,set} also uses regular memory accesses, so I think you'll get some surprises there when you add explicit memory barriers and expect things to be visible between threads.

Do memory barriers have different semantics depending on the state of your coherency fabric?

Cheers,

Will

#8 lore raw

PATCH v2 1/5 arm: mvebu: Added support for coherency fabric in mach-mvebu

Quoted 19 lines · Will Deacon
Quoted 18 lines · Gregory CLEMENT
Quoted 16 lines · Will Deacon
Quoted 10 lines · Gregory CLEMENT
Quoted 4 lines · Will Deacon

These writels may expand to code containing calls to outer_sync(), which will attempt to take a spinlock for the aurora l2. Given that the CPU isn't coherent, how does this play out with the exclusive store instruction in the lock?

I dug a little this subject: and I am not sure there is problem. In SMP mode, only the system cache mode of Aurora is used. In this mode, outer_cache.sync is void then outer_sync() won't call any function, so there will be no access to any spinlock.

Hmm, that is pretty subtle and it doesn't really solve the bigger picture. printk takes logbuf_lock, for example, and I'm sure that by the time you get to this code you will have relied on exclusives behaving correctly.

Quoted 5 lines
Quoted 3 lines

Hi Will,
I get an answer from Marvell engineers:
"STREX on non-shareable and/or non-cacheable memory regions is supported."

Interesting, thanks for asking them about this. Does this mean that:

Here come the answers to your new questions

Quoted 4 lines

1. When not running coherently (i.e. before initialising the coherency fabric), memory is treated as non-shareable, non-cacheable?

It can be cacheable. The shared memory (as defined on the page table) will NOT be coherent by HW.

Quoted 3 lines

2. If (1), then are exclusive accesses the only way to achieve coherent memory accesses in this scenario?

I quote: "I suspect there is terminology miss-use: exclusive accesses are NOT used to achieve memory coherency - they are used to achieve atomicity. To achieve memory coherency while fabric is configured to be non-coherent, SW should use maintenance operations over the L1 caches.suspect there is terminology miss-use: exclusive accesses are NOT used to achieve memory coherency - "they are used to achieve atomicity. To achieve memory coherency while fabric is configured to be non-coherent, SW should use maintenance operations over the L1 caches.

Quoted 7 lines

If so, you still have a problem with write locks, where the unlock code does a regular str to clear the status. atomic_{read,set} also uses regular memory accesses, so I think you'll get some surprises there when you add explicit memory barriers and expect things to be visible between threads.

Do memory barriers have different semantics depending on the state of your coherency fabric?

No

Signature · 4 lines
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
#9 lore raw

PATCH v2 1/5 arm: mvebu: Added support for coherency fabric in mach-mvebu

Hi Gregory,

Quoted 4 lines · Gregory CLEMENT
Quoted 1 line · Will Deacon

Interesting, thanks for asking them about this. Does this mean that:

Here come the answers to your new questions

Great, thanks for the quick turn-around!

Quoted 6 lines
Quoted 3 lines

1. When not running coherently (i.e. before initialising the coherency fabric), memory is treated as non-shareable, non-cacheable?

It can be cacheable. The shared memory (as defined on the page table) will NOT be coherent by HW.

Ok, so we really are incoherent before enabling the fabric.

Quoted 8 lines
Quoted 2 lines

2. If (1), then are exclusive accesses the only way to achieve coherent memory accesses in this scenario?

I quote: "I suspect there is terminology miss-use: exclusive accesses are NOT used to achieve memory coherency - they are used to achieve atomicity. To achieve memory coherency while fabric is configured to be non-coherent, SW should use maintenance operations over the L1 caches."

Ok, so if I'm understanding correctly then I don't really see the usefulness of having working exclusives that are incoherent. Surely it means that you can guarantee mutual exclusion on a lock variable, but the value you actually end up reading from the lock is junk unless you litter the accessors with cache clean operations?

Anyway, that's by-the-by as this is all called early enough that we shouldn't care. The thing I don't like now is that the fabric initialisation is done entirely differently on the primary CPU than the secondaries. The primary probes the device-tree (well, it's also now hard-coded for v2) and accesses the registers from a C function(armada_370_xp_set_cpu_coherent) whilst the secondaries have hardcoded addresses and access via asm (armada_xp_secondary_startup).

Will

#10 lore raw

PATCH v2 1/5 arm: mvebu: Added support for coherency fabric in mach-mvebu

Quoted 34 lines · Will Deacon

Hi Gregory,

Quoted 4 lines · Gregory CLEMENT
Quoted 1 line · Will Deacon

Interesting, thanks for asking them about this. Does this mean that:

Here come the answers to your new questions

Great, thanks for the quick turn-around!

Quoted 6 lines
Quoted 3 lines

1. When not running coherently (i.e. before initialising the coherency fabric), memory is treated as non-shareable, non-cacheable?

It can be cacheable. The shared memory (as defined on the page table) will NOT be coherent by HW.

Ok, so we really are incoherent before enabling the fabric.

Quoted 8 lines
Quoted 2 lines

2. If (1), then are exclusive accesses the only way to achieve coherent memory accesses in this scenario?

I quote: "I suspect there is terminology miss-use: exclusive accesses are NOT used to achieve memory coherency - they are used to achieve atomicity. To achieve memory coherency while fabric is configured to be non-coherent, SW should use maintenance operations over the L1 caches."

Ok, so if I'm understanding correctly then I don't really see the usefulness of having working exclusives that are incoherent. Surely it means that you can guarantee mutual exclusion on a lock variable, but the value you actually end up reading from the lock is junk unless you litter the accessors with cache clean operations?

Quoted 7 lines

Anyway, that's by-the-by as this is all called early enough that we shouldn't care. The thing I don't like now is that the fabric initialisation is done entirely differently on the primary CPU than the secondaries. The primary probes the device-tree (well, it's also now hard-coded for v2) and accesses the registers from a C function(armada_370_xp_set_cpu_coherent) whilst the secondaries have hardcoded addresses and access via asm (armada_xp_secondary_startup).

Now it is hardcoded in both case as you pointed it. So the last difference is setup from a C function or via asm.

The differences between primary and secondary CPU when they enable the coherency, is due to the fact that we really are in a different situation. For primary CPU, as it is the only CPU online it doesn't need to enable the coherency from the beginning, so we can wait to have MMU enable and convenient feature. Whereas for the secondary CPU they need the coherency from the very beginning are by definition they won't be alone. That's why this very first instruction are written in asm and they use physical address.

I don't see how to handle it in a different way.

Gregory

#11 lore raw

PATCH v2 1/5 arm: mvebu: Added support for coherency fabric in mach-mvebu

Quoted 22 lines · Gregory CLEMENT
Quoted 7 lines · Will Deacon

Anyway, that's by-the-by as this is all called early enough that we shouldn't care. The thing I don't like now is that the fabric initialisation is done entirely differently on the primary CPU than the secondaries. The primary probes the device-tree (well, it's also now hard-coded for v2) and accesses the registers from a C function(armada_370_xp_set_cpu_coherent) whilst the secondaries have hardcoded addresses and access via asm (armada_xp_secondary_startup).

Now it is hardcoded in both case as you pointed it. So the last difference is setup from a C function or via asm.

The differences between primary and secondary CPU when they enable the coherency, is due to the fact that we really are in a different situation. For primary CPU, as it is the only CPU online it doesn't need to enable the coherency from the beginning, so we can wait to have MMU enable and convenient feature. Whereas for the secondary CPU they need the coherency from the very beginning are by definition they won't be alone. That's why this very first instruction are written in asm and they use physical address.

Quoted 1 line

I don't see how to handle it in a different way.

The code paths are fine, I would just like to see less duplication. Can you make the asm function PCS compliant and call it from C for the primary (setting the link register to secondary_startup for the secondary cores)?

Will

#12 lore raw

PATCH v2 1/5 arm: mvebu: Added support for coherency fabric in mach-mvebu

Quoted 23 lines · Will Deacon
Quoted 22 lines · Gregory CLEMENT
Quoted 7 lines · Will Deacon

Anyway, that's by-the-by as this is all called early enough that we shouldn't care. The thing I don't like now is that the fabric initialisation is done entirely differently on the primary CPU than the secondaries. The primary probes the device-tree (well, it's also now hard-coded for v2) and accesses the registers from a C function(armada_370_xp_set_cpu_coherent) whilst the secondaries have hardcoded addresses and access via asm (armada_xp_secondary_startup).

Now it is hardcoded in both case as you pointed it. So the last difference is setup from a C function or via asm.

The differences between primary and secondary CPU when they enable the coherency, is due to the fact that we really are in a different situation. For primary CPU, as it is the only CPU online it doesn't need to enable the coherency from the beginning, so we can wait to have MMU enable and convenient feature. Whereas for the secondary CPU they need the coherency from the very beginning are by definition they won't be alone. That's why this very first instruction are written in asm and they use physical address.

Quoted 5 lines
Quoted 1 line

I don't see how to handle it in a different way.

The code paths are fine, I would just like to see less duplication. Can you make the asm function PCS compliant and call it from C for the primary (setting the link register to secondary_startup for the secondary cores)?

Have you a pointer on how to do it (make the asm function PCS compliant)?

I will also need to add a parameter, because the base address are not the same between primary CPU and secondary CPUS. With the first we use virtual address whereas with the second the physical address.

Quoted 8 lines

Will

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Signature · 4 lines
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
#13 lore raw

PATCH v2 1/5 arm: mvebu: Added support for coherency fabric in mach-mvebu

Quoted 7 lines · Gregory CLEMENT
Quoted 4 lines · Will Deacon

The code paths are fine, I would just like to see less duplication. Can you make the asm function PCS compliant and call it from C for the primary (setting the link register to secondary_startup for the secondary cores)?

Have you a pointer on how to do it (make the asm function PCS compliant)?

Take a look at the PCS document:

http://infocenter.arm.com/help/topic/com.arm.doc.ihi0042d/IHI0042D_aapcs.pdf

For your case it's really simple though as you don't need to use the stack:
just take the base address in r0 and use a subset of r1-r3 for temporaries while setting up the control registers. Then mov pc, lr at the end.

Quoted 3 lines

I will also need to add a parameter, because the base address are not the same between primary CPU and secondary CPUS. With the first we use virtual address whereas with the second the physical address.

Should be simple enough to add a secondary entry point immediately before, which initialises r0 and lr.

Will

#14 lore raw

PATCH v2 2/5 arm: mvebu: Added initial support for power managmement service unit

In reply to Gregory CLEMENT

From: Yehuda Yitschak <[email protected]>

---
 .../devicetree/bindings/arm/armada-370-xp-pmsu.txt |   20 +++++
 arch/arm/boot/dts/armada-xp.dtsi                   |    6 ++
 arch/arm/mach-mvebu/Makefile                       |    2 +-
 arch/arm/mach-mvebu/common.h                       |    1 +
 arch/arm/mach-mvebu/pmsu.c                         |   78 ++++++++++++++++++++
 arch/arm/mach-mvebu/pmsu.h                         |   16 ++++
 6 files changed, 122 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt
 create mode 100644 arch/arm/mach-mvebu/pmsu.c
 create mode 100644 arch/arm/mach-mvebu/pmsu.h
Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt new file +20 -0
new file mode 100644
index 0000000..926b4d6
@@ -0,0 +1,20 @@
+Power Management Service Unit(PMSU)
+-----------------------------------
+Available on Marvell SOCs: Armada 370 and Armada XP
+
+Required properties:
+
+- compatible: "marvell,armada-370-xp-pmsu"
+
+- reg: Should contain PMSU registers location and length. First pair
+  for the per-CPU SW Reset Control registers, second pair for the
+  Power Management Service Unit.
+
+Example:
+
+armada-370-xp-pmsu at d0022000 {
+	compatible = "marvell,armada-370-xp-pmsu";
+	reg = <0xd0022100 0x430>,
+	      <0xd0020800 0x20>;
+};
+
arch/arm/boot/dts/armada-xp.dtsi +6 -0
index a564b52..f521ed8 100644
@@ -27,6 +27,12 @@
 		    <0xd0021870 0x58>;
 	};
 
+	armada-370-xp-pmsu at d0022000 {
+		compatible = "marvell,armada-370-xp-pmsu";
+		reg = <0xd0022100 0x430>,
+		      <0xd0020800 0x20>;
+	};
+
 	cpus {
 	    #address-cells = <1>;
 	    #size-cells = <0>;
arch/arm/mach-mvebu/Makefile +1 -1
index abd6d3b..8e6e50b 100644
@@ -2,4 +2,4 @@ ccflags-$(CONFIG_ARCH_MULTIPLATFORM) := -I$(srctree)/$(src)/include \
 	-I$(srctree)/arch/arm/plat-orion/include
 
 obj-y += system-controller.o
-obj-$(CONFIG_MACH_ARMADA_370_XP) += armada-370-xp.o irq-armada-370-xp.o addr-map.o coherency.o
+obj-$(CONFIG_MACH_ARMADA_370_XP) += armada-370-xp.o irq-armada-370-xp.o addr-map.o coherency.o pmsu.o
arch/arm/mach-mvebu/common.h +1 -0
index ea08919..74ee0b2 100644
@@ -23,4 +23,5 @@ void armada_370_xp_handle_irq(struct pt_regs *regs);
 
 
 int armada_370_xp_coherency_init(void);
+int armada_370_xp_pmsu_init(void);
 #endif
arch/arm/mach-mvebu/pmsu.c new file +78 -0

78 lines not on this page · raw message

arch/arm/mach-mvebu/pmsu.h new file +16 -0
new file mode 100644
index 0000000..bafac8e
@@ -0,0 +1,16 @@
+/*
+ * Power Management Service Unit (PMSU) support for Armada 370/XP platforms.
+ *
+ * Copyright (C) 2012 Marvell
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#ifndef __MACH_MVEBU_PMSU_H
+#define __MACH_MVEBU_PMSU_H
+
+int armada_xp_boot_cpu(unsigned int cpu_id, void __iomem *phys_addr);
+
+#endif	/* __MACH_370_XP_PMSU_H */
Signature · 1 line
1.7.9.5
#15 lore raw

PATCH v2 2/5 arm: mvebu: Added initial support for power managmement service unit

Quoted 46 lines · Gregory CLEMENT
arch/arm/mach-mvebu/pmsu.c new file +40 -0
new file mode 100644
index 0000000..cee020b
@@ -0,0 +1,78 @@
+/*
+ * Power Management Service Unit(PMSU) support for Armada 370/XP platforms.
+ *
+ * Copyright (C) 2012 Marvell
+ *
+ * Yehuda Yitschak <[email protected]>
+ * Gregory Clement <[email protected]>
+ * Thomas Petazzoni <[email protected]>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ *
+ * The Armada 370 and Armada XP SOCs have a power management service
+ * unit which is responsible for powering down and waking up CPUs and
+ * other SOC units
+ */
+
+#include <linux/kernel.h>
+#include <linux/init.h>
+#include <linux/of_address.h>
+#include <linux/io.h>
+#include <linux/smp.h>
+#include <asm/smp_plat.h>
+
+static void __iomem *pmsu_mp_base;
+static void __iomem *pmsu_reset_base;
+
+#define PMSU_BOOT_ADDR_REDIRECT_OFFSET(cpu)	((cpu * 0x100) + 0x24)
+#define PMSU_RESET_CTL_OFFSET(cpu)		(cpu * 0x8)
+
+static struct of_device_id of_pmsu_table[] = {
+	{.compatible = "marvell,armada-370-xp-pmsu"},
+	{ /* end of list */ },
+};
+
+#ifdef CONFIG_SMP
+int armada_xp_boot_cpu(unsigned int cpu_id, void __iomem *boot_addr)
+{
+	int reg, hw_cpu;

arch/arm/mach-mvebu/pmsu.c line 50

Quoted 10 lines
arch/arm/mach-mvebu/pmsu.c new file +10 -0
+
+	if (!pmsu_mp_base || !pmsu_reset_base) {
+		pr_warn("Can't boot CPU. PMSU is uninitialized\n");
+		return 1;
+	}
+
+	hw_cpu = cpu_logical_map(cpu_id);
+
+	writel(virt_to_phys(boot_addr), pmsu_mp_base +
+			PMSU_BOOT_ADDR_REDIRECT_OFFSET(hw_cpu));

virt_to_phys on an __iomem * doesn't feel right to me...

arch/arm/mach-mvebu/pmsu.c line 53

Quoted 2 lines
arch/arm/mach-mvebu/pmsu.c new file +2 -0
+	/* Make sure value hits memory before reset */
+	dsb();

writel has barrier semantics -- you shouldn't need this dsb.

Will

#16 lore raw

PATCH v2 2/5 arm: mvebu: Added initial support for power managmement service unit

Quoted 17 lines · Will Deacon
Quoted 4 lines · Gregory CLEMENT
arch/arm/mach-mvebu/pmsu.c new file +4 -0
+#ifdef CONFIG_SMP
+int armada_xp_boot_cpu(unsigned int cpu_id, void __iomem *boot_addr)
+{
+	int reg, hw_cpu;

arch/arm/mach-mvebu/pmsu.c line 50

Quoted 10 lines
arch/arm/mach-mvebu/pmsu.c new file +10 -0
+
+	if (!pmsu_mp_base || !pmsu_reset_base) {
+		pr_warn("Can't boot CPU. PMSU is uninitialized\n");
+		return 1;
+	}
+
+	hw_cpu = cpu_logical_map(cpu_id);
+
+	writel(virt_to_phys(boot_addr), pmsu_mp_base +
+			PMSU_BOOT_ADDR_REDIRECT_OFFSET(hw_cpu));

virt_to_phys on an __iomem * doesn't feel right to me...

It isn't correct - I suspect the __iomem there is just wrong - and it needs the callsite also checking.

Quoted 4 lines

arch/arm/mach-mvebu/pmsu.c line 53

Quoted 2 lines
arch/arm/mach-mvebu/pmsu.c new file +2 -0
+	/* Make sure value hits memory before reset */
+	dsb();

writel has barrier semantics -- you shouldn't need this dsb.

writel has a barrier before the write (to ensure that DMA agents see data that was written to memory when they are enabled by the write). There isn't a barrier after the write.

#17 lore raw

PATCH v2 2/5 arm: mvebu: Added initial support for power managmement service unit

Quoted 10 lines · Russell King - ARM Linux
Quoted 5 lines · Will Deacon

arch/arm/mach-mvebu/pmsu.c line 53

Quoted 2 lines · Gregory CLEMENT
arch/arm/mach-mvebu/pmsu.c new file +2 -0
+	/* Make sure value hits memory before reset */
+	dsb();

writel has barrier semantics -- you shouldn't need this dsb.

writel has a barrier before the write (to ensure that DMA agents see data that was written to memory when they are enabled by the write). There isn't a barrier after the write.

Indeed, but there's a following write to actually do the reset, so the dsb should come from there.

Will

#18 lore raw

PATCH v2 3/5 arm: mvebu: Added IPI support via doorbells

In reply to Gregory CLEMENT

From: Yehuda Yitschak <[email protected]>

---
 .../devicetree/bindings/arm/armada-370-xp-mpic.txt |   12 ++-
 arch/arm/boot/dts/armada-xp.dtsi                   |    2 +-
 arch/arm/mach-mvebu/armada-370-xp.h                |   10 +++
 arch/arm/mach-mvebu/irq-armada-370-xp.c            |   92 ++++++++++++++++++--
 4 files changed, 106 insertions(+), 10 deletions(-)
Documentation/devicetree/bindings/arm/armada-370-xp-mpic.txt +9 -3
index 70c0dc5..61df564 100644
@@ -6,9 +6,15 @@ Required properties:
 - interrupt-controller: Identifies the node as an interrupt controller.
 - #interrupt-cells: The number of cells to define the interrupts. Should be 1.
   The cell is the IRQ number
+
 - reg: Should contain PMIC registers location and length. First pair
   for the main interrupt registers, second pair for the per-CPU
-  interrupt registers
+  interrupt registers. For this last pair, to be compliant with SMP
+  support, the "virtual" must be use (For the record, these registers
+  automatically map to the interrupt controller registers of the
+  current CPU)
+
+
 
 Example:
 
@@ -18,6 +24,6 @@ Example:
               #address-cells = <1>;
               #size-cells = <1>;
               interrupt-controller;
-              reg = <0xd0020000 0x1000>,
-                    <0xd0021000 0x1000>;
+              reg = <0xd0020a00 0x1d0>,
+                    <0xd0021070 0x58>;
         };
arch/arm/boot/dts/armada-xp.dtsi +1 -1
index f521ed8..531619f 100644
@@ -24,7 +24,7 @@
 
 	mpic: interrupt-controller at d0020000 {
 	      reg = <0xd0020a00 0x1d0>,
-		    <0xd0021870 0x58>;
+		    <0xd0021070 0x58>;
 	};
 
 	armada-370-xp-pmsu at d0022000 {
arch/arm/mach-mvebu/armada-370-xp.h +10 -0
index aac9beb..dce590d 100644
@@ -19,4 +19,14 @@
 #define ARMADA_370_XP_REGS_VIRT_BASE	IOMEM(0xfeb00000)
 #define ARMADA_370_XP_REGS_SIZE		SZ_1M
 
+#ifndef __ASSEMBLY__
+
+#ifdef CONFIG_SMP
+#include <linux/cpumask.h>
+
+void armada_mpic_send_doorbell(const struct cpumask *mask, unsigned int irq);
+void armada_xp_mpic_smp_cpu_init(void);
+#endif
+#endif
+
 #endif /* __MACH_ARMADA_370_XP_H */
arch/arm/mach-mvebu/irq-armada-370-xp.c +86 -6

142 lines not on this page · raw message

Signature · 1 line
1.7.9.5
#19 lore raw

PATCH v2 4/5 arm: mm: Added support for PJ4B cpu and init routines

In reply to Gregory CLEMENT

From: Yehuda Yitschak <[email protected]>

---
 arch/arm/mach-mvebu/Kconfig |    2 +-
 arch/arm/mm/Kconfig         |    4 ++++
 arch/arm/mm/proc-v7.S       |   43 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 48 insertions(+), 1 deletion(-)
arch/arm/mach-mvebu/Kconfig +1 -1
index 17d246b..9bfaa0c 100644
@@ -22,7 +22,7 @@ config MVEBU_CLK_CPU
 config MACH_ARMADA_370_XP
 	bool
 	select ARMADA_370_XP_TIMER
-	select CPU_V7
+	select CPU_PJ4B
 
 config MACH_ARMADA_370
 	bool "Marvell Armada 370 boards"
arch/arm/mm/Kconfig +4 -0
index 94186b6..3fd629d 100644
@@ -352,6 +352,10 @@ config CPU_PJ4
 	select ARM_THUMBEE
 	select CPU_V7
 
+config CPU_PJ4B
+	bool
+	select CPU_V7
+
 # ARMv6
 config CPU_V6
 	bool "Support ARM V6 processor" if ARCH_INTEGRATOR || MACH_REALVIEW_EB || MACH_REALVIEW_PBX
arch/arm/mm/proc-v7.S +43 -0

55 lines not on this page · raw message

Signature · 1 line
1.7.9.5
#20 lore raw

PATCH v2 4/5 arm: mm: Added support for PJ4B cpu and init routines

Russell,

Do you have any comments on this patch? Else do you agree to give your acked-by?

Thanks,

Gregory

Quoted 100 lines · Gregory CLEMENT

From: Yehuda Yitschak <[email protected]>

---
 arch/arm/mach-mvebu/Kconfig |    2 +-
 arch/arm/mm/Kconfig         |    4 ++++
 arch/arm/mm/proc-v7.S       |   43 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 48 insertions(+), 1 deletion(-)
arch/arm/mach-mvebu/Kconfig +1 -1
index 17d246b..9bfaa0c 100644
@@ -22,7 +22,7 @@ config MVEBU_CLK_CPU
 config MACH_ARMADA_370_XP
 	bool
 	select ARMADA_370_XP_TIMER
-	select CPU_V7
+	select CPU_PJ4B
 
 config MACH_ARMADA_370
 	bool "Marvell Armada 370 boards"
arch/arm/mm/Kconfig +4 -0
index 94186b6..3fd629d 100644
@@ -352,6 +352,10 @@ config CPU_PJ4
 	select ARM_THUMBEE
 	select CPU_V7
 
+config CPU_PJ4B
+	bool
+	select CPU_V7
+
 # ARMv6
 config CPU_V6
 	bool "Support ARM V6 processor" if ARCH_INTEGRATOR || MACH_REALVIEW_EB || MACH_REALVIEW_PBX
arch/arm/mm/proc-v7.S +43 -0
index 846d279..1a373c2 100644
@@ -169,6 +169,39 @@ __v7_ca15mp_setup:
 	orreq	r0, r0, r10			@ Enable CPU-specific SMP bits
 	mcreq	p15, 0, r0, c1, c0, 1
 #endif
+
+__v7_pj4b_setup:
+#ifdef CONFIG_CPU_PJ4B
+	/* Auxiliary Debug Modes Control 1 Register */
+	mrc	p15, 1,	r0, c15, c1, 1
+	orr	r0, r0, #(1 << 16)     @ Disable data transfer for clean line.
+	orr	r0, r0, #(1 << 5)      @ Enable the back off of STREX instr
+	orr	r0, r0, #(1 << 8)      @ Disable Internal Parity Handling
+	bic	r0, r0, #(1 << 2)      @ Disable Static BP
+	mcr	p15, 1,	r0, c15, c1, 1
+
+	/* Auxiliary Debug Modes Control 2 Register */
+	mrc	p15, 1,	r0, c15, c1, 2
+	bic	r0, r0, #(1 << 23)   @ Enable fast LDR.
+	orr	r0, r0, #(1 << 25)   @ Dont interleave write and snoop data.
+	orr	r0, r0, #(1 << 27)   @ Disable Critical Word First feature.
+	orr	r0, r0, #(1 << 29)   @ Disable outstanding non cacheable request
+	orr	r0, r0, #(1 << 30)   @ L1 replacement - Strict round robin
+	mcr	p15, 1,	r0, c15, c1, 2
+
+	/* Auxiliary Functional Modes Control Register 0 */
+	mrc	p15, 1,	r0, c15, c2, 0
+	orr	r0, r0, #(1 << 2)     @ Support L1 parity checking
+	orr	r0, r0, #(1 << 8)     @ Broadcast Cache and TLB maintenance
+	mcr	p15, 1,	r0, c15, c2, 0
+
+	/* Auxiliary Debug Modes Control 0 Register */
+	mrc	p15, 1,	r0, c15, c1, 0
+	orr	r0, r0, #(1 << 22)   @ WFI/WFE - serve the DVM and back to idle
+	mcr	p15, 1,	r0, c15, c1, 0
+
+#endif /* CONFIG_CPU_PJ4B */
+
 __v7_setup:
 	adr	r12, __v7_setup_stack		@ the local stack
 	stmia	r12, {r0-r5, r7, r9, r11, lr}
@@ -342,6 +375,16 @@ __v7_ca9mp_proc_info:
 	.long	0xff0ffff0
 	__v7_proc __v7_ca9mp_setup
 	.size	__v7_ca9mp_proc_info, . - __v7_ca9mp_proc_info
+
+	/*
+	 * Marvell PJ4B processor.
+	 */
+	.type   __v7_pj4b_proc_info, #object
+__v7_pj4b_proc_info:
+	.long	0x562f5842
+	.long	0xffffffff
+	__v7_proc __v7_pj4b_setup
+	.size	__v7_pj4b_proc_info, . - __v7_pj4b_proc_info
 #endif	/* CONFIG_ARM_LPAE */
 
 	/*
Signature · 4 lines
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
#21 lore raw

PATCH v2 4/5 arm: mm: Added support for PJ4B cpu and init routines

Quoted 7 lines · Gregory CLEMENT
Quoted 7 lines

__v7_setup:

arch/arm/mm/proc-v7.S +0 -0
 	adr	r12, __v7_setup_stack		@ the local stack
 	stmia	r12, {r0-r5, r7, r9, r11, lr}
@@ -342,6 +375,16 @@ __v7_ca9mp_proc_info:
 	.long	0xff0ffff0
 	__v7_proc __v7_ca9mp_setup
 	.size	__v7_ca9mp_proc_info, . - __v7_ca9mp_proc_info
Quoted 8 lines

arch/arm/mm/proc-v7.S __v7_ca9mp_proc_info: · line 385

Quoted 8 lines
arch/arm/mm/proc-v7.S +8 -0
+
+	/*
+	 * Marvell PJ4B processor.
+	 */
+	.type   __v7_pj4b_proc_info, #object
+__v7_pj4b_proc_info:
+	.long	0x562f5842
+	.long	0xffffffff

This mask seems a little excessive -- does your cpuid not contain revision information? If so, I'm not sure we want to update the code every time there is a new spin of the same CPU.

Will

#22 lore raw

PATCH v2 4/5 arm: mm: Added support for PJ4B cpu and init routines

In reply to Gregory CLEMENT

arch/arm/mm/proc-v7.S __v7_ca15mp_setup: · line 189

Quoted 7 lines · Gregory CLEMENT
arch/arm/mm/proc-v7.S +7 -0
+	/* Auxiliary Debug Modes Control 2 Register */
+	mrc	p15, 1,	r0, c15, c1, 2
+	bic	r0, r0, #(1 << 23)   @ Enable fast LDR.
+	orr	r0, r0, #(1 << 25)   @ Dont interleave write and snoop data.
+	orr	r0, r0, #(1 << 27)   @ Disable Critical Word First feature.
+	orr	r0, r0, #(1 << 29)   @ Disable outstanding non cacheable request
+	orr	r0, r0, #(1 << 30)   @ L1 replacement - Strict round robin

This just looks silly to me - setting five bits with five instructions when they can all be done in one instruction. Yes, I know you want to comment it, but there's other ways to achieve that.

arch/arm/mm/proc-v7.S __v7_ca9mp_proc_info: · line 385

Quoted 3 lines
arch/arm/mm/proc-v7.S +3 -0
+__v7_pj4b_proc_info:
+	.long	0x562f5842
+	.long	0xffffffff

Same comment here as Will :)

#23 lore raw

PATCH v2 5/5 arm: mvebu: Added SMP support for Armada XP

In reply to Gregory CLEMENT

From: Yehuda Yitschak <[email protected]>

1. added smp init functions in platsmp.c
2. added secondary cpu entry point in headsmp.S
3. added hotplog initial support in hotplug.c
4. added SMP support for PJ4B cpu

---
 arch/arm/boot/dts/armada-xp.dtsi    |    4 ++
 arch/arm/configs/mvebu_defconfig    |    3 +
 arch/arm/mach-mvebu/Kconfig         |    1 +
 arch/arm/mach-mvebu/Makefile        |    2 +
 arch/arm/mach-mvebu/armada-370-xp.c |    3 +
 arch/arm/mach-mvebu/common.h        |    3 +
 arch/arm/mach-mvebu/headsmp.S       |   66 +++++++++++++++++++
 arch/arm/mach-mvebu/hotplug.c       |   30 +++++++++
 arch/arm/mach-mvebu/platsmp.c       |  124 +++++++++++++++++++++++++++++++++++
 arch/arm/mm/proc-v7.S               |    3 +
 10 files changed, 239 insertions(+)
 create mode 100644 arch/arm/mach-mvebu/headsmp.S
 create mode 100644 arch/arm/mach-mvebu/hotplug.c
 create mode 100644 arch/arm/mach-mvebu/platsmp.c
arch/arm/boot/dts/armada-xp.dtsi +4 -0
index 531619f..7f968dc 100644
@@ -38,24 +38,28 @@
 	    #size-cells = <0>;
 
 	    cpu at 0 {
+		device_type = "cpu";
 	       	compatible = "marvell,sheeva-v7";
 		reg = <0>;
 		clocks = <&cpuclk 0>;
 	    };
 
 	    cpu at 1 {
+		device_type = "cpu";
 		compatible = "marvell,sheeva-v7";
 		reg = <1>;
 		clocks = <&cpuclk 1>;
 	    };
 
 	    cpu at 2 {
+		device_type = "cpu";
 		compatible = "marvell,sheeva-v7";
 		reg = <2>;
 		clocks = <&cpuclk 2>;
 	    };
 
 	    cpu at 3 {
+		device_type = "cpu";
 		compatible = "marvell,sheeva-v7";
 		reg = <3>;
 		clocks = <&cpuclk 3>;
arch/arm/configs/mvebu_defconfig +3 -0
index 3458752..da598d3 100644
@@ -12,6 +12,9 @@ CONFIG_ARCH_MVEBU=y
 CONFIG_MACH_ARMADA_370=y
 CONFIG_MACH_ARMADA_XP=y
 # CONFIG_CACHE_L2X0 is not set
+# CONFIG_SWP_EMULATE is not set
+CONFIG_SMP=y
+# CONFIG_LOCAL_TIMERS is not set
 CONFIG_AEABI=y
 CONFIG_HIGHMEM=y
 # CONFIG_COMPACTION is not set
arch/arm/mach-mvebu/Kconfig +1 -0
index 9bfaa0c..d70afe3 100644
@@ -22,6 +22,7 @@ config MVEBU_CLK_CPU
 config MACH_ARMADA_370_XP
 	bool
 	select ARMADA_370_XP_TIMER
+	select HAVE_SMP
 	select CPU_PJ4B
 
 config MACH_ARMADA_370
arch/arm/mach-mvebu/Makefile +2 -0
index 8e6e50b..eb3cbd1 100644
@@ -3,3 +3,5 @@ ccflags-$(CONFIG_ARCH_MULTIPLATFORM) := -I$(srctree)/$(src)/include \
 
 obj-y += system-controller.o
 obj-$(CONFIG_MACH_ARMADA_370_XP) += armada-370-xp.o irq-armada-370-xp.o addr-map.o coherency.o pmsu.o
+obj-$(CONFIG_SMP)                += platsmp.o headsmp.o
+obj-$(CONFIG_HOTPLUG_CPU)        += hotplug.o
arch/arm/mach-mvebu/armada-370-xp.c +3 -0
index 2af6ce5..66befa1 100644
@@ -22,6 +22,7 @@
 #include <asm/mach/time.h>
 #include "armada-370-xp.h"
 #include "common.h"
+#include "coherency.h"
 
 static struct map_desc armada_370_xp_io_desc[] __initdata = {
 	{
@@ -50,6 +51,7 @@ struct sys_timer armada_370_xp_timer = {
 static void __init armada_370_xp_dt_init(void)
 {
 	of_platform_populate(NULL, of_default_bus_match_table, NULL, NULL);
+	coherency_init();
 }
 
 static const char * const armada_370_xp_dt_board_dt_compat[] = {
@@ -59,6 +61,7 @@ static const char * const armada_370_xp_dt_board_dt_compat[] = {
 };
 
 DT_MACHINE_START(ARMADA_XP_DT, "Marvell Aramada 370/XP (Device Tree)")
+	.smp		= smp_ops(armada_xp_smp_ops),
 	.init_machine	= armada_370_xp_dt_init,
 	.map_io		= armada_370_xp_map_io,
 	.init_irq	= armada_370_xp_init_irq,
arch/arm/mach-mvebu/common.h +3 -0
index 74ee0b2..86484bb 100644
@@ -21,7 +21,10 @@ void mvebu_clocks_init(void);
 void armada_370_xp_init_irq(void);
 void armada_370_xp_handle_irq(struct pt_regs *regs);
 
+void armada_xp_cpu_die(unsigned int cpu);
 
 int armada_370_xp_coherency_init(void);
 int armada_370_xp_pmsu_init(void);
+void armada_xp_secondary_startup(void);
+extern struct smp_operations armada_xp_smp_ops;
 #endif
arch/arm/mach-mvebu/headsmp.S new file +66 -0

66 lines not on this page · raw message

arch/arm/mach-mvebu/hotplug.c new file +30 -0
new file mode 100644
index 0000000..b228b6a
@@ -0,0 +1,30 @@
+/*
+ * Symmetric Multi Processing (SMP) support for Armada XP
+ *
+ * Copyright (C) 2012 Marvell
+ *
+ * Lior Amsalem <[email protected]>
+ * Gregory CLEMENT <[email protected]>
+ * Thomas Petazzoni <[email protected]>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/smp.h>
+#include <asm/proc-fns.h>
+
+/*
+ * platform-specific code to shutdown a CPU
+ *
+ * Called with IRQs disabled
+ */
+void __ref armada_xp_cpu_die(unsigned int cpu)
+{
+	cpu_do_idle();
+
+	/* We should never return from idle */
+	panic("mvebu: cpu %d unexpectedly exit from shutdown\n", cpu);
+}
arch/arm/mach-mvebu/platsmp.c new file +124 -0

124 lines not on this page · raw message

arch/arm/mm/proc-v7.S +3 -0
index 1a373c2..dcd97e4 100644
@@ -191,6 +191,9 @@ __v7_pj4b_setup:
 
 	/* Auxiliary Functional Modes Control Register 0 */
 	mrc	p15, 1,	r0, c15, c2, 0
+#ifdef CONFIG_SMP
+	orr	r0, r0, #(1 << 1)     @ Set SMP mode. Join the coherency fabric
+#endif
 	orr	r0, r0, #(1 << 2)     @ Support L1 parity checking
 	orr	r0, r0, #(1 << 8)     @ Broadcast Cache and TLB maintenance
 	mcr	p15, 1,	r0, c15, c2, 0
Signature · 1 line
1.7.9.5
#24 lore raw

v2 cover SMP support for Armada XP

In reply to Gregory CLEMENT

Hello,

On 10/29/2012 10:11 PM, Gregory CLEMENT wrote:
[...]

Quoted 3 lines

Arnd, Olof, as this code is SoC specific will you be able to pull it through Jason tree? Or should I push the part added in arch/arm/mm/proc-v7.S to support PJ4B CPU through Russell tree?

Before sending a new series with the fixes related to Will comments, I would like to know if I need to split the patches in order to send some parts to Russell or if you (Arnd and Olof) will be able to take the whole series (through Jason Cooper's git tree of course).

Quoted 60 lines

Changelog:
V1 -> V2:
- Rebased on to v3.7-rc3
- Fixed typos found by Alexandre Belloni
- Added clk_prepare_enable() before getting rate clk in set_secondary_cpus_clock()
- Add explanation in the binding documentation about the per-CPU interrupt registers: the address of the virtual register must be used.
- Removed the armada_xp prefix in the coherency.c file to be more compliant with the name convention of the other files.
- Coherency_init is now called from armada_370_xp_dt_init() and is no more an early_init() call. As the device tree is not available from an early_init(), it was useless to call coherency_init() so early. The need to be able to call some function very early during the boot were already resolved by using the hard code address of the register.

Regards,

Yehuda Yitschak (5):
arm: mvebu: Added support for coherency fabric in mach-mvebu
arm: mvebu: Added initial support for power managmement service unit
arm: mvebu: Added IPI support via doorbells
arm: mm: Added support for PJ4B cpu and init routines
arm: mvebu: Added SMP support for Armada XP

 .../devicetree/bindings/arm/armada-370-xp-mpic.txt |   12 +-
 .../devicetree/bindings/arm/armada-370-xp-pmsu.txt |   20 ++++
 .../devicetree/bindings/arm/coherency-fabric.txt   |   16 +++
 arch/arm/boot/dts/armada-370-xp.dtsi               |    5 +
 arch/arm/boot/dts/armada-xp.dtsi                   |   12 +-
 arch/arm/configs/mvebu_defconfig                   |    3 +
 arch/arm/mach-mvebu/Kconfig                        |    3 +-
 arch/arm/mach-mvebu/Makefile                       |    4 +-
 arch/arm/mach-mvebu/armada-370-xp.c                |    3 +
 arch/arm/mach-mvebu/armada-370-xp.h                |   10 ++
 arch/arm/mach-mvebu/coherency.c                    |   89 ++++++++++++++
 arch/arm/mach-mvebu/coherency.h                    |   21 ++++
 arch/arm/mach-mvebu/common.h                       |    6 +
 arch/arm/mach-mvebu/headsmp.S                      |   66 +++++++++++
 arch/arm/mach-mvebu/hotplug.c                      |   30 +++++
 arch/arm/mach-mvebu/irq-armada-370-xp.c            |   92 ++++++++++++++-
 arch/arm/mach-mvebu/platsmp.c                      |  124 ++++++++++++++++++++
 arch/arm/mach-mvebu/pmsu.c                         |   78 ++++++++++++
 arch/arm/mach-mvebu/pmsu.h                         |   16 +++
 arch/arm/mm/Kconfig                                |    4 +
 arch/arm/mm/proc-v7.S                              |   46 ++++++++
 21 files changed, 648 insertions(+), 12 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt
 create mode 100644 Documentation/devicetree/bindings/arm/coherency-fabric.txt
 create mode 100644 arch/arm/mach-mvebu/coherency.c
 create mode 100644 arch/arm/mach-mvebu/coherency.h
 create mode 100644 arch/arm/mach-mvebu/headsmp.S
 create mode 100644 arch/arm/mach-mvebu/hotplug.c
 create mode 100644 arch/arm/mach-mvebu/platsmp.c
 create mode 100644 arch/arm/mach-mvebu/pmsu.c
 create mode 100644 arch/arm/mach-mvebu/pmsu.h
Signature · 4 lines
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
#25 lore raw

v2 cover SMP support for Armada XP

Quoted 5 lines · Gregory CLEMENT

Before sending a new series with the fixes related to Will comments, I would like to know if I need to split the patches in order to send some parts to Russell or if you (Arnd and Olof) will be able to take the whole series (through Jason Cooper's git tree of course).

We can take the whole series through arm-soc, but it would be good to get an Ack from Russell for patch 4.

Arnd