SMP support for Armada XP

From: Yehuda Yitschak <[email protected]>

Signed-off-by Yehuda Yitschak <[email protected]>

Signed-off-by Gregory CLEMENT <[email protected]>

---
 arch/arm/boot/dts/armada-xp.dtsi        |    2 +-
 arch/arm/mach-mvebu/armada-370-xp.h     |   10 ++++
 arch/arm/mach-mvebu/irq-armada-370-xp.c |   92 +++++++++++++++++++++++++++++--
 3 files changed, 97 insertions(+), 7 deletions(-)

arch/arm/boot/dts/armada-xp.dtsi +0 -0

index f521ed8..531619f 100644

Quoted 15 lines · Gregory CLEMENT

From: Yehuda Yitschak <[email protected]>

Signed-off-by Yehuda Yitschak <[email protected]>

Signed-off-by Gregory CLEMENT <[email protected]>

---
 arch/arm/boot/dts/armada-xp.dtsi        |    2 +-
 arch/arm/mach-mvebu/armada-370-xp.h     |   10 ++++
 arch/arm/mach-mvebu/irq-armada-370-xp.c |   92 +++++++++++++++++++++++++++++--
 3 files changed, 97 insertions(+), 7 deletions(-)

arch/arm/boot/dts/armada-xp.dtsi +0 -0

index f521ed8..531619f 100644

arch/arm/boot/dts/armada-xp.dtsi line 28

Quoted 6 lines

arch/arm/boot/dts/armada-xp.dtsi +1 -1

@@ -24,7 +24,7 @@
 
 	mpic: interrupt-controller at d0020000 {
 	      reg = <0xd0020a00 0x1d0>,
-		    <0xd0021870 0x58>;
+		    <0xd0021070 0x58>;
 	};

Hi Gregory

Is this a bug fix needed for 3.7?

Andrew

Quoted 25 lines · Andrew Lunn

Quoted 15 lines · Gregory CLEMENT

From: Yehuda Yitschak <[email protected]>

Signed-off-by Yehuda Yitschak <[email protected]>

Signed-off-by Gregory CLEMENT <[email protected]>

---
 arch/arm/boot/dts/armada-xp.dtsi        |    2 +-
 arch/arm/mach-mvebu/armada-370-xp.h     |   10 ++++
 arch/arm/mach-mvebu/irq-armada-370-xp.c |   92 +++++++++++++++++++++++++++++--
 3 files changed, 97 insertions(+), 7 deletions(-)

arch/arm/boot/dts/armada-xp.dtsi +0 -0

index f521ed8..531619f 100644

arch/arm/boot/dts/armada-xp.dtsi line 28

Quoted 6 lines

arch/arm/boot/dts/armada-xp.dtsi +1 -1

@@ -24,7 +24,7 @@
 
 	mpic: interrupt-controller at d0020000 {
 	      reg = <0xd0020a00 0x1d0>,
-		    <0xd0021870 0x58>;
+		    <0xd0021070 0x58>;
 	};

Hi Gregory

Quoted 4 lines

Is this a bug fix needed for 3.7?

Andrew

I don't think so.
We extended the reg map to be able to use registers only used for SMP. When we didn't have SMP support the mapping was correct, and now by introducing SMP we extend it.

Quoted 30 lines · Gregory CLEMENT

Quoted 29 lines · Andrew Lunn

Quoted 15 lines · Gregory CLEMENT

From: Yehuda Yitschak <[email protected]>

Signed-off-by Yehuda Yitschak <[email protected]>

Signed-off-by Gregory CLEMENT <[email protected]>

---
 arch/arm/boot/dts/armada-xp.dtsi        |    2 +-
 arch/arm/mach-mvebu/armada-370-xp.h     |   10 ++++
 arch/arm/mach-mvebu/irq-armada-370-xp.c |   92 +++++++++++++++++++++++++++++--
 3 files changed, 97 insertions(+), 7 deletions(-)

arch/arm/boot/dts/armada-xp.dtsi +0 -0

index f521ed8..531619f 100644

arch/arm/boot/dts/armada-xp.dtsi line 28

Quoted 6 lines

arch/arm/boot/dts/armada-xp.dtsi +1 -1

@@ -24,7 +24,7 @@
 
 	mpic: interrupt-controller at d0020000 {
 	      reg = <0xd0020a00 0x1d0>,
-		    <0xd0021870 0x58>;
+		    <0xd0021070 0x58>;
 	};

Hi Gregory

Is this a bug fix needed for 3.7?

Andrew

Quoted 4 lines

I don't think so.
We extended the reg map to be able to use registers only used for SMP. When we didn't have SMP support the mapping was correct, and now by introducing SMP we extend it.

The length has stayed the same, so its not extended. Its now 0x800 bytes earlier in the address space.

Andrew

The correct explanation is that the offset +21070 is a CPU virtual offset. That means that depending of the CPU core which will access to this register, the controller will internally change the offset automagically to point the correct offset.

I should have added an explanation in the commit log. I will do it for V2.

Quoted 6 lines · Gregory CLEMENT

The correct explanation is that the offset +21070 is a CPU virtual offset. That means that depending of the CPU core which will access to this register, the controller will internally change the offset automagically to point the correct offset.

I should have added an explanation in the commit log. I will do it for V2.

Just to expand on Gr??gory's comment: there is per-CPU banking for the interrupt controller registers. At 0x21070, you have "virtual" registers that automatically map to the interrupt controller registers of the current CPU. At 0x21870, you have the interrupt controllers registers of CPU0, regardless of which CPU you are running on.

Before this patch set, there was no SMP support for Armada 370/XP, so accessing the interrupt controller registers at 0x21870 was OK (accessing them from 0x21070 would have been OK as well). With the introduction of SMP support, accessing them from 0x21870 no longer works, so we switch to the virtual registers at 0x21070.

In other words: no it is not a bug fix and it therefore doesn't need to go into 3.7.

Thanks for the clear reply. Does the binding documentation make this clear? Should it be extended to indicate which address range should be used?

arch/arm/mach-mvebu/platsmp.c new file +30 -0

+void __init set_secondary_cpus_clock(void)
+{
+	int cpu;
+	unsigned long rate;
+	struct clk *cpu_clk = NULL;
+	struct device_node *np = NULL;
+
+	cpu = smp_processor_id();
+	np = of_find_node_by_type(np, "cpu");
+	np = NULL;
+	while ((np = of_find_node_by_type(np, "cpu"))) {
+		const u32 *reg;
+		int len;
+		reg = of_get_property(np, "reg", &len);
+		if (!reg || len != 4) {
+			pr_err("%s missing reg property\n", np->full_name);
+			continue;
+		}
+		if (be32_to_cpup(reg) == cpu) {
+			cpu_clk = of_clk_get(np, 0);
+			break;
+		}
+	}
+	WARN_ON(IS_ERR(cpu_clk));
+	rate = clk_get_rate(cpu_clk);
+
+	/* set all the other CPU clk to the same rate than the boot CPU */
+	np = NULL;
+	while ((np = of_find_node_by_type(np, "cpu"))) {
+		const u32 *reg;

Hi Gregory

Quoted 35 lines

arch/arm/mach-mvebu/platsmp.c new file +35 -0

+void __init set_secondary_cpus_clock(void)
+{
+	int cpu;
+	unsigned long rate;
+	struct clk *cpu_clk = NULL;
+	struct device_node *np = NULL;
+
+	cpu = smp_processor_id();
+	np = of_find_node_by_type(np, "cpu");
+	np = NULL;
+	while ((np = of_find_node_by_type(np, "cpu"))) {
+		const u32 *reg;
+		int len;
+		reg = of_get_property(np, "reg", &len);
+		if (!reg || len != 4) {
+			pr_err("%s missing reg property\n", np->full_name);
+			continue;
+		}
+		if (be32_to_cpup(reg) == cpu) {
+			cpu_clk = of_clk_get(np, 0);
+			break;
+		}
+	}
+	WARN_ON(IS_ERR(cpu_clk));
+	rate = clk_get_rate(cpu_clk);
+
+	/* set all the other CPU clk to the same rate than the boot CPU */
+	np = NULL;
+	while ((np = of_find_node_by_type(np, "cpu"))) {
+		const u32 *reg;
+		int len;
+		reg = of_get_property(np, "reg", &len);
+		if (!reg || len != 4) {
+			pr_err("%s missing reg property\n", np->full_name);
+			continue;

arch/arm/mach-mvebu/platsmp.c line 70

Quoted 5 lines

arch/arm/mach-mvebu/platsmp.c new file +5 -0

+		}
+		if (be32_to_cpup(reg) != cpu) {
+			cpu_clk = of_clk_get(np, 0);
+			clk_set_rate(cpu_clk, rate);
+		}

Maybe its hiding somewhere, but where is the clk_prepare_enable() for this cpu_clk clock?

Quoted 37 lines · Andrew Lunn

Hi Gregory

Quoted 35 lines

arch/arm/mach-mvebu/platsmp.c new file +35 -0

+void __init set_secondary_cpus_clock(void)
+{
+	int cpu;
+	unsigned long rate;
+	struct clk *cpu_clk = NULL;
+	struct device_node *np = NULL;
+
+	cpu = smp_processor_id();
+	np = of_find_node_by_type(np, "cpu");
+	np = NULL;
+	while ((np = of_find_node_by_type(np, "cpu"))) {
+		const u32 *reg;
+		int len;
+		reg = of_get_property(np, "reg", &len);
+		if (!reg || len != 4) {
+			pr_err("%s missing reg property\n", np->full_name);
+			continue;
+		}
+		if (be32_to_cpup(reg) == cpu) {
+			cpu_clk = of_clk_get(np, 0);
+			break;
+		}
+	}
+	WARN_ON(IS_ERR(cpu_clk));
+	rate = clk_get_rate(cpu_clk);
+
+	/* set all the other CPU clk to the same rate than the boot CPU */
+	np = NULL;
+	while ((np = of_find_node_by_type(np, "cpu"))) {
+		const u32 *reg;
+		int len;
+		reg = of_get_property(np, "reg", &len);
+		if (!reg || len != 4) {
+			pr_err("%s missing reg property\n", np->full_name);
+			continue;

Quoted 9 lines

arch/arm/mach-mvebu/platsmp.c line 70

Quoted 5 lines

arch/arm/mach-mvebu/platsmp.c new file +5 -0

+		}
+		if (be32_to_cpup(reg) != cpu) {
+			cpu_clk = of_clk_get(np, 0);
+			clk_set_rate(cpu_clk, rate);
+		}

Maybe its hiding somewhere, but where is the clk_prepare_enable() for this cpu_clk clock?

Well the clocks are always enable. In the clock framework, the cpu clocks don't provide an enable function.

But maybe it is cleaner to call clk_prepare_enable() even if it does nothing except update the usage count.

Quoted 48 lines · Gregory CLEMENT

Quoted 46 lines · Andrew Lunn

Hi Gregory

Quoted 30 lines

arch/arm/mach-mvebu/platsmp.c new file +30 -0

+void __init set_secondary_cpus_clock(void)
+{
+	int cpu;
+	unsigned long rate;
+	struct clk *cpu_clk = NULL;
+	struct device_node *np = NULL;
+
+	cpu = smp_processor_id();
+	np = of_find_node_by_type(np, "cpu");
+	np = NULL;
+	while ((np = of_find_node_by_type(np, "cpu"))) {
+		const u32 *reg;
+		int len;
+		reg = of_get_property(np, "reg", &len);
+		if (!reg || len != 4) {
+			pr_err("%s missing reg property\n", np->full_name);
+			continue;
+		}
+		if (be32_to_cpup(reg) == cpu) {
+			cpu_clk = of_clk_get(np, 0);
+			break;
+		}
+	}
+	WARN_ON(IS_ERR(cpu_clk));
+	rate = clk_get_rate(cpu_clk);
+
+	/* set all the other CPU clk to the same rate than the boot CPU */
+	np = NULL;
+	while ((np = of_find_node_by_type(np, "cpu"))) {
+		const u32 *reg;

arch/arm/mach-mvebu/platsmp.c line 70

Quoted 10 lines

arch/arm/mach-mvebu/platsmp.c new file +10 -0

+		int len;
+		reg = of_get_property(np, "reg", &len);
+		if (!reg || len != 4) {
+			pr_err("%s missing reg property\n", np->full_name);
+			continue;
+		}
+		if (be32_to_cpup(reg) != cpu) {
+			cpu_clk = of_clk_get(np, 0);
+			clk_set_rate(cpu_clk, rate);
+		}

Maybe its hiding somewhere, but where is the clk_prepare_enable() for this cpu_clk clock?

Quoted 2 lines

Well the clocks are always enable. In the clock framework, the cpu clocks don't provide an enable function.

Is it possible in the hardware to disable them? Turning them off might save some power. If these chips end up in some data center server, it might be that CPU hotplugging is used.

Quoted 3 lines

But maybe it is cleaner to call clk_prepare_enable() even if it does nothing except update the usage count.

You also have to watch out for the late_initcall which will disable all clocks which are running but nobody has claimed. This will become an issue if you do have enable/disable function in later versions of the clock functions.

Andrew

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel