ARM: mvebu: add suspend to RAM support for Armada 38x

Hello,

This is the v2 of the patch series adding suspend to RAM support for Armada 38x.

Since v1 (posted June 16th) :

- Rebased on top of v4.2-rc1, so dropped the patch "ARM: mvebu: fix suspend to RAM on big-endian configurations" which was merged.

- Added Acked-by from Gregory Clement on the relevant patches.

- Fixed the Armada 388 GP board name in the "model" property of the DT, as suggested by Gregory Clement.

Thanks,

Thomas

Nadav Haklai (1):
ARM: mvebu: prepare set_cpu_coherent() for future extension

Thomas Petazzoni (7):
ARM: mvebu: do not check machine in mvebu_pm_init()
ARM: mvebu: prepare mvebu_pm_store_bootinfo() to support multiple SoCs
ARM: mvebu: prepare pm-board.c for the introduction of Armada 38x
support
ARM: mvebu: add suspend/resume support for Armada 38x
ARM: mvebu: add suspend/resume related definitions in Armada 38x
Device Tree
ARM: mvebu: add suspend/resume DT information for Armada 388 GP
ARM: mvebu: adjust board name and compatible for Armada 388 GP

 arch/arm/boot/dts/armada-388-gp.dts | 20 +++++++++--
 arch/arm/boot/dts/armada-38x.dtsi   |  7 +++-
 arch/arm/mach-mvebu/board-v7.c      |  1 +
 arch/arm/mach-mvebu/coherency.c     | 33 +++++++++++-------
 arch/arm/mach-mvebu/pm-board.c      | 25 +++++++-------
 arch/arm/mach-mvebu/pm.c            | 69 +++++++++++++++++++++++++++++++++----
 arch/arm/mach-mvebu/pmsu.h          |  1 +
 arch/arm/mach-mvebu/pmsu_ll.S       | 43 +++++++++++++++++++++++
 8 files changed, 165 insertions(+), 34 deletions(-)

2.4.5

From: Nadav Haklai <[email protected]>

This patch prepares the set_cpu_coherent() function in coherency.c to be extended to support other SoCs than Armada XP. It will be needed on Armada 38x to re-enable the coherency after exiting from suspend to RAM.

This preparation simply moves the function further down in coherency.c so that it can use coherency_type(), and uses that function to only do the Armada XP specific work if we are on Armada XP.

---
 arch/arm/mach-mvebu/coherency.c | 29 +++++++++++++++++------------
 1 file changed, 17 insertions(+), 12 deletions(-)

index e46e9ea..44eedf3 100644
@@ -65,18 +65,6 @@ static const struct of_device_id of_coherency_table[] = {
 int ll_enable_coherency(void);
 void ll_add_cpu_to_smp_group(void);
 
-int set_cpu_coherent(void)
-{
-	if (!coherency_base) {
-		pr_warn("Can't make current CPU cache coherent.\n");
-		pr_warn("Coherency fabric is not initialized\n");
-		return 1;
-	}
-
-	ll_add_cpu_to_smp_group();
-	return ll_enable_coherency();
-}
-
 static int mvebu_hwcc_notifier(struct notifier_block *nb,
 			       unsigned long event, void *__dev)
 {

@@ -206,6 +194,23 @@ static int coherency_type(void)
 	return type;
 }
 
+int set_cpu_coherent(void)
+{
+	int type = coherency_type();
+
+	if (type == COHERENCY_FABRIC_TYPE_ARMADA_370_XP) {
+		if (!coherency_base) {
+			pr_warn("Can't make current CPU cache coherent.\n");
+			pr_warn("Coherency fabric is not initialized\n");
+			return 1;
+		}
+		ll_add_cpu_to_smp_group();
+		return ll_enable_coherency();
+	}
+
+	return 0;
+}
+
 int coherency_available(void)
 {
 	return coherency_type() != COHERENCY_FABRIC_TYPE_NONE;

2.4.5

The mvebu_pm_init() initializes the support for suspend/resume, and before doing that, it checks if we are on a board on which suspend/resume is actually supported. However, this check is already done by mvebu_armada_xp_gp_pm_init(), and there is no need to duplicate the check: callers of mvebu_pm_init() should now what they are doing.

This commit is done in preparation to the addition of suspend/resume support on Armada 38x.

---
 arch/arm/mach-mvebu/pm.c | 3 ---
 1 file changed, 3 deletions(-)

index 6573a8f..eca650b 100644
@@ -182,9 +182,6 @@ int mvebu_pm_init(void (*board_pm_enter)(void __iomem *sdram_reg, u32 srcmd))
 	struct device_node *np;
 	struct resource res;
 
-	if (!of_machine_is_compatible("marvell,armadaxp"))
-		return -ENODEV;
-
 	np = of_find_compatible_node(NULL, NULL,
 				     "marvell,armada-xp-sdram-controller");
 	if (!np)

2.4.5

As we are going to introduce support for Armada 38x in pm.c, split out the Armada XP part of mvebu_pm_store_bootinfo() into mvebu_pm_store_armadaxp_bootinfo(), and make the former retunr an error when an unsupported SoC is used.

---
 arch/arm/mach-mvebu/pm.c | 25 +++++++++++++++++++++----
 1 file changed, 21 insertions(+), 4 deletions(-)

index eca650b..02fdf67 100644
@@ -105,12 +105,10 @@ static phys_addr_t mvebu_internal_reg_base(void)
 	return of_translate_address(np, in_addr);
 }
 
-static void mvebu_pm_store_bootinfo(void)
+static void mvebu_pm_store_armadaxp_bootinfo(u32 *store_addr)
 {
-	u32 *store_addr;
 	phys_addr_t resume_pc;
 
-	store_addr = phys_to_virt(BOOT_INFO_ADDR);
 	resume_pc = virt_to_phys(armada_370_xp_cpu_resume);
 
 	/*

@@ -151,14 +149,33 @@ static void mvebu_pm_store_bootinfo(void)
 	writel(BOOT_MAGIC_LIST_END, store_addr);
 }
 
+static int mvebu_pm_store_bootinfo(void)
+{
+	u32 *store_addr;
+
+	store_addr = phys_to_virt(BOOT_INFO_ADDR);
+
+	if (of_machine_is_compatible("marvell,armadaxp"))
+		mvebu_pm_store_armadaxp_bootinfo(store_addr);
+	else
+		return -ENODEV;
+
+	return 0;
+}
+
 static int mvebu_pm_enter(suspend_state_t state)
 {
+	int ret;
+
 	if (state != PM_SUSPEND_MEM)
 		return -EINVAL;
 
+	ret = mvebu_pm_store_bootinfo();
+	if (ret)
+		return ret;
+
 	cpu_pm_enter();
 
-	mvebu_pm_store_bootinfo();
 	cpu_suspend(0, mvebu_pm_powerdown);
 
 	outer_resume();

2.4.5

The pm-board.c code contains the board-specific logic to enter suspend to RAM. Until now, the code supported only the Armada XP GP board, so all functions and symbols were named with armada_xp_gp. However, it turns out that the Armada 388 GP also uses the same 3 GPIOs protocol to talk to the PIC microcontroller that controls the power supply.

Since we are going to re-use the same code with no change for Armada 38x, this commit renames the functions and symbols to use just "armada" instead of "armada_xp_gp". Better names can be found if one day other boards having a different protocol/mechanism are supported in the kernel.

---
 arch/arm/mach-mvebu/pm-board.c | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

index 301ab38..b8c26cb 100644
@@ -1,7 +1,7 @@
 /*
  * Board-level suspend/resume support.
  *
- * Copyright (C) 2014 Marvell
+ * Copyright (C) 2014-2015 Marvell
  *
  * Thomas Petazzoni <[email protected]>
  *

@@ -20,27 +20,27 @@
 #include <linux/slab.h>
 #include "common.h"
 
-#define ARMADA_XP_GP_PIC_NR_GPIOS 3
+#define ARMADA_PIC_NR_GPIOS 3
 
 static void __iomem *gpio_ctrl;
-static int pic_gpios[ARMADA_XP_GP_PIC_NR_GPIOS];
-static int pic_raw_gpios[ARMADA_XP_GP_PIC_NR_GPIOS];
+static int pic_gpios[ARMADA_PIC_NR_GPIOS];
+static int pic_raw_gpios[ARMADA_PIC_NR_GPIOS];
 
-static void mvebu_armada_xp_gp_pm_enter(void __iomem *sdram_reg, u32 srcmd)
+static void mvebu_armada_pm_enter(void __iomem *sdram_reg, u32 srcmd)
 {
 	u32 reg, ackcmd;
 	int i;
 
 	/* Put 001 as value on the GPIOs */
 	reg = readl(gpio_ctrl);
-	for (i = 0; i < ARMADA_XP_GP_PIC_NR_GPIOS; i++)
+	for (i = 0; i < ARMADA_PIC_NR_GPIOS; i++)
 		reg &= ~BIT(pic_raw_gpios[i]);
 	reg |= BIT(pic_raw_gpios[0]);
 	writel(reg, gpio_ctrl);
 
 	/* Prepare writing 111 to the GPIOs */
 	ackcmd = readl(gpio_ctrl);
-	for (i = 0; i < ARMADA_XP_GP_PIC_NR_GPIOS; i++)
+	for (i = 0; i < ARMADA_PIC_NR_GPIOS; i++)
 		ackcmd |= BIT(pic_raw_gpios[i]);
 
 	srcmd = cpu_to_le32(srcmd);

@@ -76,7 +76,7 @@ static void mvebu_armada_xp_gp_pm_enter(void __iomem *sdram_reg, u32 srcmd)
 		  [ackcmd] "r" (ackcmd), [gpio_ctrl] "r" (gpio_ctrl) : "r1");
 }
 
-static int mvebu_armada_xp_gp_pm_init(void)
+static int mvebu_armada_pm_init(void)
 {
 	struct device_node *np;
 	struct device_node *gpio_ctrl_np;

@@ -89,7 +89,7 @@ static int mvebu_armada_xp_gp_pm_init(void)
 	if (!np)
 		return -ENODEV;
 
-	for (i = 0; i < ARMADA_XP_GP_PIC_NR_GPIOS; i++) {
+	for (i = 0; i < ARMADA_PIC_NR_GPIOS; i++) {
 		char *name;
 		struct of_phandle_args args;
 

@@ -134,11 +134,11 @@ static int mvebu_armada_xp_gp_pm_init(void)
 	if (!gpio_ctrl)
 		return -ENOMEM;
 
-	mvebu_pm_init(mvebu_armada_xp_gp_pm_enter);
+	mvebu_pm_init(mvebu_armada_pm_enter);
 
 out:
 	of_node_put(np);
 	return ret;
 }
 
-late_initcall(mvebu_armada_xp_gp_pm_init);
+late_initcall(mvebu_armada_pm_init);

2.4.5

This commit adds support for suspend/resume on Armada 38x, and specifically for the Armada 388 GP board (since on Marvell EBU systems, suspend/resume requires board-level specific details). In details, the needed changes are:

- Register mvebu_memblock_reserve() as the ->reserve() callback in DT_MACHINE_START. This is needed to make sure that the small portions of RAM used by the bootloader to do the DDR3 training are not used by the kernel, since this training is done again when existing from suspend to RAM.

- Add support for Armada 38x in set_cpu_coherent() by enabling the SCU. This will make sure the SCU gets re-enabled after existing from suspend to RAM.

- Add marvell,a388-gp to the list of supported boards in the board-specific code pm-board.c. No other changes are needed since the Armada 388 GP uses a 3 GPIOs protocol with the PIC micro-controller, like the one used on Armada XP GP.

- Add mvebu_pm_store_armada38x_bootinfo() in pm.c to prepare the entry to suspend to RAM by creating the boot information structure expected by the bootloader.

- Add the assembly code in pmsu_ll.S used when returning from suspend to RAM.

---
 arch/arm/mach-mvebu/board-v7.c  |  1 +
 arch/arm/mach-mvebu/coherency.c |  4 ++++
 arch/arm/mach-mvebu/pm-board.c  |  3 ++-
 arch/arm/mach-mvebu/pm.c        | 41 +++++++++++++++++++++++++++++++++++++++
 arch/arm/mach-mvebu/pmsu.h      |  1 +
 arch/arm/mach-mvebu/pmsu_ll.S   | 43 +++++++++++++++++++++++++++++++++++++++++
 6 files changed, 92 insertions(+), 1 deletion(-)

index afee908..e5911de 100644
@@ -230,6 +230,7 @@ DT_MACHINE_START(ARMADA_38X_DT, "Marvell Armada 380/385 (Device Tree)")
 	.l2c_aux_mask	= ~0,
 	.init_irq       = mvebu_init_irq,
 	.restart	= mvebu_restart,
+	.reserve        = mvebu_memblock_reserve,
 	.dt_compat	= armada_38x_dt_compat,
 MACHINE_END
 

index 44eedf3..caf4769 100644
@@ -31,9 +31,11 @@
 #include <linux/mbus.h>
 #include <linux/pci.h>
 #include <asm/smp_plat.h>
+#include <asm/smp_scu.h>
 #include <asm/cacheflush.h>
 #include <asm/mach/map.h>
 #include <asm/dma-mapping.h>
+#include "common.h"
 #include "coherency.h"
 #include "mvebu-soc-id.h"
 

@@ -206,6 +208,8 @@ int set_cpu_coherent(void)
 		}
 		ll_add_cpu_to_smp_group();
 		return ll_enable_coherency();
+	} else if (type == COHERENCY_FABRIC_TYPE_ARMADA_380) {
+		scu_enable(mvebu_get_scu_base());
 	}
 
 	return 0;

index b8c26cb..0f471fa 100644
@@ -82,7 +82,8 @@ static int mvebu_armada_pm_init(void)
 	struct device_node *gpio_ctrl_np;
 	int ret = 0, i;
 
-	if (!of_machine_is_compatible("marvell,axp-gp"))
+	if (!of_machine_is_compatible("marvell,axp-gp") &&
+	    !of_machine_is_compatible("marvell,a388-gp"))
 		return -ENODEV;
 
 	np = of_find_node_by_name(NULL, "pm_pic");

index ea79269..9166e94 100644
@@ -18,6 +18,7 @@ int mvebu_setup_boot_addr_wa(unsigned int crypto_eng_target,
 
 void mvebu_v7_pmsu_idle_exit(void);
 void armada_370_xp_cpu_resume(void);
+void armada_38x_mem_resume(void);
 
 int armada_370_xp_pmsu_idle_enter(unsigned long deepidle);
 int armada_38x_do_cpu_suspend(unsigned long deepidle);

2.4.5

In order to support suspend/resume, the SoC-level Armada 38x Device Tree file must be improved with the following additions:

- Description of the SDRAM controller registers, which are needed in order to allow the suspend/resume code to put the RAM in self-refresh. This is similar to what was done in commit 6e6db2bea3ea ("ARM: mvebu: add SDRAM controller description for Armada XP") for Armada XP.

- Description of additional registers for the MBus controller, which need to be saved/restored accross a suspend/resume cycle. This is similar to what was done in commit 8b7dc9d37a44e ("ARM: mvebu:
adjust mbus controller description on Armada 370/XP") for Armada XP.

---
 arch/arm/boot/dts/armada-38x.dtsi | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

index 04ecfe6..60adb44 100644
@@ -138,6 +138,11 @@
 			#size-cells = <1>;
 			ranges = <0 MBUS_ID(0xf0, 0x01) 0 0x100000>;
 
+			sdramc at 1400 {
+				compatible = "marvell,armada-xp-sdram-controller";
+				reg = <0x1400 0x500>;
+			};
+
 			L2: cache-controller at 8000 {
 				compatible = "arm,pl310-cache";
 				reg = <0x8000 0x1000>;

@@ -371,7 +376,7 @@
 
 			mbusc: mbus-controller at 20000 {
 				compatible = "marvell,mbus-controller";
-				reg = <0x20000 0x100>, <0x20180 0x20>;
+				reg = <0x20000 0x100>, <0x20180 0x20>, <0x20250 0x8>;
 			};
 
 			mpic: interrupt-controller at 20a00 {

2.4.5

This commit improves the Armada 388 GP Device Tree description to describe the 3 GPIOs that are used to connect the SoC to the PIC micro-controller that we talk to shutdown the SoC when entering suspend to RAM.

---
 arch/arm/boot/dts/armada-388-gp.dts | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

index fd4f6fd..31e5ede 100644
@@ -56,6 +56,14 @@
 		reg = <0x00000000 0x80000000>; /* 2 GB */
 	};
 
+	cpus {
+		pm_pic {
+			ctrl-gpios = <&gpio1 1 GPIO_ACTIVE_LOW>,
+				     <&gpio1 2 GPIO_ACTIVE_LOW>,
+				     <&gpio1 3 GPIO_ACTIVE_LOW>;
+		};
+	};
+
 	soc {
 		ranges = <MBUS_ID(0xf0, 0x01) 0 0xf1000000 0x100000
 			  MBUS_ID(0x01, 0x1d) 0 0xfff00000 0x100000>;

@@ -406,8 +414,16 @@
 };
 
 &pinctrl {
+	pinctrl-0 = <&pic_pins>;
+	pinctrl-names = "default";
+
 	pca0_pins: pca0_pins {
 		marvell,pins = "mpp18";
 		marvell,function = "gpio";
 	};
+
+	pic_pins: pic-pins-0 {
+		marvell,pins = "mpp33", "mpp34", "mpp35";
+		marvell,function = "gpio";
+	};
 };

2.4.5

As the name of the Device Tree file name suggests, the Armada 388 GP really contains an Armada 388 SoC, so this commit updates the board name and compatible string in the Device Tree file.

---
 arch/arm/boot/dts/armada-388-gp.dts | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

index 31e5ede..2f6c31f 100644
@@ -44,8 +44,8 @@
 #include <dt-bindings/gpio/gpio.h>
 
 / {
-	model = "Marvell Armada 385 GP";
-	compatible = "marvell,a385-gp", "marvell,armada388", "marvell,armada380";
+	model = "Marvell Armada 388 DB-88F6820-GP";
+	compatible = "marvell,a388-gp", "marvell,armada388", "marvell,armada380";
 
 	chosen {
 		stdout-path = "serial0:115200n8";

2.4.5

Hi Thomas,

patch 1 to 4 applied on mvebu/soc

Thanks,

Gregory

Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com